2023-11-18T20:31:13.520 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-18T20:31:13.520 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} --verbose --owner scheduled_yuriw@teuthology --name yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690/orig.config.yaml 2023-11-18T20:31:13.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-18T20:31:13.650 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} email: yweinste@redhat.com first_in_suite: false job_id: '7461690' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1491 sha1: b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP/jp6CjnY96YaqBnfvGbW0BxolDYOlg13z8a86sb9oBZzGAkKns6lahWcxSXV8W7JKoP3jW9dkMLLOaAzH47nc= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfh9p0ioMlgydSxkwEIUrNH/H9D8ZcH6UBaugbWq0YsmL7fhRTHJJ9GD2Z6oh0HuGRq1O9TFZp+IzykXO+C/J8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-18_13:51:50 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-18T20:31:13.650 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/qa; will attempt to use it 2023-11-18T20:31:13.651 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/qa/tasks 2023-11-18T20:31:13.651 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-18T20:31:13.652 INFO:teuthology.task.internal:Checking packages... 2023-11-18T20:31:13.675 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe' 2023-11-18T20:31:13.675 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-18T20:31:13.675 INFO:teuthology.packaging:ref: None 2023-11-18T20:31:13.676 INFO:teuthology.packaging:tag: None 2023-11-18T20:31:13.676 INFO:teuthology.packaging:branch: main 2023-11-18T20:31:13.676 INFO:teuthology.packaging:sha1: b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe 2023-11-18T20:31:13.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2023-11-18T20:31:13.932 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7424.g6adc03ac 2023-11-18T20:31:13.933 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-18T20:31:13.940 INFO:teuthology.task.internal:no buildpackages task found 2023-11-18T20:31:13.940 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-18T20:31:13.990 INFO:teuthology.task.internal:Saving configuration 2023-11-18T20:31:14.005 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-18T20:31:14.018 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-18T20:31:14.044 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-18 20:26:17.545141', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP/jp6CjnY96YaqBnfvGbW0BxolDYOlg13z8a86sb9oBZzGAkKns6lahWcxSXV8W7JKoP3jW9dkMLLOaAzH47nc='} 2023-11-18T20:31:14.068 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-18 20:26:17.546923', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfh9p0ioMlgydSxkwEIUrNH/H9D8ZcH6UBaugbWq0YsmL7fhRTHJJ9GD2Z6oh0HuGRq1O9TFZp+IzykXO+C/J8='} 2023-11-18T20:31:14.069 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-18T20:31:14.075 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-18T20:31:14.075 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-18T20:31:14.076 INFO:teuthology.run_tasks:Running task console_log... 2023-11-18T20:31:14.187 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0b501ef0d0>, signals=[15]) 2023-11-18T20:31:14.187 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-18T20:31:14.223 INFO:teuthology.task.internal:Opening connections... 2023-11-18T20:31:14.224 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:31:14.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-18T20:31:14.326 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:31:14.327 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-18T20:31:14.411 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-18T20:31:14.417 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2023-11-18T20:31:14.436 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2023-11-18T20:31:14.437 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2023-11-18T20:31:14.494 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2023-11-18T20:31:14.494 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2023-11-18T20:31:14.494 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2023-11-18T20:31:14.494 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2023-11-18T20:31:14.495 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2023-11-18T20:31:14.495 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2023-11-18T20:31:14.495 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-18T20:31:14.495 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2023-11-18T20:31:14.495 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2023-11-18T20:31:14.495 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-18T20:31:14.495 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2023-11-18T20:31:14.495 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-18T20:31:14.495 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-18T20:31:14.495 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-18T20:31:14.496 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2023-11-18T20:31:14.516 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2023-11-18T20:31:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2023-11-18T20:31:14.533 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2023-11-18T20:31:14.590 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2023-11-18T20:31:14.594 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2023-11-18T20:31:14.594 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2023-11-18T20:31:14.594 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2023-11-18T20:31:14.594 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2023-11-18T20:31:14.594 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2023-11-18T20:31:14.594 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-18T20:31:14.594 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2023-11-18T20:31:14.595 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2023-11-18T20:31:14.595 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-18T20:31:14.595 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2023-11-18T20:31:14.595 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-18T20:31:14.595 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-18T20:31:14.595 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-18T20:31:14.595 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2023-11-18T20:31:14.616 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-18T20:31:14.626 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-18T20:31:14.656 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-18T20:31:14.656 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2023-11-18T20:31:14.659 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2023-11-18T20:31:14.675 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-18T20:31:14.687 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-18T20:31:14.687 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2023-11-18T20:31:14.716 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2023-11-18T20:31:14.802 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-18T20:31:14.855 INFO:teuthology.run_tasks:Running task kernel... 2023-11-18T20:31:14.868 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-11-18T20:31:14.868 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-11-18T20:31:14.869 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-18T20:31:14.869 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-18T20:31:14.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:31:14.886 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2023-11-18T20:31:14.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:31:14.887 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2023-11-18T20:31:14.944 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-361.el9.x86_64 2023-11-18T20:31:14.944 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-361.el9.x86_64 2023-11-18T20:31:14.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2023-11-18T20:31:14.945 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-361.el9.x86_64 2023-11-18T20:31:14.946 INFO:teuthology.task.kernel:Running kernel on smithi060: 5.14.0-361.el9.x86_64 2023-11-18T20:31:14.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum install -y kernel 2023-11-18T20:31:16.971 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 17 MB/s | 6.4 MB 00:00 2023-11-18T20:31:17.211 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - AppStream 11 MB/s | 6.4 MB 00:00 2023-11-18T20:31:18.288 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.0 MB 00:00 2023-11-18T20:31:18.603 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2023-11-18T20:31:19.568 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 7.9 MB 00:00 2023-11-18T20:31:19.953 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 7.9 MB 00:01 2023-11-18T20:31:22.729 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-11-18T20:31:27.488 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 5.8 MB 00:01 2023-11-18T20:31:28.889 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 15 kB 00:00 2023-11-18T20:31:35.460 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 18 MB 00:14 2023-11-18T20:31:39.794 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 5.8 MB 00:00 2023-11-18T20:31:40.180 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 1.8 MB/s | 20 MB 00:11 2023-11-18T20:31:41.145 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 15 kB 00:00 2023-11-18T20:31:42.746 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2023-11-18T20:31:44.914 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2023-11-18T20:31:47.245 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-11-18T20:31:47.245 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-11-18T20:31:47.335 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2023-11-18T20:31:47.339 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2023-11-18T20:31:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2023-11-18T20:31:47.340 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2023-11-18T20:31:47.340 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2023-11-18T20:31:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-384.el9 CentOS-BaseOS 5.5 M 2023-11-18T20:31:47.340 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2023-11-18T20:31:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 5.14.0-384.el9 CentOS-BaseOS 20 M 2023-11-18T20:31:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 5.14.0-384.el9 CentOS-BaseOS 38 M 2023-11-18T20:31:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core x86_64 5.14.0-384.el9 CentOS-BaseOS 32 M 2023-11-18T20:31:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:31:47.340 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2023-11-18T20:31:47.341 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2023-11-18T20:31:47.341 INFO:teuthology.orchestra.run.smithi100.stdout:Install 4 Packages 2023-11-18T20:31:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:31:47.341 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 95 M 2023-11-18T20:31:47.341 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 121 M 2023-11-18T20:31:47.342 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2023-11-18T20:31:47.582 INFO:teuthology.orchestra.run.smithi100.stdout:(1/4): kernel-5.14.0-384.el9.x86_64.rpm 23 MB/s | 5.5 MB 00:00 2023-11-18T20:31:47.680 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2023-11-18T20:31:47.832 INFO:teuthology.orchestra.run.smithi100.stdout:(2/4): kernel-core-5.14.0-384.el9.x86_64.rpm 41 MB/s | 20 MB 00:00 2023-11-18T20:31:48.224 INFO:teuthology.orchestra.run.smithi100.stdout:(3/4): kernel-modules-5.14.0-384.el9.x86_64.rpm 43 MB/s | 38 MB 00:00 2023-11-18T20:31:48.541 INFO:teuthology.orchestra.run.smithi100.stdout:(4/4): kernel-modules-core-5.14.0-384.el9.x86_6 34 MB/s | 32 MB 00:00 2023-11-18T20:31:48.544 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2023-11-18T20:31:48.544 INFO:teuthology.orchestra.run.smithi100.stdout:Total 79 MB/s | 95 MB 00:01 2023-11-18T20:31:48.544 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2023-11-18T20:31:48.908 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2023-11-18T20:31:48.908 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2023-11-18T20:31:49.935 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2023-11-18T20:31:49.935 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2023-11-18T20:31:50.136 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-11-18T20:31:50.137 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-11-18T20:31:50.220 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-11-18T20:31:50.233 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-11-18T20:31:50.234 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-11-18T20:31:50.234 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-11-18T20:31:50.234 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-11-18T20:31:50.234 INFO:teuthology.orchestra.run.smithi060.stdout: kernel x86_64 5.14.0-384.el9 CentOS-BaseOS 5.5 M 2023-11-18T20:31:50.234 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-11-18T20:31:50.234 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core x86_64 5.14.0-384.el9 CentOS-BaseOS 20 M 2023-11-18T20:31:50.235 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules x86_64 5.14.0-384.el9 CentOS-BaseOS 38 M 2023-11-18T20:31:50.235 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core x86_64 5.14.0-384.el9 CentOS-BaseOS 32 M 2023-11-18T20:31:50.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:31:50.235 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-11-18T20:31:50.235 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-11-18T20:31:50.235 INFO:teuthology.orchestra.run.smithi060.stdout:Install 4 Packages 2023-11-18T20:31:50.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:31:50.236 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 95 M 2023-11-18T20:31:50.236 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 121 M 2023-11-18T20:31:50.236 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-11-18T20:31:50.464 INFO:teuthology.orchestra.run.smithi060.stdout:(1/4): kernel-5.14.0-384.el9.x86_64.rpm 24 MB/s | 5.5 MB 00:00 2023-11-18T20:31:50.723 INFO:teuthology.orchestra.run.smithi060.stdout:(2/4): kernel-core-5.14.0-384.el9.x86_64.rpm 41 MB/s | 20 MB 00:00 2023-11-18T20:31:51.181 INFO:teuthology.orchestra.run.smithi060.stdout:(3/4): kernel-modules-5.14.0-384.el9.x86_64.rpm 40 MB/s | 38 MB 00:00 2023-11-18T20:31:51.481 INFO:teuthology.orchestra.run.smithi060.stdout:(4/4): kernel-modules-core-5.14.0-384.el9.x86_6 32 MB/s | 32 MB 00:01 2023-11-18T20:31:51.485 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-11-18T20:31:51.485 INFO:teuthology.orchestra.run.smithi060.stdout:Total 76 MB/s | 95 MB 00:01 2023-11-18T20:31:51.485 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-11-18T20:31:51.930 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-11-18T20:31:51.930 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-11-18T20:31:52.341 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2023-11-18T20:31:52.959 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-11-18T20:31:52.959 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-11-18T20:31:53.046 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-core-5.14.0-384.el9.x86_64 1/4 2023-11-18T20:31:53.191 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-5.14.0-384.el9.x86_64 2/4 2023-11-18T20:31:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-11-18T20:31:55.931 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-384.el9.x86_64 2/4 2023-11-18T20:31:56.138 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-core-5.14.0-384.el9.x86_64 1/4 2023-11-18T20:31:56.171 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-5.14.0-384.el9.x86_64 3/4 2023-11-18T20:31:56.274 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-core-5.14.0-384.el9.x86_64 2/4 2023-11-18T20:31:59.151 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-384.el9.x86_64 2/4 2023-11-18T20:31:59.329 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-5.14.0-384.el9.x86_64 3/4 2023-11-18T20:32:00.742 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-384.el9.x86_64 3/4 2023-11-18T20:32:00.964 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-384.el9.x86_64 3/4 2023-11-18T20:32:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:05.592 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-core-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:09.334 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-core-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:34.637 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:34.680 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-384.el9.x86_64 1/4 2023-11-18T20:32:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-5.14.0-384.el9.x86_64 2/4 2023-11-18T20:32:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-5.14.0-384.el9.x86_64 3/4 2023-11-18T20:32:35.460 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-core-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:35.461 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:32:35.461 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2023-11-18T20:32:35.461 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-384.el9.x86_64 2023-11-18T20:32:35.461 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core-5.14.0-384.el9.x86_64 2023-11-18T20:32:35.461 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-5.14.0-384.el9.x86_64 2023-11-18T20:32:35.461 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core-5.14.0-384.el9.x86_64 2023-11-18T20:32:35.461 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:32:35.461 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2023-11-18T20:32:35.939 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2023-11-18T20:32:36.075 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-384.el9.x86_64 2023-11-18T20:32:36.076 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-384.el9.x86_64 2023-11-18T20:32:36.076 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-384.el9.x86_64 2023-11-18T20:32:36.076 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-18T20:32:36.076 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-18T20:32:36.076 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-11-18T20:32:36.076 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2023-11-18T20:32:36.077 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2023-11-18T20:32:36.182 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-384.el9.x86_64 2023-11-18T20:32:36.182 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-384.el9.x86_64 2023-11-18T20:32:36.182 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2023-11-18T20:32:36.226 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2023-11-18T20:32:36.226 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2023-11-18T20:32:36.226 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2023-11-18T20:32:36.226 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 68.el9 2023-11-18T20:32:36.226 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2023-11-18T20:32:36.226 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-18T20:32:36.226 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8264254 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2023-11-18T20:32:36.227 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:32:36.228 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-18T20:32:36.228 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-18T20:32:36.228 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-18T20:32:36.228 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2023-11-18T20:32:36.228 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:32:36.228 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2023-11-18T20:32:36.229 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-18T20:32:36.229 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 5.14.0-384.el9.x86_64 2023-11-18T20:32:36.229 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-18T20:32:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2023-11-18T20:32:38.286 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-18T20:32:38.319 INFO:teuthology.orchestra.run.smithi100.stderr:done 2023-11-18T20:32:38.321 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2023-11-18T20:32:38.337 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.2HvqtOOLLU 2023-11-18T20:32:38.337 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2HvqtOOLLU 2023-11-18T20:32:38.405 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.2HvqtOOLLU 2023-11-18T20:32:38.544 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.2HvqtOOLLU is 6KB 2023-11-18T20:32:38.593 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.2HvqtOOLLU 2023-11-18T20:32:38.607 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2023-11-18T20:32:38.672 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-18T20:32:38.672 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-18T20:32:38.672 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-18T20:32:38.672 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-384.el9.x86_64.conf 2023-11-18T20:32:38.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-384.el9.x86_64 2023-11-18T20:32:38.828 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2023-11-18T20:32:39.762 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:40.239 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-5.14.0-384.el9.x86_64 1/4 2023-11-18T20:32:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-core-5.14.0-384.el9.x86_64 2/4 2023-11-18T20:32:40.240 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-5.14.0-384.el9.x86_64 3/4 2023-11-18T20:32:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-core-5.14.0-384.el9.x86_64 4/4 2023-11-18T20:32:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:32:40.678 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-11-18T20:32:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-5.14.0-384.el9.x86_64 2023-11-18T20:32:40.678 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core-5.14.0-384.el9.x86_64 2023-11-18T20:32:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-5.14.0-384.el9.x86_64 2023-11-18T20:32:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core-5.14.0-384.el9.x86_64 2023-11-18T20:32:40.679 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:32:40.679 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-11-18T20:32:41.158 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2023-11-18T20:32:41.285 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-384.el9.x86_64 2023-11-18T20:32:41.286 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-384.el9.x86_64 2023-11-18T20:32:41.286 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-384.el9.x86_64 2023-11-18T20:32:41.286 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-18T20:32:41.286 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-18T20:32:41.286 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-11-18T20:32:41.286 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=None, version=distro) 2023-11-18T20:32:41.287 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2023-11-18T20:32:41.397 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-384.el9.x86_64 2023-11-18T20:32:41.398 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-384.el9.x86_64 2023-11-18T20:32:41.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2023-11-18T20:32:41.436 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2023-11-18T20:32:41.436 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2023-11-18T20:32:41.436 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.06 2023-11-18T20:32:41.436 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 68.el9 2023-11-18T20:32:41.436 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2023-11-18T20:32:41.436 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:Group : Unspecified 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 8264254 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : builder@centos.org 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2023-11-18T20:32:41.437 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:32:41.438 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-18T20:32:41.438 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-18T20:32:41.438 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-18T20:32:41.438 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2023-11-18T20:32:41.438 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:32:41.438 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2023-11-18T20:32:41.439 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-18T20:32:41.439 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 5.14.0-384.el9.x86_64 2023-11-18T20:32:41.439 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-18T20:32:42.055 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2023-11-18T20:32:43.538 INFO:teuthology.orchestra.run.smithi060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-18T20:32:43.558 INFO:teuthology.orchestra.run.smithi060.stderr:done 2023-11-18T20:32:43.560 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2023-11-18T20:32:43.575 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.7lwn456VTA 2023-11-18T20:32:43.576 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7lwn456VTA 2023-11-18T20:32:43.650 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.7lwn456VTA 2023-11-18T20:32:43.797 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.7lwn456VTA is 6KB 2023-11-18T20:32:43.848 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.7lwn456VTA 2023-11-18T20:32:43.863 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2023-11-18T20:32:43.931 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-18T20:32:43.931 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-18T20:32:43.931 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-18T20:32:43.931 INFO:teuthology.orchestra.run.smithi060.stdout:15949e3c88704300882482eddfd7baae-5.14.0-384.el9.x86_64.conf 2023-11-18T20:32:43.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-384.el9.x86_64 2023-11-18T20:32:44.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2023-11-18T20:33:08.854 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-18T20:33:08.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-11-18T20:33:08.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-18T20:33:14.095 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-18T20:33:14.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-11-18T20:33:14.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-18T20:33:43.523 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-11-18T20:33:52.526 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-11-18T20:33:52.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-18T20:34:10.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-11-18T20:34:14.100 DEBUG:teuthology.orchestra.remote:timed out 2023-11-18T20:34:22.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-11-18T20:34:22.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-18T20:34:23.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-11-18T20:34:23.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-18T20:34:23.305 DEBUG:teuthology.orchestra.run.smithi100:> true 2023-11-18T20:34:23.408 DEBUG:teuthology.orchestra.run.smithi060:> true 2023-11-18T20:34:23.702 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-11-18T20:34:23.702 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-384.el9.x86_64"... 2023-11-18T20:34:23.702 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2023-11-18T20:34:23.716 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-384.el9.x86_64 2023-11-18T20:34:23.717 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-384.el9.x86_64 vs 5.14.0-384.el9.x86_64 2023-11-18T20:34:23.717 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-18T20:34:23.717 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-18T20:34:23.850 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-11-18T20:34:23.850 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-384.el9.x86_64"... 2023-11-18T20:34:23.850 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2023-11-18T20:34:23.865 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-384.el9.x86_64 2023-11-18T20:34:23.865 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-384.el9.x86_64 vs 5.14.0-384.el9.x86_64 2023-11-18T20:34:23.865 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-18T20:34:23.865 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-18T20:34:24.717 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-18T20:34:24.718 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-18T20:34:24.787 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2023-11-18T20:34:24.824 DEBUG:teuthology.parallel:result is None 2023-11-18T20:34:24.866 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-18T20:34:24.867 DEBUG:teuthology.orchestra.run.smithi060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-18T20:34:24.932 INFO:teuthology.orchestra.run.smithi060.stdout:ttyS1 2023-11-18T20:34:24.972 DEBUG:teuthology.parallel:result is None 2023-11-18T20:34:24.972 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-18T20:34:24.979 INFO:teuthology.task.internal:Creating test directory... 2023-11-18T20:34:24.979 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-18T20:34:24.981 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-18T20:34:24.995 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-18T20:34:25.004 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-18T20:34:25.012 INFO:teuthology.task.internal:Creating archive directory... 2023-11-18T20:34:25.013 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-18T20:34:25.040 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-18T20:34:25.069 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-18T20:34:25.076 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-18T20:34:25.076 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-18T20:34:25.108 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-18T20:34:25.138 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-18T20:34:25.153 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-18T20:34:25.154 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-18T20:34:25.166 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-18T20:34:25.168 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-18T20:34:25.176 INFO:teuthology.task.internal:Configuring sudo... 2023-11-18T20:34:25.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-18T20:34:25.202 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-18T20:34:25.230 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-18T20:34:25.238 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-18T20:34:25.239 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-18T20:34:25.276 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-18T20:34:25.290 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-18T20:34:25.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-18T20:34:25.436 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-18T20:34:25.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-18T20:34:25.571 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:34:25.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-18T20:34:25.645 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-18T20:34:25.684 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-18T20:34:25.740 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-18T20:34:25.781 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-18T20:34:25.819 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:34:25.819 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-18T20:34:25.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2023-11-18T20:34:25.884 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2023-11-18T20:34:25.933 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-18T20:34:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-18T20:34:26.443 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-18T20:34:26.451 INFO:teuthology.task.internal:Starting timer... 2023-11-18T20:34:26.451 INFO:teuthology.run_tasks:Running task pcp... 2023-11-18T20:34:26.464 INFO:teuthology.run_tasks:Running task selinux... 2023-11-18T20:34:26.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2023-11-18T20:34:26.527 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2023-11-18T20:34:26.529 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2023-11-18T20:34:26.635 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2023-11-18T20:34:26.639 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-18T20:34:26.640 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2023-11-18T20:34:26.663 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2023-11-18T20:34:26.663 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2023-11-18T20:34:26.697 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2023-11-18T20:34:26.697 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive'} 2023-11-18T20:34:26.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-18T20:34:26.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:34:26.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-18T20:34:26.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:34:26.784 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-18T20:34:26.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2023-11-18T20:34:26.823 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2023-11-18T20:34:27.107 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-18T20:34:27.116 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-18T20:34:27.121 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-18T20:34:27.261 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-18T20:34:27.276 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-18T20:34:27.277 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi100.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-18T20:39:50.959 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com')] 2023-11-18T20:39:50.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-11-18T20:39:50.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-18T20:39:51.059 DEBUG:teuthology.orchestra.run.smithi060:> true 2023-11-18T20:39:51.126 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-11-18T20:39:51.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-11-18T20:39:51.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-18T20:39:51.210 DEBUG:teuthology.orchestra.run.smithi100:> true 2023-11-18T20:39:51.282 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-11-18T20:39:51.282 INFO:teuthology.run_tasks:Running task clock... 2023-11-18T20:39:51.298 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-18T20:39:51.298 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-18T20:39:51.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-18T20:39:51.301 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-18T20:39:51.302 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-18T20:39:51.334 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-18T20:39:51.354 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-18T20:39:51.354 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-18T20:39:51.372 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-18T20:39:51.390 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2023-11-18T20:39:51.402 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2023-11-18T20:39:51.407 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2023-11-18T20:39:51.416 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2023-11-18T20:39:51.426 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-18T20:39:51.432 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-18T20:39:51.446 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-18T20:39:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-18T20:39:51.492 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2023-11-18T20:39:51.493 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2023-11-18T20:39:51.495 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-18T20:39:51.495 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2023-11-18T20:39:51.495 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-18T20:39:51.496 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-18T20:39:51.496 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-18T20:39:51.496 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-18T20:39:51.497 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-18T20:39:51.497 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2023-11-18T20:39:51.497 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-18T20:39:51.498 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-18T20:39:51.498 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-18T20:39:51.498 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-18T20:39:51.498 INFO:teuthology.run_tasks:Running task install... 2023-11-18T20:39:51.506 DEBUG:teuthology.task.install:project ceph 2023-11-18T20:39:51.507 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe'}} 2023-11-18T20:39:51.507 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe'} 2023-11-18T20:39:51.507 INFO:teuthology.task.install:Using flavor: default 2023-11-18T20:39:51.512 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-18T20:39:51.513 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-18T20:39:51.513 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe', 'tag': None, 'wait_for_package': False} 2023-11-18T20:39:51.513 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe 2023-11-18T20:39:51.515 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe', 'tag': None, 'wait_for_package': False} 2023-11-18T20:39:51.515 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe 2023-11-18T20:39:51.737 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/centos/9/flavors/default/ 2023-11-18T20:39:51.737 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7418.gb0f45a51 2023-11-18T20:39:51.747 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/centos/9/flavors/default/ 2023-11-18T20:39:51.747 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7418.gb0f45a51 2023-11-18T20:39:51.855 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-18T20:39:51.856 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:39:51.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-18T20:39:51.885 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-18T20:39:51.885 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-18T20:39:51.905 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-18T20:39:51.905 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:39:51.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-18T20:39:51.938 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-18T20:39:51.938 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-18T20:39:51.952 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-18T20:39:52.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-18T20:39:52.031 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-18T20:39:52.098 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2023-11-18T20:39:52.099 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2023-11-18T20:39:52.100 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-18T20:39:52.130 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2023-11-18T20:39:52.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2023-11-18T20:39:52.339 INFO:teuthology.orchestra.run.smithi100.stdout:72 files removed 2023-11-18T20:39:52.366 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-18T20:39:52.379 INFO:teuthology.orchestra.run.smithi060.stdout:72 files removed 2023-11-18T20:39:52.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-18T20:39:53.153 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 191 kB/s | 78 kB 00:00 2023-11-18T20:39:53.158 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 164 kB/s | 78 kB 00:00 2023-11-18T20:39:53.525 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 30 kB/s | 9.5 kB 00:00 2023-11-18T20:39:53.578 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 25 kB/s | 9.5 kB 00:00 2023-11-18T20:39:53.859 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 6.3 kB/s | 1.9 kB 00:00 2023-11-18T20:39:53.937 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2023-11-18T20:39:54.323 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - AppStream 18 MB/s | 6.4 MB 00:00 2023-11-18T20:39:54.378 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 13 MB/s | 6.4 MB 00:00 2023-11-18T20:39:55.687 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 7.9 MB/s | 2.0 MB 00:00 2023-11-18T20:39:55.732 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2023-11-18T20:39:56.739 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 7.9 MB 00:00 2023-11-18T20:39:58.244 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 7.9 MB 00:02 2023-11-18T20:40:00.004 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 29 MB/s | 18 MB 00:00 2023-11-18T20:40:04.230 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 5.8 MB 00:00 2023-11-18T20:40:06.225 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 15 kB 00:00 2023-11-18T20:40:11.698 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 18 MB 00:13 2023-11-18T20:40:15.982 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 5.8 MB 00:00 2023-11-18T20:40:17.400 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 15 kB 00:00 2023-11-18T20:40:18.754 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 16 MB/s | 20 MB 00:01 2023-11-18T20:40:23.610 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2023-11-18T20:40:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 20 MB 00:19 2023-11-18T20:40:26.388 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-18T20:40:26.389 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-18T20:40:26.400 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-18T20:40:26.401 INFO:teuthology.orchestra.run.smithi060.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-18T20:40:26.402 INFO:teuthology.orchestra.run.smithi060.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-18T20:40:26.403 INFO:teuthology.orchestra.run.smithi060.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-18T20:40:26.405 INFO:teuthology.orchestra.run.smithi060.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-18T20:40:26.407 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-18T20:40:26.408 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-18T20:40:26.485 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-11-18T20:40:26.511 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================ 2023-11-18T20:40:26.511 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-11-18T20:40:26.511 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================ 2023-11-18T20:40:26.511 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-11-18T20:40:26.511 INFO:teuthology.orchestra.run.smithi060.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-18T20:40:26.511 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 6.5 k 2023-11-18T20:40:26.511 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 5.2 M 2023-11-18T20:40:26.511 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 860 k 2023-11-18T20:40:26.511 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 141 k 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 1.5 M 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 133 k 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 1.8 M 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 7.4 M 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 50 k 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 8.6 M 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 41 M 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 262 k 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 774 k 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 217 k 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 18 k 2023-11-18T20:40:26.512 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 31 k 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 718 k 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 127 k 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel x86_64 1:3.0.7-24.el9 CentOS-AppStream 3.0 M 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-18T20:40:26.513 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 162 k 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 321 k 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 299 k 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 99 k 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 85 k 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 3.0 M 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 170 k 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-18T20:40:26.514 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 3.3 M 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 3.0 M 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-11-18T20:40:26.515 INFO:teuthology.orchestra.run.smithi060.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 19 M 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 23 k 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 2.1 M 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 243 k 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 4.4 M 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 18 M 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 15 k 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 24 k 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-18T20:40:26.516 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 163 k 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-18T20:40:26.517 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 475 k 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 4.8 M 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-18T20:40:26.518 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 45 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 129 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-18T20:40:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-18T20:40:26.520 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-18T20:40:26.521 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-18T20:40:26.522 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout:Installing weak dependencies: 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-18T20:40:26.523 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:40:26.524 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-11-18T20:40:26.525 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================ 2023-11-18T20:40:26.525 INFO:teuthology.orchestra.run.smithi060.stdout:Install 132 Packages 2023-11-18T20:40:26.525 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 13 Packages 2023-11-18T20:40:26.525 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:40:26.525 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 200 M 2023-11-18T20:40:26.525 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-11-18T20:40:27.061 INFO:teuthology.orchestra.run.smithi060.stdout:(1/145): ceph-18.0.0-7418.gb0f45a51.el9.x86_64. 30 kB/s | 6.5 kB 00:00 2023-11-18T20:40:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:(2/145): ceph-fuse-18.0.0-7418.gb0f45a51.el9.x8 3.1 MB/s | 860 kB 00:00 2023-11-18T20:40:27.411 INFO:teuthology.orchestra.run.smithi060.stdout:(3/145): ceph-immutable-object-cache-18.0.0-741 1.7 MB/s | 141 kB 00:00 2023-11-18T20:40:27.711 INFO:teuthology.orchestra.run.smithi060.stdout:(4/145): ceph-base-18.0.0-7418.gb0f45a51.el9.x8 6.0 MB/s | 5.2 MB 00:00 2023-11-18T20:40:27.754 INFO:teuthology.orchestra.run.smithi060.stdout:(5/145): ceph-mds-18.0.0-7418.gb0f45a51.el9.x86 6.1 MB/s | 2.1 MB 00:00 2023-11-18T20:40:27.929 INFO:teuthology.orchestra.run.smithi060.stdout:(6/145): ceph-mgr-18.0.0-7418.gb0f45a51.el9.x86 6.7 MB/s | 1.5 MB 00:00 2023-11-18T20:40:28.404 INFO:teuthology.orchestra.run.smithi060.stdout:(7/145): ceph-mon-18.0.0-7418.gb0f45a51.el9.x86 6.8 MB/s | 4.4 MB 00:00 2023-11-18T20:40:28.963 INFO:teuthology.orchestra.run.smithi060.stdout:(8/145): ceph-common-18.0.0-7418.gb0f45a51.el9. 9.0 MB/s | 19 MB 00:02 2023-11-18T20:40:29.054 INFO:teuthology.orchestra.run.smithi060.stdout:(9/145): ceph-selinux-18.0.0-7418.gb0f45a51.el9 263 kB/s | 24 kB 00:00 2023-11-18T20:40:29.205 INFO:teuthology.orchestra.run.smithi060.stdout:(10/145): ceph-radosgw-18.0.0-7418.gb0f45a51.el 11 MB/s | 8.6 MB 00:00 2023-11-18T20:40:29.297 INFO:teuthology.orchestra.run.smithi060.stdout:(11/145): cephfs-mirror-18.0.0-7418.gb0f45a51.e 2.3 MB/s | 217 kB 00:00 2023-11-18T20:40:29.381 INFO:teuthology.orchestra.run.smithi060.stdout:(12/145): libcephfs-devel-18.0.0-7418.gb0f45a51 374 kB/s | 31 kB 00:00 2023-11-18T20:40:29.489 INFO:teuthology.orchestra.run.smithi060.stdout:(13/145): libcephfs2-18.0.0-7418.gb0f45a51.el9. 6.5 MB/s | 718 kB 00:00 2023-11-18T20:40:29.706 INFO:teuthology.orchestra.run.smithi060.stdout:(14/145): libcephsqlite-18.0.0-7418.gb0f45a51.e 753 kB/s | 163 kB 00:00 2023-11-18T20:40:29.882 INFO:teuthology.orchestra.run.smithi060.stdout:(15/145): ceph-osd-18.0.0-7418.gb0f45a51.el9.x8 9.1 MB/s | 18 MB 00:01 2023-11-18T20:40:29.907 INFO:teuthology.orchestra.run.smithi060.stdout:(16/145): librados-devel-18.0.0-7418.gb0f45a51. 632 kB/s | 127 kB 00:00 2023-11-18T20:40:29.949 INFO:teuthology.orchestra.run.smithi060.stdout:(17/145): libradosstriper1-18.0.0-7418.gb0f45a5 7.0 MB/s | 475 kB 00:00 2023-11-18T20:40:30.049 INFO:teuthology.orchestra.run.smithi060.stdout:(18/145): python3-ceph-argparse-18.0.0-7418.gb0 453 kB/s | 45 kB 00:00 2023-11-18T20:40:30.158 INFO:teuthology.orchestra.run.smithi060.stdout:(19/145): python3-ceph-common-18.0.0-7418.gb0f4 1.2 MB/s | 129 kB 00:00 2023-11-18T20:40:30.299 INFO:teuthology.orchestra.run.smithi060.stdout:(20/145): librgw2-18.0.0-7418.gb0f45a51.el9.x86 12 MB/s | 4.8 MB 00:00 2023-11-18T20:40:30.304 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2023-11-18T20:40:30.325 INFO:teuthology.orchestra.run.smithi060.stdout:(21/145): python3-cephfs-18.0.0-7418.gb0f45a51. 969 kB/s | 162 kB 00:00 2023-11-18T20:40:30.375 INFO:teuthology.orchestra.run.smithi060.stdout:(22/145): python3-rados-18.0.0-7418.gb0f45a51.e 4.2 MB/s | 321 kB 00:00 2023-11-18T20:40:30.425 INFO:teuthology.orchestra.run.smithi060.stdout:(23/145): python3-rbd-18.0.0-7418.gb0f45a51.el9 2.9 MB/s | 299 kB 00:00 2023-11-18T20:40:30.450 INFO:teuthology.orchestra.run.smithi060.stdout:(24/145): python3-rgw-18.0.0-7418.gb0f45a51.el9 1.3 MB/s | 99 kB 00:00 2023-11-18T20:40:30.501 INFO:teuthology.orchestra.run.smithi060.stdout:(25/145): rbd-fuse-18.0.0-7418.gb0f45a51.el9.x8 1.1 MB/s | 85 kB 00:00 2023-11-18T20:40:30.618 INFO:teuthology.orchestra.run.smithi060.stdout:(26/145): rbd-nbd-18.0.0-7418.gb0f45a51.el9.x86 1.4 MB/s | 170 kB 00:00 2023-11-18T20:40:30.718 INFO:teuthology.orchestra.run.smithi060.stdout:(27/145): rbd-mirror-18.0.0-7418.gb0f45a51.el9. 11 MB/s | 3.0 MB 00:00 2023-11-18T20:40:30.743 INFO:teuthology.orchestra.run.smithi060.stdout:(28/145): ceph-grafana-dashboards-18.0.0-7418.g 186 kB/s | 23 kB 00:00 2023-11-18T20:40:30.785 INFO:teuthology.orchestra.run.smithi060.stdout:(29/145): ceph-mgr-cephadm-18.0.0-7418.gb0f45a5 1.9 MB/s | 133 kB 00:00 2023-11-18T20:40:31.036 INFO:teuthology.orchestra.run.smithi060.stdout:(30/145): ceph-mgr-dashboard-18.0.0-7418.gb0f45 6.0 MB/s | 1.8 MB 00:00 2023-11-18T20:40:31.144 INFO:teuthology.orchestra.run.smithi060.stdout:(31/145): ceph-mgr-modules-core-18.0.0-7418.gb0 2.2 MB/s | 243 kB 00:00 2023-11-18T20:40:31.253 INFO:teuthology.orchestra.run.smithi060.stdout:(32/145): ceph-mgr-rook-18.0.0-7418.gb0f45a51.e 463 kB/s | 50 kB 00:00 2023-11-18T20:40:31.353 INFO:teuthology.orchestra.run.smithi060.stdout:(33/145): ceph-prometheus-alerts-18.0.0-7418.gb 151 kB/s | 15 kB 00:00 2023-11-18T20:40:31.445 INFO:teuthology.orchestra.run.smithi060.stdout:(34/145): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-11-18T20:40:31.470 INFO:teuthology.orchestra.run.smithi060.stdout:(35/145): ceph-volume-18.0.0-7418.gb0f45a51.el9 2.2 MB/s | 262 kB 00:00 2023-11-18T20:40:31.562 INFO:teuthology.orchestra.run.smithi060.stdout:(36/145): cephadm-18.0.0-7418.gb0f45a51.el9.noa 6.5 MB/s | 774 kB 00:00 2023-11-18T20:40:31.587 INFO:teuthology.orchestra.run.smithi060.stdout:(37/145): cephfs-top-18.0.0-7418.gb0f45a51.el9. 156 kB/s | 18 kB 00:00 2023-11-18T20:40:31.730 INFO:teuthology.orchestra.run.smithi060.stdout:(38/145): bison-3.7.4-5.el9.x86_64.rpm 6.3 MB/s | 920 kB 00:00 2023-11-18T20:40:31.755 INFO:teuthology.orchestra.run.smithi060.stdout:(39/145): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2023-11-18T20:40:31.780 INFO:teuthology.orchestra.run.smithi060.stdout:(40/145): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2023-11-18T20:40:31.805 INFO:teuthology.orchestra.run.smithi060.stdout:(41/145): elfutils-libelf-devel-0.190-1.el9.x86 429 kB/s | 21 kB 00:00 2023-11-18T20:40:31.831 INFO:teuthology.orchestra.run.smithi060.stdout:(42/145): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2023-11-18T20:40:31.856 INFO:teuthology.orchestra.run.smithi060.stdout:(43/145): flexiblas-3.0.4-8.el9.x86_64.rpm 631 kB/s | 32 kB 00:00 2023-11-18T20:40:31.931 INFO:teuthology.orchestra.run.smithi060.stdout:(44/145): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2023-11-18T20:40:31.948 INFO:teuthology.orchestra.run.smithi060.stdout:(45/145): flexiblas-openblas-openmp-3.0.4-8.el9 189 kB/s | 17 kB 00:00 2023-11-18T20:40:32.316 INFO:teuthology.orchestra.run.smithi060.stdout:(46/145): ceph-test-18.0.0-7418.gb0f45a51.el9.x 13 MB/s | 41 MB 00:03 2023-11-18T20:40:32.332 INFO:teuthology.orchestra.run.smithi060.stdout:(47/145): libaio-devel-0.3.111-13.el9.x86_64.rp 26 kB/s | 10 kB 00:00 2023-11-18T20:40:32.358 INFO:teuthology.orchestra.run.smithi060.stdout:(48/145): jq-1.6-16.el9.x86_64.rpm 455 kB/s | 186 kB 00:00 2023-11-18T20:40:32.383 INFO:teuthology.orchestra.run.smithi060.stdout:(49/145): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2023-11-18T20:40:32.400 INFO:teuthology.orchestra.run.smithi060.stdout:(50/145): librabbitmq-0.11.0-7.el9.x86_64.rpm 666 kB/s | 44 kB 00:00 2023-11-18T20:40:32.433 INFO:teuthology.orchestra.run.smithi060.stdout:(51/145): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2023-11-18T20:40:32.459 INFO:teuthology.orchestra.run.smithi060.stdout:(52/145): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 578 kB 00:00 2023-11-18T20:40:32.484 INFO:teuthology.orchestra.run.smithi060.stdout:(53/145): libuuid-devel-2.37.4-15.el9.x86_64.rp 293 kB/s | 24 kB 00:00 2023-11-18T20:40:32.509 INFO:teuthology.orchestra.run.smithi060.stdout:(54/145): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2023-11-18T20:40:32.534 INFO:teuthology.orchestra.run.smithi060.stdout:(55/145): libzstd-devel-1.5.1-2.el9.x86_64.rpm 615 kB/s | 46 kB 00:00 2023-11-18T20:40:32.568 INFO:teuthology.orchestra.run.smithi060.stdout:(56/145): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2023-11-18T20:40:32.593 INFO:teuthology.orchestra.run.smithi060.stdout:(57/145): openblas-0.3.21-2.el9.x86_64.rpm 411 kB/s | 34 kB 00:00 2023-11-18T20:40:32.661 INFO:teuthology.orchestra.run.smithi060.stdout:(58/145): openssl-devel-3.0.7-24.el9.x86_64.rpm 33 MB/s | 3.0 MB 00:00 2023-11-18T20:40:32.744 INFO:teuthology.orchestra.run.smithi060.stdout:(59/145): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2023-11-18T20:40:32.777 INFO:teuthology.orchestra.run.smithi060.stdout:(60/145): postgresql-13.11-1.el9.x86_64.rpm 8.3 MB/s | 1.5 MB 00:00 2023-11-18T20:40:32.819 INFO:teuthology.orchestra.run.smithi060.stdout:(61/145): postgresql-contrib-13.11-1.el9.x86_64 5.1 MB/s | 811 kB 00:00 2023-11-18T20:40:32.844 INFO:teuthology.orchestra.run.smithi060.stdout:(62/145): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2023-11-18T20:40:32.920 INFO:teuthology.orchestra.run.smithi060.stdout:(63/145): postgresql-server-13.11-1.el9.x86_64. 40 MB/s | 5.7 MB 00:00 2023-11-18T20:40:33.020 INFO:teuthology.orchestra.run.smithi060.stdout:(64/145): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2023-11-18T20:40:33.045 INFO:teuthology.orchestra.run.smithi060.stdout:(65/145): python3-jinja2-2.11.3-4.el9.noarch.rp 1.1 MB/s | 228 kB 00:00 2023-11-18T20:40:33.070 INFO:teuthology.orchestra.run.smithi060.stdout:(66/145): python3-jsonpatch-1.21-16.el9.noarch. 166 kB/s | 24 kB 00:00 2023-11-18T20:40:33.082 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-18T20:40:33.083 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-18T20:40:33.094 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-18T20:40:33.094 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-18T20:40:33.095 INFO:teuthology.orchestra.run.smithi060.stdout:(67/145): python3-jsonpointer-2.0-4.el9.noarch. 242 kB/s | 18 kB 00:00 2023-11-18T20:40:33.096 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-18T20:40:33.096 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-18T20:40:33.098 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-18T20:40:33.100 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-18T20:40:33.101 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-18T20:40:33.121 INFO:teuthology.orchestra.run.smithi060.stdout:(68/145): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2023-11-18T20:40:33.146 INFO:teuthology.orchestra.run.smithi060.stdout:(69/145): python3-markupsafe-1.1.1-12.el9.x86_6 420 kB/s | 32 kB 00:00 2023-11-18T20:40:33.172 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2023-11-18T20:40:33.190 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2023-11-18T20:40:33.190 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2023-11-18T20:40:33.190 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2023-11-18T20:40:33.190 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2023-11-18T20:40:33.190 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-18T20:40:33.190 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 6.5 k 2023-11-18T20:40:33.190 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 5.2 M 2023-11-18T20:40:33.190 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 860 k 2023-11-18T20:40:33.190 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 141 k 2023-11-18T20:40:33.190 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 1.5 M 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 133 k 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 1.8 M 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 7.4 M 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 50 k 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 8.6 M 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 41 M 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 262 k 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 774 k 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 217 k 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 18 k 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-18T20:40:33.191 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 31 k 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 718 k 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 127 k 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.0.7-24.el9 CentOS-AppStream 3.0 M 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 162 k 2023-11-18T20:40:33.192 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 321 k 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 299 k 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 99 k 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 85 k 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 3.0 M 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 170 k 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-18T20:40:33.193 INFO:teuthology.orchestra.run.smithi100.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 3.3 M 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 3.0 M 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-18T20:40:33.194 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 19 M 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 23 k 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 2.1 M 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 243 k 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 4.4 M 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 18 M 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7418.gb0f45a51.el9 ceph-noarch 15 k 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 24 k 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-18T20:40:33.195 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 163 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 475 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-18T20:40:33.196 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 4.8 M 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-18T20:40:33.197 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 45 k 2023-11-18T20:40:33.198 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:18.0.0-7418.gb0f45a51.el9 ceph 129 k 2023-11-18T20:40:33.198 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-18T20:40:33.198 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-18T20:40:33.198 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-18T20:40:33.198 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-18T20:40:33.198 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-18T20:40:33.198 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-18T20:40:33.198 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-18T20:40:33.198 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-18T20:40:33.199 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-18T20:40:33.200 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-18T20:40:33.201 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-18T20:40:33.202 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:40:33.203 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2023-11-18T20:40:33.203 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2023-11-18T20:40:33.203 INFO:teuthology.orchestra.run.smithi100.stdout:Install 132 Packages 2023-11-18T20:40:33.203 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 13 Packages 2023-11-18T20:40:33.203 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:40:33.203 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 200 M 2023-11-18T20:40:33.203 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2023-11-18T20:40:33.238 INFO:teuthology.orchestra.run.smithi060.stdout:(70/145): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2023-11-18T20:40:33.263 INFO:teuthology.orchestra.run.smithi060.stdout:(71/145): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.9 MB/s | 217 kB 00:00 2023-11-18T20:40:33.289 INFO:teuthology.orchestra.run.smithi060.stdout:(72/145): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2023-11-18T20:40:33.313 INFO:teuthology.orchestra.run.smithi060.stdout:(73/145): python3-prettytable-0.7.2-27.el9.noar 546 kB/s | 41 kB 00:00 2023-11-18T20:40:33.339 INFO:teuthology.orchestra.run.smithi060.stdout:(74/145): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2023-11-18T20:40:33.364 INFO:teuthology.orchestra.run.smithi060.stdout:(75/145): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2023-11-18T20:40:33.389 INFO:teuthology.orchestra.run.smithi060.stdout:(76/145): python3-pytz-2021.1-5.el9.noarch.rpm 628 kB/s | 47 kB 00:00 2023-11-18T20:40:33.415 INFO:teuthology.orchestra.run.smithi060.stdout:(77/145): python3-requests-oauthlib-1.3.0-12.el 851 kB/s | 43 kB 00:00 2023-11-18T20:40:33.490 INFO:teuthology.orchestra.run.smithi060.stdout:(78/145): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 39 kB 00:00 2023-11-18T20:40:33.682 INFO:teuthology.orchestra.run.smithi060.stdout:(79/145): python3-scipy-1.6.2-8.el9.x86_64.rpm 49 MB/s | 16 MB 00:00 2023-11-18T20:40:33.705 INFO:teuthology.orchestra.run.smithi060.stdout:(80/145): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2023-11-18T20:40:33.722 INFO:teuthology.orchestra.run.smithi100.stdout:(1/145): ceph-18.0.0-7418.gb0f45a51.el9.x86_64. 41 kB/s | 6.5 kB 00:00 2023-11-18T20:40:33.731 INFO:teuthology.orchestra.run.smithi060.stdout:(81/145): uuid-1.6.2-55.el9.x86_64.rpm 237 kB/s | 56 kB 00:00 2023-11-18T20:40:33.756 INFO:teuthology.orchestra.run.smithi060.stdout:(82/145): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.1 MB/s | 85 kB 00:00 2023-11-18T20:40:33.781 INFO:teuthology.orchestra.run.smithi060.stdout:(83/145): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-11-18T20:40:33.807 INFO:teuthology.orchestra.run.smithi060.stdout:(84/145): zlib-devel-1.2.11-40.el9.x86_64.rpm 584 kB/s | 44 kB 00:00 2023-11-18T20:40:33.832 INFO:teuthology.orchestra.run.smithi060.stdout:(85/145): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2023-11-18T20:40:33.865 INFO:teuthology.orchestra.run.smithi060.stdout:(86/145): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2023-11-18T20:40:33.899 INFO:teuthology.orchestra.run.smithi060.stdout:(87/145): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2023-11-18T20:40:33.906 INFO:teuthology.orchestra.run.smithi100.stdout:(2/145): ceph-fuse-18.0.0-7418.gb0f45a51.el9.x8 4.6 MB/s | 860 kB 00:00 2023-11-18T20:40:33.924 INFO:teuthology.orchestra.run.smithi060.stdout:(88/145): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2023-11-18T20:40:33.956 INFO:teuthology.orchestra.run.smithi100.stdout:(3/145): ceph-immutable-object-cache-18.0.0-741 2.8 MB/s | 141 kB 00:00 2023-11-18T20:40:33.958 INFO:teuthology.orchestra.run.smithi060.stdout:(89/145): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2023-11-18T20:40:33.983 INFO:teuthology.orchestra.run.smithi060.stdout:(90/145): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2023-11-18T20:40:34.009 INFO:teuthology.orchestra.run.smithi060.stdout:(91/145): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2023-11-18T20:40:34.034 INFO:teuthology.orchestra.run.smithi060.stdout:(92/145): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-11-18T20:40:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:(93/145): python3-urllib3-1.26.5-4.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2023-11-18T20:40:34.223 INFO:teuthology.orchestra.run.smithi100.stdout:(4/145): ceph-mds-18.0.0-7418.gb0f45a51.el9.x86 7.8 MB/s | 2.1 MB 00:00 2023-11-18T20:40:34.251 INFO:teuthology.orchestra.run.smithi060.stdout:(94/145): python3-devel-3.9.17-2.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2023-11-18T20:40:34.448 INFO:teuthology.orchestra.run.smithi100.stdout:(5/145): ceph-mgr-18.0.0-7418.gb0f45a51.el9.x86 6.5 MB/s | 1.5 MB 00:00 2023-11-18T20:40:34.451 INFO:teuthology.orchestra.run.smithi060.stdout:(95/145): gperftools-libs-2.9.1-2.el9.x86_64.rp 741 kB/s | 309 kB 00:00 2023-11-18T20:40:34.477 INFO:teuthology.orchestra.run.smithi060.stdout:(96/145): libarrow-doc-9.0.0-9.el9.noarch.rpm 108 kB/s | 24 kB 00:00 2023-11-18T20:40:34.519 INFO:teuthology.orchestra.run.smithi060.stdout:(97/145): liboath-2.6.7-2.el9.x86_64.rpm 726 kB/s | 49 kB 00:00 2023-11-18T20:40:34.602 INFO:teuthology.orchestra.run.smithi060.stdout:(98/145): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2023-11-18T20:40:34.727 INFO:teuthology.orchestra.run.smithi060.stdout:(99/145): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.9 MB/s | 837 kB 00:00 2023-11-18T20:40:34.828 INFO:teuthology.orchestra.run.smithi060.stdout:(100/145): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2023-11-18T20:40:34.861 INFO:teuthology.orchestra.run.smithi060.stdout:(101/145): python3-asyncssh-2.13.2-3.el9.noarch 2.1 MB/s | 548 kB 00:00 2023-11-18T20:40:34.886 INFO:teuthology.orchestra.run.smithi060.stdout:(102/145): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2023-11-18T20:40:34.932 INFO:teuthology.orchestra.run.smithi100.stdout:(6/145): ceph-mon-18.0.0-7418.gb0f45a51.el9.x86 9.1 MB/s | 4.4 MB 00:00 2023-11-18T20:40:34.945 INFO:teuthology.orchestra.run.smithi060.stdout:(103/145): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2023-11-18T20:40:34.970 INFO:teuthology.orchestra.run.smithi060.stdout:(104/145): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2023-11-18T20:40:34.998 INFO:teuthology.orchestra.run.smithi100.stdout:(7/145): ceph-base-18.0.0-7418.gb0f45a51.el9.x8 3.6 MB/s | 5.2 MB 00:01 2023-11-18T20:40:35.046 INFO:teuthology.orchestra.run.smithi060.stdout:(105/145): libarrow-9.0.0-9.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2023-11-18T20:40:35.071 INFO:teuthology.orchestra.run.smithi060.stdout:(106/145): python3-google-auth-2.23.4-1.el9.noa 1.9 MB/s | 197 kB 00:00 2023-11-18T20:40:35.096 INFO:teuthology.orchestra.run.smithi060.stdout:(107/145): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2023-11-18T20:40:35.122 INFO:teuthology.orchestra.run.smithi060.stdout:(108/145): python3-jaraco-8.2.1-3.el9.noarch.rp 141 kB/s | 11 kB 00:00 2023-11-18T20:40:35.147 INFO:teuthology.orchestra.run.smithi060.stdout:(109/145): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2023-11-18T20:40:35.172 INFO:teuthology.orchestra.run.smithi060.stdout:(110/145): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2023-11-18T20:40:35.191 INFO:teuthology.orchestra.run.smithi100.stdout:(8/145): ceph-common-18.0.0-7418.gb0f45a51.el9. 12 MB/s | 19 MB 00:01 2023-11-18T20:40:35.197 INFO:teuthology.orchestra.run.smithi060.stdout:(111/145): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2023-11-18T20:40:35.214 INFO:teuthology.orchestra.run.smithi060.stdout:(112/145): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2023-11-18T20:40:35.224 INFO:teuthology.orchestra.run.smithi100.stdout:(9/145): ceph-selinux-18.0.0-7418.gb0f45a51.el9 726 kB/s | 24 kB 00:00 2023-11-18T20:40:35.248 INFO:teuthology.orchestra.run.smithi060.stdout:(113/145): python3-jwt+crypto-2.4.0-1.el9.noarc 120 kB/s | 9.0 kB 00:00 2023-11-18T20:40:35.273 INFO:teuthology.orchestra.run.smithi060.stdout:(114/145): python3-jwt-2.4.0-1.el9.noarch.rpm 543 kB/s | 41 kB 00:00 2023-11-18T20:40:35.332 INFO:teuthology.orchestra.run.smithi060.stdout:(115/145): python3-logutils-0.3.5-21.el9.noarch 553 kB/s | 46 kB 00:00 2023-11-18T20:40:35.357 INFO:teuthology.orchestra.run.smithi060.stdout:(116/145): python3-more-itertools-8.12.0-2.el9. 943 kB/s | 79 kB 00:00 2023-11-18T20:40:35.391 INFO:teuthology.orchestra.run.smithi060.stdout:(117/145): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2023-11-18T20:40:35.416 INFO:teuthology.orchestra.run.smithi060.stdout:(118/145): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2023-11-18T20:40:35.450 INFO:teuthology.orchestra.run.smithi060.stdout:(119/145): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2023-11-18T20:40:35.475 INFO:teuthology.orchestra.run.smithi060.stdout:(120/145): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2023-11-18T20:40:35.500 INFO:teuthology.orchestra.run.smithi060.stdout:(121/145): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2023-11-18T20:40:35.525 INFO:teuthology.orchestra.run.smithi060.stdout:(122/145): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2023-11-18T20:40:35.550 INFO:teuthology.orchestra.run.smithi060.stdout:(123/145): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2023-11-18T20:40:35.576 INFO:teuthology.orchestra.run.smithi060.stdout:(124/145): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2023-11-18T20:40:35.601 INFO:teuthology.orchestra.run.smithi060.stdout:(125/145): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2023-11-18T20:40:35.643 INFO:teuthology.orchestra.run.smithi060.stdout:(126/145): python3-typing-extensions-4.4.0-2.el 552 kB/s | 51 kB 00:00 2023-11-18T20:40:35.668 INFO:teuthology.orchestra.run.smithi060.stdout:(127/145): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2023-11-18T20:40:35.702 INFO:teuthology.orchestra.run.smithi060.stdout:(128/145): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2023-11-18T20:40:35.728 INFO:teuthology.orchestra.run.smithi060.stdout:(129/145): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2023-11-18T20:40:35.761 INFO:teuthology.orchestra.run.smithi060.stdout:(130/145): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2023-11-18T20:40:35.828 INFO:teuthology.orchestra.run.smithi060.stdout:(131/145): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2023-11-18T20:40:36.037 INFO:teuthology.orchestra.run.smithi060.stdout:(132/145): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2023-11-18T20:40:36.070 INFO:teuthology.orchestra.run.smithi060.stdout:(133/145): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-11-18T20:40:36.145 INFO:teuthology.orchestra.run.smithi060.stdout:(134/145): NetworkManager-1.45.6-1.el9.x86_64.r 30 MB/s | 2.2 MB 00:00 2023-11-18T20:40:36.204 INFO:teuthology.orchestra.run.smithi060.stdout:(135/145): NetworkManager-libnm-1.45.6-1.el9.x8 30 MB/s | 1.8 MB 00:00 2023-11-18T20:40:36.241 INFO:teuthology.orchestra.run.smithi060.stdout:(136/145): NetworkManager-team-1.45.6-1.el9.x86 1.1 MB/s | 42 kB 00:00 2023-11-18T20:40:36.275 INFO:teuthology.orchestra.run.smithi060.stdout:(137/145): NetworkManager-tui-1.45.6-1.el9.x86_ 7.2 MB/s | 245 kB 00:00 2023-11-18T20:40:36.299 INFO:teuthology.orchestra.run.smithi100.stdout:(10/145): ceph-radosgw-18.0.0-7418.gb0f45a51.el 6.6 MB/s | 8.6 MB 00:01 2023-11-18T20:40:36.308 INFO:teuthology.orchestra.run.smithi060.stdout:(138/145): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2023-11-18T20:40:36.342 INFO:teuthology.orchestra.run.smithi060.stdout:(139/145): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-11-18T20:40:36.376 INFO:teuthology.orchestra.run.smithi060.stdout:(140/145): elfutils-libelf-0.190-1.el9.x86_64.r 5.6 MB/s | 192 kB 00:00 2023-11-18T20:40:36.400 INFO:teuthology.orchestra.run.smithi100.stdout:(11/145): cephfs-mirror-18.0.0-7418.gb0f45a51.e 2.1 MB/s | 217 kB 00:00 2023-11-18T20:40:36.409 INFO:teuthology.orchestra.run.smithi060.stdout:(141/145): elfutils-libs-0.190-1.el9.x86_64.rpm 7.5 MB/s | 255 kB 00:00 2023-11-18T20:40:36.443 INFO:teuthology.orchestra.run.smithi060.stdout:(142/145): libacl-2.3.1-4.el9.x86_64.rpm 658 kB/s | 22 kB 00:00 2023-11-18T20:40:36.476 INFO:teuthology.orchestra.run.smithi060.stdout:(143/145): xfsdump-3.1.12-4.el9.x86_64.rpm 9.4 MB/s | 321 kB 00:00 2023-11-18T20:40:36.483 INFO:teuthology.orchestra.run.smithi100.stdout:(12/145): libcephfs-devel-18.0.0-7418.gb0f45a51 374 kB/s | 31 kB 00:00 2023-11-18T20:40:36.725 INFO:teuthology.orchestra.run.smithi100.stdout:(13/145): libcephfs2-18.0.0-7418.gb0f45a51.el9. 2.9 MB/s | 718 kB 00:00 2023-11-18T20:40:36.735 INFO:teuthology.orchestra.run.smithi060.stdout:(144/145): librados2-18.0.0-7418.gb0f45a51.el9. 3.4 MB/s | 3.3 MB 00:00 2023-11-18T20:40:36.850 INFO:teuthology.orchestra.run.smithi100.stdout:(14/145): libcephsqlite-18.0.0-7418.gb0f45a51.e 1.3 MB/s | 163 kB 00:00 2023-11-18T20:40:37.076 INFO:teuthology.orchestra.run.smithi100.stdout:(15/145): ceph-osd-18.0.0-7418.gb0f45a51.el9.x8 8.3 MB/s | 18 MB 00:02 2023-11-18T20:40:37.101 INFO:teuthology.orchestra.run.smithi100.stdout:(16/145): librados-devel-18.0.0-7418.gb0f45a51. 506 kB/s | 127 kB 00:00 2023-11-18T20:40:37.168 INFO:teuthology.orchestra.run.smithi100.stdout:(17/145): libradosstriper1-18.0.0-7418.gb0f45a5 5.1 MB/s | 475 kB 00:00 2023-11-18T20:40:37.177 INFO:teuthology.orchestra.run.smithi060.stdout:(145/145): librbd1-18.0.0-7418.gb0f45a51.el9.x8 2.2 MB/s | 3.0 MB 00:01 2023-11-18T20:40:37.181 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-11-18T20:40:37.181 INFO:teuthology.orchestra.run.smithi060.stdout:Total 19 MB/s | 200 MB 00:10 2023-11-18T20:40:37.202 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-11-18T20:40:37.226 INFO:teuthology.orchestra.run.smithi100.stdout:(18/145): python3-ceph-argparse-18.0.0-7418.gb0 777 kB/s | 45 kB 00:00 2023-11-18T20:40:37.285 INFO:teuthology.orchestra.run.smithi100.stdout:(19/145): python3-ceph-common-18.0.0-7418.gb0f4 2.2 MB/s | 129 kB 00:00 2023-11-18T20:40:37.344 INFO:teuthology.orchestra.run.smithi100.stdout:(20/145): python3-cephfs-18.0.0-7418.gb0f45a51. 2.7 MB/s | 162 kB 00:00 2023-11-18T20:40:37.397 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-11-18T20:40:37.397 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-11-18T20:40:37.410 INFO:teuthology.orchestra.run.smithi100.stdout:(21/145): python3-rados-18.0.0-7418.gb0f45a51.e 4.7 MB/s | 321 kB 00:00 2023-11-18T20:40:37.477 INFO:teuthology.orchestra.run.smithi100.stdout:(22/145): python3-rbd-18.0.0-7418.gb0f45a51.el9 4.4 MB/s | 299 kB 00:00 2023-11-18T20:40:37.536 INFO:teuthology.orchestra.run.smithi100.stdout:(23/145): python3-rgw-18.0.0-7418.gb0f45a51.el9 1.7 MB/s | 99 kB 00:00 2023-11-18T20:40:37.595 INFO:teuthology.orchestra.run.smithi100.stdout:(24/145): rbd-fuse-18.0.0-7418.gb0f45a51.el9.x8 1.4 MB/s | 85 kB 00:00 2023-11-18T20:40:37.962 INFO:teuthology.orchestra.run.smithi100.stdout:(25/145): rbd-mirror-18.0.0-7418.gb0f45a51.el9. 8.2 MB/s | 3.0 MB 00:00 2023-11-18T20:40:38.020 INFO:teuthology.orchestra.run.smithi100.stdout:(26/145): rbd-nbd-18.0.0-7418.gb0f45a51.el9.x86 2.9 MB/s | 170 kB 00:00 2023-11-18T20:40:38.070 INFO:teuthology.orchestra.run.smithi100.stdout:(27/145): ceph-grafana-dashboards-18.0.0-7418.g 469 kB/s | 23 kB 00:00 2023-11-18T20:40:38.129 INFO:teuthology.orchestra.run.smithi100.stdout:(28/145): ceph-mgr-cephadm-18.0.0-7418.gb0f45a5 2.2 MB/s | 133 kB 00:00 2023-11-18T20:40:38.371 INFO:teuthology.orchestra.run.smithi100.stdout:(29/145): ceph-mgr-dashboard-18.0.0-7418.gb0f45 7.3 MB/s | 1.8 MB 00:00 2023-11-18T20:40:38.813 INFO:teuthology.orchestra.run.smithi100.stdout:(30/145): ceph-test-18.0.0-7418.gb0f45a51.el9.x 11 MB/s | 41 MB 00:03 2023-11-18T20:40:38.880 INFO:teuthology.orchestra.run.smithi100.stdout:(31/145): librgw2-18.0.0-7418.gb0f45a51.el9.x86 2.7 MB/s | 4.8 MB 00:01 2023-11-18T20:40:38.905 INFO:teuthology.orchestra.run.smithi100.stdout:(32/145): ceph-mgr-modules-core-18.0.0-7418.gb0 2.6 MB/s | 243 kB 00:00 2023-11-18T20:40:38.939 INFO:teuthology.orchestra.run.smithi100.stdout:(33/145): ceph-mgr-rook-18.0.0-7418.gb0f45a51.e 858 kB/s | 50 kB 00:00 2023-11-18T20:40:38.964 INFO:teuthology.orchestra.run.smithi100.stdout:(34/145): ceph-prometheus-alerts-18.0.0-7418.gb 259 kB/s | 15 kB 00:00 2023-11-18T20:40:39.048 INFO:teuthology.orchestra.run.smithi100.stdout:(35/145): ceph-volume-18.0.0-7418.gb0f45a51.el9 2.4 MB/s | 262 kB 00:00 2023-11-18T20:40:39.073 INFO:teuthology.orchestra.run.smithi100.stdout:(36/145): cephadm-18.0.0-7418.gb0f45a51.el9.noa 7.0 MB/s | 774 kB 00:00 2023-11-18T20:40:39.132 INFO:teuthology.orchestra.run.smithi100.stdout:(37/145): cephfs-top-18.0.0-7418.gb0f45a51.el9. 218 kB/s | 18 kB 00:00 2023-11-18T20:40:39.207 INFO:teuthology.orchestra.run.smithi100.stdout:(38/145): automake-1.16.2-8.el9.noarch.rpm 4.8 MB/s | 662 kB 00:00 2023-11-18T20:40:39.241 INFO:teuthology.orchestra.run.smithi100.stdout:(39/145): bison-3.7.4-5.el9.x86_64.rpm 8.4 MB/s | 920 kB 00:00 2023-11-18T20:40:39.266 INFO:teuthology.orchestra.run.smithi100.stdout:(40/145): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2023-11-18T20:40:39.275 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-11-18T20:40:39.275 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-11-18T20:40:39.291 INFO:teuthology.orchestra.run.smithi100.stdout:(41/145): elfutils-libelf-devel-0.190-1.el9.x86 432 kB/s | 21 kB 00:00 2023-11-18T20:40:39.317 INFO:teuthology.orchestra.run.smithi100.stdout:(42/145): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2023-11-18T20:40:39.342 INFO:teuthology.orchestra.run.smithi100.stdout:(43/145): flexiblas-3.0.4-8.el9.x86_64.rpm 635 kB/s | 32 kB 00:00 2023-11-18T20:40:39.434 INFO:teuthology.orchestra.run.smithi100.stdout:(44/145): ceph-mgr-diskprediction-local-18.0.0- 7.0 MB/s | 7.4 MB 00:01 2023-11-18T20:40:39.484 INFO:teuthology.orchestra.run.smithi100.stdout:(45/145): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2023-11-18T20:40:39.509 INFO:teuthology.orchestra.run.smithi100.stdout:(46/145): flexiblas-openblas-openmp-3.0.4-8.el9 104 kB/s | 17 kB 00:00 2023-11-18T20:40:39.534 INFO:teuthology.orchestra.run.smithi100.stdout:(47/145): jq-1.6-16.el9.x86_64.rpm 1.8 MB/s | 186 kB 00:00 2023-11-18T20:40:39.560 INFO:teuthology.orchestra.run.smithi100.stdout:(48/145): libaio-devel-0.3.111-13.el9.x86_64.rp 202 kB/s | 10 kB 00:00 2023-11-18T20:40:39.585 INFO:teuthology.orchestra.run.smithi100.stdout:(49/145): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2023-11-18T20:40:39.610 INFO:teuthology.orchestra.run.smithi100.stdout:(50/145): librabbitmq-0.11.0-7.el9.x86_64.rpm 589 kB/s | 44 kB 00:00 2023-11-18T20:40:39.644 INFO:teuthology.orchestra.run.smithi100.stdout:(51/145): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 578 kB 00:00 2023-11-18T20:40:39.677 INFO:teuthology.orchestra.run.smithi100.stdout:(52/145): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2023-11-18T20:40:39.702 INFO:teuthology.orchestra.run.smithi100.stdout:(53/145): libuuid-devel-2.37.4-15.el9.x86_64.rp 266 kB/s | 24 kB 00:00 2023-11-18T20:40:39.728 INFO:teuthology.orchestra.run.smithi100.stdout:(54/145): libzstd-devel-1.5.1-2.el9.x86_64.rpm 557 kB/s | 46 kB 00:00 2023-11-18T20:40:39.753 INFO:teuthology.orchestra.run.smithi100.stdout:(55/145): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2023-11-18T20:40:39.778 INFO:teuthology.orchestra.run.smithi100.stdout:(56/145): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2023-11-18T20:40:39.803 INFO:teuthology.orchestra.run.smithi100.stdout:(57/145): openblas-0.3.21-2.el9.x86_64.rpm 459 kB/s | 34 kB 00:00 2023-11-18T20:40:39.862 INFO:teuthology.orchestra.run.smithi100.stdout:(58/145): openssl-devel-3.0.7-24.el9.x86_64.rpm 36 MB/s | 3.0 MB 00:00 2023-11-18T20:40:39.937 INFO:teuthology.orchestra.run.smithi100.stdout:(59/145): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2023-11-18T20:40:39.979 INFO:teuthology.orchestra.run.smithi100.stdout:(60/145): postgresql-13.11-1.el9.x86_64.rpm 8.7 MB/s | 1.5 MB 00:00 2023-11-18T20:40:40.013 INFO:teuthology.orchestra.run.smithi100.stdout:(61/145): postgresql-contrib-13.11-1.el9.x86_64 5.4 MB/s | 811 kB 00:00 2023-11-18T20:40:40.038 INFO:teuthology.orchestra.run.smithi100.stdout:(62/145): postgresql-private-libs-13.11-1.el9.x 1.4 MB/s | 134 kB 00:00 2023-11-18T20:40:40.163 INFO:teuthology.orchestra.run.smithi100.stdout:(63/145): postgresql-server-13.11-1.el9.x86_64. 38 MB/s | 5.7 MB 00:00 2023-11-18T20:40:40.256 INFO:teuthology.orchestra.run.smithi100.stdout:(64/145): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 5.8 MB 00:00 2023-11-18T20:40:40.280 INFO:teuthology.orchestra.run.smithi100.stdout:(65/145): python3-jinja2-2.11.3-4.el9.noarch.rp 943 kB/s | 228 kB 00:00 2023-11-18T20:40:40.297 INFO:teuthology.orchestra.run.smithi100.stdout:(66/145): python3-jsonpatch-1.21-16.el9.noarch. 185 kB/s | 24 kB 00:00 2023-11-18T20:40:40.326 INFO:teuthology.orchestra.run.smithi100.stdout:(67/145): python3-jsonpointer-2.0-4.el9.noarch. 403 kB/s | 18 kB 00:00 2023-11-18T20:40:40.351 INFO:teuthology.orchestra.run.smithi100.stdout:(68/145): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 144 kB 00:00 2023-11-18T20:40:40.376 INFO:teuthology.orchestra.run.smithi100.stdout:(69/145): python3-markupsafe-1.1.1-12.el9.x86_6 402 kB/s | 32 kB 00:00 2023-11-18T20:40:40.401 INFO:teuthology.orchestra.run.smithi100.stdout:(70/145): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 217 kB 00:00 2023-11-18T20:40:40.486 INFO:teuthology.orchestra.run.smithi100.stdout:(71/145): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2023-11-18T20:40:40.511 INFO:teuthology.orchestra.run.smithi100.stdout:(72/145): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2023-11-18T20:40:40.536 INFO:teuthology.orchestra.run.smithi100.stdout:(73/145): python3-prettytable-0.7.2-27.el9.noar 313 kB/s | 41 kB 00:00 2023-11-18T20:40:40.561 INFO:teuthology.orchestra.run.smithi100.stdout:(74/145): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2023-11-18T20:40:40.587 INFO:teuthology.orchestra.run.smithi100.stdout:(75/145): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2023-11-18T20:40:40.612 INFO:teuthology.orchestra.run.smithi100.stdout:(76/145): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2023-11-18T20:40:40.637 INFO:teuthology.orchestra.run.smithi100.stdout:(77/145): python3-requests-oauthlib-1.3.0-12.el 568 kB/s | 43 kB 00:00 2023-11-18T20:40:40.721 INFO:teuthology.orchestra.run.smithi100.stdout:(78/145): python3-toml-0.10.2-6.el9.noarch.rpm 481 kB/s | 39 kB 00:00 2023-11-18T20:40:40.746 INFO:teuthology.orchestra.run.smithi100.stdout:(79/145): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 300 kB 00:00 2023-11-18T20:40:40.896 INFO:teuthology.orchestra.run.smithi100.stdout:(80/145): python3-scipy-1.6.2-8.el9.x86_64.rpm 51 MB/s | 16 MB 00:00 2023-11-18T20:40:40.921 INFO:teuthology.orchestra.run.smithi100.stdout:(81/145): uuid-1.6.2-55.el9.x86_64.rpm 282 kB/s | 56 kB 00:00 2023-11-18T20:40:40.938 INFO:teuthology.orchestra.run.smithi100.stdout:(82/145): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 449 kB/s | 85 kB 00:00 2023-11-18T20:40:40.963 INFO:teuthology.orchestra.run.smithi100.stdout:(83/145): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2023-11-18T20:40:40.989 INFO:teuthology.orchestra.run.smithi100.stdout:(84/145): zlib-devel-1.2.11-40.el9.x86_64.rpm 664 kB/s | 44 kB 00:00 2023-11-18T20:40:41.006 INFO:teuthology.orchestra.run.smithi100.stdout:(85/145): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2023-11-18T20:40:41.031 INFO:teuthology.orchestra.run.smithi100.stdout:(86/145): libquadmath-11.4.1-2.3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2023-11-18T20:40:41.073 INFO:teuthology.orchestra.run.smithi100.stdout:(87/145): libgfortran-11.4.1-2.3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2023-11-18T20:40:41.098 INFO:teuthology.orchestra.run.smithi100.stdout:(88/145): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2023-11-18T20:40:41.132 INFO:teuthology.orchestra.run.smithi100.stdout:(89/145): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2023-11-18T20:40:41.157 INFO:teuthology.orchestra.run.smithi100.stdout:(90/145): python3-pycparser-2.20-6.el9.noarch.r 2.2 MB/s | 124 kB 00:00 2023-11-18T20:40:41.173 INFO:teuthology.orchestra.run.smithi100.stdout:(91/145): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2023-11-18T20:40:41.199 INFO:teuthology.orchestra.run.smithi100.stdout:(92/145): python3-requests-2.25.1-7.el9.noarch. 1.6 MB/s | 113 kB 00:00 2023-11-18T20:40:41.224 INFO:teuthology.orchestra.run.smithi100.stdout:(93/145): python3-urllib3-1.26.5-4.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2023-11-18T20:40:41.366 INFO:teuthology.orchestra.run.smithi100.stdout:(94/145): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2023-11-18T20:40:41.566 INFO:teuthology.orchestra.run.smithi100.stdout:(95/145): libarrow-doc-9.0.0-9.el9.noarch.rpm 122 kB/s | 24 kB 00:00 2023-11-18T20:40:41.633 INFO:teuthology.orchestra.run.smithi100.stdout:(96/145): gperftools-libs-2.9.1-2.el9.x86_64.rp 713 kB/s | 309 kB 00:00 2023-11-18T20:40:41.725 INFO:teuthology.orchestra.run.smithi100.stdout:(97/145): libunwind-1.6.2-1.el9.x86_64.rpm 736 kB/s | 67 kB 00:00 2023-11-18T20:40:41.750 INFO:teuthology.orchestra.run.smithi100.stdout:(98/145): liboath-2.6.7-2.el9.x86_64.rpm 264 kB/s | 49 kB 00:00 2023-11-18T20:40:41.934 INFO:teuthology.orchestra.run.smithi100.stdout:(99/145): libarrow-9.0.0-9.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2023-11-18T20:40:41.959 INFO:teuthology.orchestra.run.smithi100.stdout:(100/145): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.5 MB/s | 837 kB 00:00 2023-11-18T20:40:42.018 INFO:teuthology.orchestra.run.smithi100.stdout:(101/145): python3-bcrypt-3.2.2-1.el9.x86_64.rp 521 kB/s | 43 kB 00:00 2023-11-18T20:40:42.052 INFO:teuthology.orchestra.run.smithi100.stdout:(102/145): python3-asyncssh-2.13.2-3.el9.noarch 1.8 MB/s | 548 kB 00:00 2023-11-18T20:40:42.077 INFO:teuthology.orchestra.run.smithi100.stdout:(103/145): python3-cachetools-4.2.4-1.el9.noarc 274 kB/s | 32 kB 00:00 2023-11-18T20:40:42.103 INFO:teuthology.orchestra.run.smithi100.stdout:(104/145): python3-certifi-2021.10.8-2.el9.noar 178 kB/s | 15 kB 00:00 2023-11-18T20:40:42.144 INFO:teuthology.orchestra.run.smithi100.stdout:(105/145): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2023-11-18T20:40:42.186 INFO:teuthology.orchestra.run.smithi100.stdout:(106/145): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2023-11-18T20:40:42.211 INFO:teuthology.orchestra.run.smithi100.stdout:(107/145): python3-google-auth-2.23.4-1.el9.noa 1.8 MB/s | 197 kB 00:00 2023-11-18T20:40:42.237 INFO:teuthology.orchestra.run.smithi100.stdout:(108/145): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2023-11-18T20:40:42.270 INFO:teuthology.orchestra.run.smithi100.stdout:(109/145): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2023-11-18T20:40:42.298 INFO:teuthology.orchestra.run.smithi100.stdout:(110/145): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2023-11-18T20:40:42.321 INFO:teuthology.orchestra.run.smithi100.stdout:(111/145): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2023-11-18T20:40:42.354 INFO:teuthology.orchestra.run.smithi100.stdout:(112/145): python3-jaraco-text-3.2.0-6.el9.noar 234 kB/s | 20 kB 00:00 2023-11-18T20:40:42.380 INFO:teuthology.orchestra.run.smithi100.stdout:(113/145): python3-jwt+crypto-2.4.0-1.el9.noarc 108 kB/s | 9.0 kB 00:00 2023-11-18T20:40:42.405 INFO:teuthology.orchestra.run.smithi100.stdout:(114/145): python3-jwt-2.4.0-1.el9.noarch.rpm 488 kB/s | 41 kB 00:00 2023-11-18T20:40:42.463 INFO:teuthology.orchestra.run.smithi100.stdout:(115/145): python3-logutils-0.3.5-21.el9.noarch 555 kB/s | 46 kB 00:00 2023-11-18T20:40:42.489 INFO:teuthology.orchestra.run.smithi100.stdout:(116/145): python3-more-itertools-8.12.0-2.el9. 942 kB/s | 79 kB 00:00 2023-11-18T20:40:42.522 INFO:teuthology.orchestra.run.smithi100.stdout:(117/145): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2023-11-18T20:40:42.547 INFO:teuthology.orchestra.run.smithi100.stdout:(118/145): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2023-11-18T20:40:42.598 INFO:teuthology.orchestra.run.smithi100.stdout:(119/145): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2023-11-18T20:40:42.623 INFO:teuthology.orchestra.run.smithi100.stdout:(120/145): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2023-11-18T20:40:42.648 INFO:teuthology.orchestra.run.smithi100.stdout:(121/145): python3-pyOpenSSL-21.0.0-1.el9.noarc 896 kB/s | 90 kB 00:00 2023-11-18T20:40:42.682 INFO:teuthology.orchestra.run.smithi100.stdout:(122/145): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2023-11-18T20:40:42.707 INFO:teuthology.orchestra.run.smithi100.stdout:(123/145): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2023-11-18T20:40:42.741 INFO:teuthology.orchestra.run.smithi100.stdout:(124/145): python3-rsa-4.9-2.el9.noarch.rpm 636 kB/s | 59 kB 00:00 2023-11-18T20:40:42.767 INFO:teuthology.orchestra.run.smithi100.stdout:(125/145): python3-tempora-5.0.0-2.el9.noarch.r 424 kB/s | 36 kB 00:00 2023-11-18T20:40:42.792 INFO:teuthology.orchestra.run.smithi100.stdout:(126/145): python3-typing-extensions-4.4.0-2.el 604 kB/s | 51 kB 00:00 2023-11-18T20:40:42.825 INFO:teuthology.orchestra.run.smithi100.stdout:(127/145): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2023-11-18T20:40:42.851 INFO:teuthology.orchestra.run.smithi100.stdout:(128/145): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2023-11-18T20:40:42.893 INFO:teuthology.orchestra.run.smithi100.stdout:(129/145): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2023-11-18T20:40:42.918 INFO:teuthology.orchestra.run.smithi100.stdout:(130/145): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2023-11-18T20:40:42.943 INFO:teuthology.orchestra.run.smithi100.stdout:(131/145): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2023-11-18T20:40:43.069 INFO:teuthology.orchestra.run.smithi100.stdout:(132/145): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2023-11-18T20:40:43.102 INFO:teuthology.orchestra.run.smithi100.stdout:(133/145): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-11-18T20:40:43.161 INFO:teuthology.orchestra.run.smithi100.stdout:(134/145): NetworkManager-1.45.6-1.el9.x86_64.r 38 MB/s | 2.2 MB 00:00 2023-11-18T20:40:43.219 INFO:teuthology.orchestra.run.smithi100.stdout:(135/145): NetworkManager-libnm-1.45.6-1.el9.x8 30 MB/s | 1.8 MB 00:00 2023-11-18T20:40:43.244 INFO:teuthology.orchestra.run.smithi100.stdout:(136/145): NetworkManager-team-1.45.6-1.el9.x86 1.7 MB/s | 42 kB 00:00 2023-11-18T20:40:43.287 INFO:teuthology.orchestra.run.smithi100.stdout:(137/145): NetworkManager-tui-1.45.6-1.el9.x86_ 5.7 MB/s | 245 kB 00:00 2023-11-18T20:40:43.321 INFO:teuthology.orchestra.run.smithi100.stdout:(138/145): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2023-11-18T20:40:43.354 INFO:teuthology.orchestra.run.smithi100.stdout:(139/145): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-11-18T20:40:43.387 INFO:teuthology.orchestra.run.smithi100.stdout:(140/145): elfutils-libelf-0.190-1.el9.x86_64.r 5.7 MB/s | 192 kB 00:00 2023-11-18T20:40:43.421 INFO:teuthology.orchestra.run.smithi100.stdout:(141/145): elfutils-libs-0.190-1.el9.x86_64.rpm 7.5 MB/s | 255 kB 00:00 2023-11-18T20:40:43.480 INFO:teuthology.orchestra.run.smithi100.stdout:(142/145): librados2-18.0.0-7418.gb0f45a51.el9. 5.9 MB/s | 3.3 MB 00:00 2023-11-18T20:40:43.505 INFO:teuthology.orchestra.run.smithi100.stdout:(143/145): libacl-2.3.1-4.el9.x86_64.rpm 263 kB/s | 22 kB 00:00 2023-11-18T20:40:43.530 INFO:teuthology.orchestra.run.smithi100.stdout:(144/145): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2023-11-18T20:40:43.730 INFO:teuthology.orchestra.run.smithi100.stdout:(145/145): librbd1-18.0.0-7418.gb0f45a51.el9.x8 3.8 MB/s | 3.0 MB 00:00 2023-11-18T20:40:43.734 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2023-11-18T20:40:43.734 INFO:teuthology.orchestra.run.smithi100.stdout:Total 19 MB/s | 200 MB 00:10 2023-11-18T20:40:43.753 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2023-11-18T20:40:43.930 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2023-11-18T20:40:43.930 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2023-11-18T20:40:45.301 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-11-18T20:40:45.371 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/158 2023-11-18T20:40:45.430 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/158 2023-11-18T20:40:45.627 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/158 2023-11-18T20:40:45.698 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2023-11-18T20:40:45.699 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2023-11-18T20:40:45.975 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/158 2023-11-18T20:40:46.031 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:18.0.0-7418.gb0f45a51.el9.x86_64 5/158 2023-11-18T20:40:46.152 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:18.0.0-7418.gb0f45a51.el9.x86_64 5/158 2023-11-18T20:40:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:18.0.0-7418.gb0f45a51.el9.x86_64 6/158 2023-11-18T20:40:46.257 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:18.0.0-7418.gb0f45a51.el9.x86_64 6/158 2023-11-18T20:40:46.314 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:18.0.0-7418.gb0f45a51.el9.x86_64 7/158 2023-11-18T20:40:46.344 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/158 2023-11-18T20:40:46.386 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/158 2023-11-18T20:40:46.456 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 10/158 2023-11-18T20:40:46.623 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/158 2023-11-18T20:40:46.678 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-13.11-1.el9.x86_64 12/158 2023-11-18T20:40:46.746 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:18.0.0-7418.gb0f45a51.el9.x86_64 13/158 2023-11-18T20:40:46.849 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7418.gb0f45a51.el9.x86_64 13/158 2023-11-18T20:40:46.872 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:18.0.0-7418.gb0f45a51.el9.x86 14/158 2023-11-18T20:40:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7418.gb0f45a51.el9.x86 14/158 2023-11-18T20:40:47.100 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 15/158 2023-11-18T20:40:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 16/158 2023-11-18T20:40:47.294 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 16/158 2023-11-18T20:40:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 17/158 2023-11-18T20:40:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 17/158 2023-11-18T20:40:47.749 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 17/158 2023-11-18T20:40:47.792 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/158 2023-11-18T20:40:47.863 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/158 2023-11-18T20:40:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/158 2023-11-18T20:40:47.971 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/158 2023-11-18T20:40:48.014 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/158 2023-11-18T20:40:48.054 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/158 2023-11-18T20:40:48.137 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/158 2023-11-18T20:40:48.170 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/158 2023-11-18T20:40:48.197 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/158 2023-11-18T20:40:48.268 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/158 2023-11-18T20:40:48.302 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:18.0.0-7418.gb0f45a51.el9. 28/158 2023-11-18T20:40:48.346 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:18.0.0-7418.gb0f45a51.el 29/158 2023-11-18T20:40:48.422 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:18.0.0-7418.gb0f45a51.el9.x86_6 30/158 2023-11-18T20:40:48.569 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 31/158 2023-11-18T20:40:48.624 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 32/158 2023-11-18T20:40:49.181 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 33/158 2023-11-18T20:40:49.261 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 34/158 2023-11-18T20:40:49.310 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 35/158 2023-11-18T20:40:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 36/158 2023-11-18T20:40:49.386 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:18.0.0-7418.gb0f45a51.el9.x86_6 37/158 2023-11-18T20:40:49.416 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 38/158 2023-11-18T20:40:49.527 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-3.el9.x86_64 39/158 2023-11-18T20:40:49.561 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 40/158 2023-11-18T20:40:49.618 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/158 2023-11-18T20:40:49.691 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/158 2023-11-18T20:40:49.729 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 43/158 2023-11-18T20:40:49.766 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 44/158 2023-11-18T20:40:49.813 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/158 2023-11-18T20:40:49.858 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/158 2023-11-18T20:40:49.900 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/158 2023-11-18T20:40:50.043 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/158 2023-11-18T20:40:50.093 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/158 2023-11-18T20:40:50.135 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 50/158 2023-11-18T20:40:50.231 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/158 2023-11-18T20:40:50.268 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 52/158 2023-11-18T20:40:50.317 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/158 2023-11-18T20:40:50.350 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 54/158 2023-11-18T20:40:50.449 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 55/158 2023-11-18T20:40:50.494 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 56/158 2023-11-18T20:40:50.544 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/158 2023-11-18T20:40:50.608 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/158 2023-11-18T20:40:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/158 2023-11-18T20:40:50.857 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/158 2023-11-18T20:40:51.001 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/158 2023-11-18T20:40:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/158 2023-11-18T20:40:51.164 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/158 2023-11-18T20:40:51.198 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/158 2023-11-18T20:40:51.231 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/158 2023-11-18T20:40:51.260 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/158 2023-11-18T20:40:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 67/158 2023-11-18T20:40:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/158 2023-11-18T20:40:51.352 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2023-11-18T20:40:51.368 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/158 2023-11-18T20:40:51.402 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/158 2023-11-18T20:40:51.432 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/158 2023-11-18T20:40:51.432 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/158 2023-11-18T20:40:51.458 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/158 2023-11-18T20:40:51.512 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/158 2023-11-18T20:40:51.513 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/158 2023-11-18T20:40:51.551 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/158 2023-11-18T20:40:51.584 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/158 2023-11-18T20:40:51.626 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/158 2023-11-18T20:40:51.660 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/158 2023-11-18T20:40:51.752 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/158 2023-11-18T20:40:51.796 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/158 2023-11-18T20:40:51.902 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/158 2023-11-18T20:40:51.911 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/158 2023-11-18T20:40:51.942 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:18.0.0-7418.gb0f45a51.el9.x86_64 5/158 2023-11-18T20:40:51.955 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/158 2023-11-18T20:40:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:18.0.0-7418.gb0f45a51.el9.x86_64 5/158 2023-11-18T20:40:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:18.0.0-7418.gb0f45a51.el9.x86_64 6/158 2023-11-18T20:40:52.177 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:18.0.0-7418.gb0f45a51.el9.x86_64 6/158 2023-11-18T20:40:52.222 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:18.0.0-7418.gb0f45a51.el9.x86_64 7/158 2023-11-18T20:40:52.256 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/158 2023-11-18T20:40:52.301 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/158 2023-11-18T20:40:52.360 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 10/158 2023-11-18T20:40:52.527 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/158 2023-11-18T20:40:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.11-1.el9.x86_64 12/158 2023-11-18T20:40:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:18.0.0-7418.gb0f45a51.el9.x86_64 13/158 2023-11-18T20:40:52.702 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7418.gb0f45a51.el9.x86_64 13/158 2023-11-18T20:40:52.725 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:18.0.0-7418.gb0f45a51.el9.x86 14/158 2023-11-18T20:40:52.729 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/158 2023-11-18T20:40:52.785 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/158 2023-11-18T20:40:52.787 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7418.gb0f45a51.el9.x86 14/158 2023-11-18T20:40:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/158 2023-11-18T20:40:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/158 2023-11-18T20:40:52.977 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 15/158 2023-11-18T20:40:53.036 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/158 2023-11-18T20:40:53.066 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/158 2023-11-18T20:40:53.096 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 16/158 2023-11-18T20:40:53.140 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 16/158 2023-11-18T20:40:53.146 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/158 2023-11-18T20:40:53.236 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 17/158 2023-11-18T20:40:53.283 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 17/158 2023-11-18T20:40:53.569 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 17/158 2023-11-18T20:40:53.603 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/158 2023-11-18T20:40:53.680 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/158 2023-11-18T20:40:53.734 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/158 2023-11-18T20:40:53.768 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/158 2023-11-18T20:40:53.811 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/158 2023-11-18T20:40:53.831 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/158 2023-11-18T20:40:53.851 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/158 2023-11-18T20:40:53.915 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/158 2023-11-18T20:40:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/158 2023-11-18T20:40:53.950 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/158 2023-11-18T20:40:53.977 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/158 2023-11-18T20:40:54.048 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/158 2023-11-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:18.0.0-7418.gb0f45a51.el9. 28/158 2023-11-18T20:40:54.142 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:18.0.0-7418.gb0f45a51.el 29/158 2023-11-18T20:40:54.216 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:18.0.0-7418.gb0f45a51.el9.x86_6 30/158 2023-11-18T20:40:54.351 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 31/158 2023-11-18T20:40:54.429 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 32/158 2023-11-18T20:40:54.959 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 33/158 2023-11-18T20:40:55.041 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 34/158 2023-11-18T20:40:55.086 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 35/158 2023-11-18T20:40:55.130 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 36/158 2023-11-18T20:40:55.166 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:18.0.0-7418.gb0f45a51.el9.x86_6 37/158 2023-11-18T20:40:55.197 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 38/158 2023-11-18T20:40:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/158 2023-11-18T20:40:55.304 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 39/158 2023-11-18T20:40:55.334 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 40/158 2023-11-18T20:40:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/158 2023-11-18T20:40:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/158 2023-11-18T20:40:55.389 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/158 2023-11-18T20:40:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : jq-1.6-16.el9.x86_64 93/158 2023-11-18T20:40:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/158 2023-11-18T20:40:55.454 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/158 2023-11-18T20:40:55.482 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 95/158 2023-11-18T20:40:55.484 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 43/158 2023-11-18T20:40:55.521 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 44/158 2023-11-18T20:40:55.568 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/158 2023-11-18T20:40:55.613 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/158 2023-11-18T20:40:55.656 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/158 2023-11-18T20:40:55.792 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/158 2023-11-18T20:40:55.806 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/158 2023-11-18T20:40:55.828 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:18.0.0-7418.gb0f45a51.el9.x86_64 97/158 2023-11-18T20:40:55.832 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/158 2023-11-18T20:40:55.873 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 50/158 2023-11-18T20:40:55.919 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:18.0.0-7418.gb0f45a51.el9.x86_64 97/158 2023-11-18T20:40:55.949 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 98/158 2023-11-18T20:40:55.960 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/158 2023-11-18T20:40:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/158 2023-11-18T20:40:55.991 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 52/158 2023-11-18T20:40:56.031 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/158 2023-11-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 54/158 2023-11-18T20:40:56.161 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 55/158 2023-11-18T20:40:56.200 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 56/158 2023-11-18T20:40:56.250 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/158 2023-11-18T20:40:56.305 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/158 2023-11-18T20:40:56.444 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/158 2023-11-18T20:40:56.472 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:18.0.0-7418.gb0f45a51.el9.x86_64 101/158 2023-11-18T20:40:56.496 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/158 2023-11-18T20:40:56.538 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/158 2023-11-18T20:40:56.561 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:18.0.0-7418.gb0f45a51.el9.x86_64 101/158 2023-11-18T20:40:56.599 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 102/158 2023-11-18T20:40:56.676 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/158 2023-11-18T20:40:56.724 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/158 2023-11-18T20:40:56.753 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/158 2023-11-18T20:40:56.787 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/158 2023-11-18T20:40:56.820 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/158 2023-11-18T20:40:56.850 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/158 2023-11-18T20:40:56.877 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 67/158 2023-11-18T20:40:56.922 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/158 2023-11-18T20:40:57.007 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/158 2023-11-18T20:40:57.050 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/158 2023-11-18T20:40:57.080 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/158 2023-11-18T20:40:57.150 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/158 2023-11-18T20:40:57.190 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/158 2023-11-18T20:40:57.223 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/158 2023-11-18T20:40:57.265 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/158 2023-11-18T20:40:57.299 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/158 2023-11-18T20:40:57.390 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/158 2023-11-18T20:40:57.427 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/158 2023-11-18T20:40:57.530 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/158 2023-11-18T20:40:57.587 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/158 2023-11-18T20:40:58.329 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/158 2023-11-18T20:40:58.375 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/158 2023-11-18T20:40:58.408 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/158 2023-11-18T20:40:58.441 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/158 2023-11-18T20:40:58.609 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/158 2023-11-18T20:40:58.639 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/158 2023-11-18T20:40:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/158 2023-11-18T20:40:58.746 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 103/158 2023-11-18T20:40:58.779 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 103/158 2023-11-18T20:40:59.345 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 103/158 2023-11-18T20:40:59.379 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/158 2023-11-18T20:40:59.382 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:18.0.0-7418.gb0f45a51.el9.x86_64 104/158 2023-11-18T20:40:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:18.0.0-7418.gb0f45a51.el9.x86_64 104/158 2023-11-18T20:40:59.451 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-18T20:40:59.451 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-18T20:40:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:40:59.488 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/158 2023-11-18T20:40:59.500 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:18.0.0-7418.gb0f45a51.el9.x86_64 105/158 2023-11-18T20:41:00.760 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/158 2023-11-18T20:41:00.842 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/158 2023-11-18T20:41:00.873 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/158 2023-11-18T20:41:00.904 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : jq-1.6-16.el9.x86_64 93/158 2023-11-18T20:41:00.932 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/158 2023-11-18T20:41:00.965 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 95/158 2023-11-18T20:41:01.283 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/158 2023-11-18T20:41:01.312 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:18.0.0-7418.gb0f45a51.el9.x86_64 97/158 2023-11-18T20:41:01.385 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:18.0.0-7418.gb0f45a51.el9.x86_64 97/158 2023-11-18T20:41:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 98/158 2023-11-18T20:41:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/158 2023-11-18T20:41:01.872 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/158 2023-11-18T20:41:01.898 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:18.0.0-7418.gb0f45a51.el9.x86_64 101/158 2023-11-18T20:41:01.961 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:18.0.0-7418.gb0f45a51.el9.x86_64 101/158 2023-11-18T20:41:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 102/158 2023-11-18T20:41:04.257 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 103/158 2023-11-18T20:41:04.297 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 103/158 2023-11-18T20:41:04.835 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 103/158 2023-11-18T20:41:04.875 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:18.0.0-7418.gb0f45a51.el9.x86_64 104/158 2023-11-18T20:41:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:18.0.0-7418.gb0f45a51.el9.x86_64 104/158 2023-11-18T20:41:04.965 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-18T20:41:04.965 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-18T20:41:04.965 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:05.001 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:18.0.0-7418.gb0f45a51.el9.x86_64 105/158 2023-11-18T20:41:10.996 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7418.gb0f45a51.el9.x86_64 105/158 2023-11-18T20:41:10.996 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2023-11-18T20:41:10.996 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2023-11-18T20:41:10.996 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2023-11-18T20:41:10.996 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2023-11-18T20:41:10.996 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2023-11-18T20:41:10.996 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2023-11-18T20:41:10.996 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2023-11-18T20:41:10.997 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:11.162 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:18.0.0-7418.gb0f45a51.el9.noarch 106/158 2023-11-18T20:41:11.178 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:18.0.0-7418.gb0f45a51.el9.noarch 106/158 2023-11-18T20:41:11.178 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:11.178 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-18T20:41:11.178 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:12.799 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:18.0.0-7418.gb0f45a51.el9.x86_64 107/158 2023-11-18T20:41:12.826 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:18.0.0-7418.gb0f45a51.el9.x86_64 107/158 2023-11-18T20:41:12.826 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:12.826 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-18T20:41:12.826 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-18T20:41:12.826 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-18T20:41:12.826 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:13.032 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:18.0.0-7418.gb0f45a51.el9.x86_64 108/158 2023-11-18T20:41:13.056 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:18.0.0-7418.gb0f45a51.el9.x86_64 108/158 2023-11-18T20:41:13.056 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:13.056 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-18T20:41:13.056 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-18T20:41:13.057 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-18T20:41:13.057 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:13.525 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:18.0.0-7418.gb0f45a51.el9.x86_64 109/158 2023-11-18T20:41:13.548 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:18.0.0-7418.gb0f45a51.el9.x86_64 109/158 2023-11-18T20:41:13.548 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:13.548 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-18T20:41:13.549 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-18T20:41:13.549 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-18T20:41:13.549 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : automake-1.16.2-8.el9.noarch 110/158 2023-11-18T20:41:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:18.0.0-7418.gb0f45a51.el9.noarch 111/158 2023-11-18T20:41:14.150 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:18.0.0-7418.gb0f45a51.el9.noarch 111/158 2023-11-18T20:41:14.179 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7418.gb0f45a51.e 112/158 2023-11-18T20:41:14.243 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7418.gb0f45a51. 113/158 2023-11-18T20:41:14.273 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7418.gb0f45a51.el9.noa 114/158 2023-11-18T20:41:14.504 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7418.gb0f45a51.el9.noa 114/158 2023-11-18T20:41:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7418.gb0f45a51.el9.n 115/158 2023-11-18T20:41:15.552 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7418.gb0f45a51.el9.n 115/158 2023-11-18T20:41:15.582 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7418.gb0f 116/158 2023-11-18T20:41:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7418.gb0f 116/158 2023-11-18T20:41:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7418.gb0f45a51.el 117/158 2023-11-18T20:41:15.876 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:18.0.0-7418.gb0f45a51.el9.x86_64 118/158 2023-11-18T20:41:15.900 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7418.gb0f45a51.el9.x86_64 118/158 2023-11-18T20:41:15.901 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:15.901 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-18T20:41:15.901 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-18T20:41:15.901 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-18T20:41:15.901 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:15.943 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:18.0.0-7418.gb0f45a51.el9.noarch 119/158 2023-11-18T20:41:15.954 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7418.gb0f45a51.el9.noarch 119/158 2023-11-18T20:41:16.011 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:18.0.0-7418.gb0f45a51.el9.x86_64 120/158 2023-11-18T20:41:16.287 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7418.gb0f45a51.el9.x86_64 105/158 2023-11-18T20:41:16.287 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2023-11-18T20:41:16.287 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2023-11-18T20:41:16.287 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2023-11-18T20:41:16.287 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2023-11-18T20:41:16.287 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2023-11-18T20:41:16.287 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2023-11-18T20:41:16.288 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2023-11-18T20:41:16.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:16.447 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:18.0.0-7418.gb0f45a51.el9.noarch 106/158 2023-11-18T20:41:16.462 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:18.0.0-7418.gb0f45a51.el9.noarch 106/158 2023-11-18T20:41:16.462 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:16.462 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-18T20:41:16.463 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:16.833 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libtool-2.4.6-45.el9.x86_64 121/158 2023-11-18T20:41:16.870 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 122/158 2023-11-18T20:41:16.894 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 122/158 2023-11-18T20:41:16.894 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:16.894 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-18T20:41:16.894 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-18T20:41:16.894 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-18T20:41:16.894 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:16.937 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7418.gb0f45 123/158 2023-11-18T20:41:16.960 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7418.gb0f45 123/158 2023-11-18T20:41:16.961 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:16.961 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-18T20:41:16.961 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:17.013 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 124/158 2023-11-18T20:41:17.037 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 124/158 2023-11-18T20:41:17.037 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:17.037 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-18T20:41:17.037 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-18T20:41:17.037 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-18T20:41:17.037 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:17.347 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 125/158 2023-11-18T20:41:17.371 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 125/158 2023-11-18T20:41:17.371 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:17.371 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-18T20:41:17.371 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-18T20:41:17.371 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-18T20:41:17.371 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:18.058 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:18.0.0-7418.gb0f45a51.el9.x86_64 107/158 2023-11-18T20:41:18.083 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:18.0.0-7418.gb0f45a51.el9.x86_64 107/158 2023-11-18T20:41:18.084 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:18.084 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-18T20:41:18.084 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-18T20:41:18.084 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-18T20:41:18.084 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:18.284 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:18.0.0-7418.gb0f45a51.el9.x86_64 108/158 2023-11-18T20:41:18.306 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:18.0.0-7418.gb0f45a51.el9.x86_64 108/158 2023-11-18T20:41:18.306 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:18.306 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-18T20:41:18.306 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-18T20:41:18.306 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-18T20:41:18.306 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:18.751 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:18.0.0-7418.gb0f45a51.el9.x86_64 109/158 2023-11-18T20:41:18.773 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:18.0.0-7418.gb0f45a51.el9.x86_64 109/158 2023-11-18T20:41:18.773 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:18.773 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-18T20:41:18.773 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-18T20:41:18.773 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-18T20:41:18.774 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:18.861 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 110/158 2023-11-18T20:41:19.191 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:18.0.0-7418.gb0f45a51.el9.noarch 111/158 2023-11-18T20:41:19.218 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:18.0.0-7418.gb0f45a51.el9.noarch 111/158 2023-11-18T20:41:19.238 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7418.gb0f45a51.e 112/158 2023-11-18T20:41:19.291 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7418.gb0f45a51. 113/158 2023-11-18T20:41:19.316 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7418.gb0f45a51.el9.noa 114/158 2023-11-18T20:41:19.534 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7418.gb0f45a51.el9.noa 114/158 2023-11-18T20:41:19.567 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7418.gb0f45a51.el9.n 115/158 2023-11-18T20:41:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7418.gb0f45a51.el9.n 115/158 2023-11-18T20:41:20.550 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7418.gb0f 116/158 2023-11-18T20:41:20.650 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7418.gb0f 116/158 2023-11-18T20:41:20.806 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7418.gb0f45a51.el 117/158 2023-11-18T20:41:20.835 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:18.0.0-7418.gb0f45a51.el9.x86_64 118/158 2023-11-18T20:41:20.859 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7418.gb0f45a51.el9.x86_64 118/158 2023-11-18T20:41:20.859 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:20.859 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-18T20:41:20.859 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-18T20:41:20.859 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-18T20:41:20.859 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:20.903 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:18.0.0-7418.gb0f45a51.el9.noarch 119/158 2023-11-18T20:41:20.914 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7418.gb0f45a51.el9.noarch 119/158 2023-11-18T20:41:20.961 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:18.0.0-7418.gb0f45a51.el9.x86_64 120/158 2023-11-18T20:41:21.412 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:18.0.0-7418.gb0f45a51.el9.x86_64 126/158 2023-11-18T20:41:21.459 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 127/158 2023-11-18T20:41:21.500 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 128/158 2023-11-18T20:41:21.531 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 129/158 2023-11-18T20:41:21.716 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 130/158 2023-11-18T20:41:21.747 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-45.el9.x86_64 121/158 2023-11-18T20:41:21.780 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 122/158 2023-11-18T20:41:21.802 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 122/158 2023-11-18T20:41:21.803 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:21.803 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-18T20:41:21.803 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-18T20:41:21.803 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-18T20:41:21.803 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:21.834 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 131/158 2023-11-18T20:41:21.863 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 132/158 2023-11-18T20:41:21.866 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7418.gb0f45 123/158 2023-11-18T20:41:21.888 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7418.gb0f45 123/158 2023-11-18T20:41:21.888 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:21.888 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-18T20:41:21.888 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:21.896 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:18.0.0-7418.gb0f45a51.el9.x86_ 133/158 2023-11-18T20:41:21.928 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 134/158 2023-11-18T20:41:21.940 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 124/158 2023-11-18T20:41:21.958 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 135/158 2023-11-18T20:41:21.962 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 124/158 2023-11-18T20:41:21.962 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:21.962 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-18T20:41:21.962 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-18T20:41:21.962 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-18T20:41:21.962 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:22.032 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 136/158 2023-11-18T20:41:22.097 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 137/158 2023-11-18T20:41:22.134 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 138/158 2023-11-18T20:41:22.257 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 125/158 2023-11-18T20:41:22.279 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 125/158 2023-11-18T20:41:22.279 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-18T20:41:22.279 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-18T20:41:22.279 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-18T20:41:22.279 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-18T20:41:22.279 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:22.808 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 139/158 2023-11-18T20:41:22.847 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 139/158 2023-11-18T20:41:22.868 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 139/158 2023-11-18T20:41:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-top-2:18.0.0-7418.gb0f45a51.el9.noarch 140/158 2023-11-18T20:41:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 141/158 2023-11-18T20:41:23.868 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 142/158 2023-11-18T20:41:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 143/158 2023-11-18T20:41:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flex-2.6.4-9.el9.x86_64 144/158 2023-11-18T20:41:24.030 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : bison-3.7.4-5.el9.x86_64 145/158 2023-11-18T20:41:24.055 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/158 2023-11-18T20:41:24.088 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 147/158 2023-11-18T20:41:24.115 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 148/158 2023-11-18T20:41:24.115 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 149/158 2023-11-18T20:41:24.191 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 149/158 2023-11-18T20:41:24.216 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 150/158 2023-11-18T20:41:24.243 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 151/158 2023-11-18T20:41:24.268 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 152/158 2023-11-18T20:41:24.275 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 153/158 2023-11-18T20:41:24.275 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 153/158 2023-11-18T20:41:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 153/158 2023-11-18T20:41:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 154/158 2023-11-18T20:41:24.537 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 154/158 2023-11-18T20:41:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 155/158 2023-11-18T20:41:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 156/158 2023-11-18T20:41:24.647 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 156/158 2023-11-18T20:41:24.672 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 157/158 2023-11-18T20:41:24.748 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 158/158 2023-11-18T20:41:26.069 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:18.0.0-7418.gb0f45a51.el9.x86_64 126/158 2023-11-18T20:41:26.109 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 127/158 2023-11-18T20:41:26.150 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 128/158 2023-11-18T20:41:26.174 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 129/158 2023-11-18T20:41:26.353 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 130/158 2023-11-18T20:41:26.465 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 131/158 2023-11-18T20:41:26.498 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 132/158 2023-11-18T20:41:26.531 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:18.0.0-7418.gb0f45a51.el9.x86_ 133/158 2023-11-18T20:41:26.571 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 134/158 2023-11-18T20:41:26.601 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 135/158 2023-11-18T20:41:26.704 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 136/158 2023-11-18T20:41:26.780 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 137/158 2023-11-18T20:41:26.816 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 138/158 2023-11-18T20:41:27.453 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 139/158 2023-11-18T20:41:27.490 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 139/158 2023-11-18T20:41:27.509 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 139/158 2023-11-18T20:41:27.557 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:18.0.0-7418.gb0f45a51.el9.noarch 140/158 2023-11-18T20:41:28.398 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 141/158 2023-11-18T20:41:28.478 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 142/158 2023-11-18T20:41:28.519 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 143/158 2023-11-18T20:41:28.603 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 144/158 2023-11-18T20:41:28.632 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 145/158 2023-11-18T20:41:28.657 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/158 2023-11-18T20:41:28.683 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 147/158 2023-11-18T20:41:28.709 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 148/158 2023-11-18T20:41:28.709 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 149/158 2023-11-18T20:41:28.810 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 149/158 2023-11-18T20:41:28.838 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 150/158 2023-11-18T20:41:28.868 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 151/158 2023-11-18T20:41:28.893 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 152/158 2023-11-18T20:41:28.904 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 153/158 2023-11-18T20:41:28.904 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 153/158 2023-11-18T20:41:29.080 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 153/158 2023-11-18T20:41:29.080 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 154/158 2023-11-18T20:41:29.142 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 158/158 2023-11-18T20:41:29.143 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:18.0.0-7418.gb0f45a51.el9.x86_64 1/158 2023-11-18T20:41:29.143 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:18.0.0-7418.gb0f45a51.el9.x86_64 2/158 2023-11-18T20:41:29.143 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 3/158 2023-11-18T20:41:29.143 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 4/158 2023-11-18T20:41:29.143 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7418.gb0f45 5/158 2023-11-18T20:41:29.143 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:18.0.0-7418.gb0f45a51.el9.x86_64 6/158 2023-11-18T20:41:29.143 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:18.0.0-7418.gb0f45a51.el9.x86_64 7/158 2023-11-18T20:41:29.143 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:18.0.0-7418.gb0f45a51.el9.x86_64 8/158 2023-11-18T20:41:29.143 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:18.0.0-7418.gb0f45a51.el9.x86_64 9/158 2023-11-18T20:41:29.143 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 10/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:18.0.0-7418.gb0f45a51.el9.x86_64 11/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:18.0.0-7418.gb0f45a51.el9.x86_64 12/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 13/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:18.0.0-7418.gb0f45a51.el9.x86_ 14/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:18.0.0-7418.gb0f45a51.el9.x86_64 15/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:18.0.0-7418.gb0f45a51.el9.x86_64 16/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:18.0.0-7418.gb0f45a51.el9.x86_6 17/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:18.0.0-7418.gb0f45a51.el9.x86 18/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:18.0.0-7418.gb0f45a51.el9.x86_64 19/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7418.gb0f45a51.el 20/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:18.0.0-7418.gb0f45a51.el9. 21/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:18.0.0-7418.gb0f45a51.el9.x86_6 22/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:18.0.0-7418.gb0f45a51.el9.x86_64 23/158 2023-11-18T20:41:29.144 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 24/158 2023-11-18T20:41:29.145 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 25/158 2023-11-18T20:41:29.145 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 26/158 2023-11-18T20:41:29.145 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 27/158 2023-11-18T20:41:29.145 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 28/158 2023-11-18T20:41:29.145 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7418.gb0f45a51. 29/158 2023-11-18T20:41:29.145 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7418.gb0f45a51.el9.noa 30/158 2023-11-18T20:41:29.157 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7418.gb0f45a51.el9.n 31/158 2023-11-18T20:41:29.157 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7418.gb0f 32/158 2023-11-18T20:41:29.157 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7418.gb0f45a51.el 33/158 2023-11-18T20:41:29.157 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7418.gb0f45a51.el9.noarch 34/158 2023-11-18T20:41:29.157 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7418.gb0f45a51.e 35/158 2023-11-18T20:41:29.157 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:18.0.0-7418.gb0f45a51.el9.noarch 36/158 2023-11-18T20:41:29.157 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:18.0.0-7418.gb0f45a51.el9.noarch 37/158 2023-11-18T20:41:29.157 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-top-2:18.0.0-7418.gb0f45a51.el9.noarch 38/158 2023-11-18T20:41:29.157 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : jq-1.6-16.el9.x86_64 47/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/158 2023-11-18T20:41:29.158 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 59/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/158 2023-11-18T20:41:29.159 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/158 2023-11-18T20:41:29.160 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 84/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 85/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 86/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 87/158 2023-11-18T20:41:29.161 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 88/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 89/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 90/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 91/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 92/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 93/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/158 2023-11-18T20:41:29.162 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/158 2023-11-18T20:41:29.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/158 2023-11-18T20:41:29.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:18.0.0-7418.gb0f45a51.el9.x86_64 133/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:18.0.0-7418.gb0f45a51.el9.x86_64 135/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/158 2023-11-18T20:41:29.165 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/158 2023-11-18T20:41:29.166 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/158 2023-11-18T20:41:29.167 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/158 2023-11-18T20:41:29.167 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/158 2023-11-18T20:41:29.167 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/158 2023-11-18T20:41:29.167 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/158 2023-11-18T20:41:29.167 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/158 2023-11-18T20:41:29.167 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/158 2023-11-18T20:41:29.167 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/158 2023-11-18T20:41:29.167 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 154/158 2023-11-18T20:41:29.188 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 155/158 2023-11-18T20:41:29.188 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 156/158 2023-11-18T20:41:29.256 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 156/158 2023-11-18T20:41:29.281 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 157/158 2023-11-18T20:41:29.346 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 158/158 2023-11-18T20:41:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/158 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: acl-2.3.1-4.el9.x86_64 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-18T20:41:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: automake-1.16.2-8.el9.noarch 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: bison-3.7.4-5.el9.x86_64 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: flex-2.6.4-9.el9.x86_64 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-18T20:41:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: jq-1.6-16.el9.x86_64 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-18T20:41:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-13.11-1.el9.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-18T20:41:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-18T20:41:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-18T20:41:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-18T20:41:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-18T20:41:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-3.el9.x86_64 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-18T20:41:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-18T20:41:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-18T20:41:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:31.726 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-11-18T20:41:32.219 DEBUG:teuthology.parallel:result is None 2023-11-18T20:41:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 158/158 2023-11-18T20:41:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:18.0.0-7418.gb0f45a51.el9.x86_64 1/158 2023-11-18T20:41:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:18.0.0-7418.gb0f45a51.el9.x86_64 2/158 2023-11-18T20:41:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 3/158 2023-11-18T20:41:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 4/158 2023-11-18T20:41:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7418.gb0f45 5/158 2023-11-18T20:41:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:18.0.0-7418.gb0f45a51.el9.x86_64 6/158 2023-11-18T20:41:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:18.0.0-7418.gb0f45a51.el9.x86_64 7/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:18.0.0-7418.gb0f45a51.el9.x86_64 8/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:18.0.0-7418.gb0f45a51.el9.x86_64 9/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 10/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:18.0.0-7418.gb0f45a51.el9.x86_64 11/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:18.0.0-7418.gb0f45a51.el9.x86_64 12/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 13/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:18.0.0-7418.gb0f45a51.el9.x86_ 14/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:18.0.0-7418.gb0f45a51.el9.x86_64 15/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:18.0.0-7418.gb0f45a51.el9.x86_64 16/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:18.0.0-7418.gb0f45a51.el9.x86_6 17/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:18.0.0-7418.gb0f45a51.el9.x86 18/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:18.0.0-7418.gb0f45a51.el9.x86_64 19/158 2023-11-18T20:41:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7418.gb0f45a51.el 20/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:18.0.0-7418.gb0f45a51.el9. 21/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:18.0.0-7418.gb0f45a51.el9.x86_6 22/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:18.0.0-7418.gb0f45a51.el9.x86_64 23/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 24/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 25/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 26/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 27/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 28/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7418.gb0f45a51. 29/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7418.gb0f45a51.el9.noa 30/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7418.gb0f45a51.el9.n 31/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7418.gb0f 32/158 2023-11-18T20:41:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7418.gb0f45a51.el 33/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7418.gb0f45a51.el9.noarch 34/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7418.gb0f45a51.e 35/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:18.0.0-7418.gb0f45a51.el9.noarch 36/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:18.0.0-7418.gb0f45a51.el9.noarch 37/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:18.0.0-7418.gb0f45a51.el9.noarch 38/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/158 2023-11-18T20:41:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : jq-1.6-16.el9.x86_64 47/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/158 2023-11-18T20:41:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 59/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/158 2023-11-18T20:41:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/158 2023-11-18T20:41:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 84/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 85/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 86/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 87/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 88/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 89/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 90/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 91/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 92/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 93/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/158 2023-11-18T20:41:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/158 2023-11-18T20:41:33.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/158 2023-11-18T20:41:33.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:18.0.0-7418.gb0f45a51.el9.x86_64 133/158 2023-11-18T20:41:33.930 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:18.0.0-7418.gb0f45a51.el9.x86_64 135/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/158 2023-11-18T20:41:33.931 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/158 2023-11-18T20:41:33.932 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/158 2023-11-18T20:41:33.932 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/158 2023-11-18T20:41:33.932 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/158 2023-11-18T20:41:33.932 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/158 2023-11-18T20:41:33.932 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/158 2023-11-18T20:41:33.932 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/158 2023-11-18T20:41:33.932 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/158 2023-11-18T20:41:33.932 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/158 2023-11-18T20:41:33.932 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/158 2023-11-18T20:41:33.932 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/158 2023-11-18T20:41:36.698 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/158 2023-11-18T20:41:36.698 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout: acl-2.3.1-4.el9.x86_64 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-18T20:41:36.699 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.700 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.701 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:18.0.0-7418.gb0f45a51.el9.noarch 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-18T20:41:36.702 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: jq-1.6-16.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-18T20:41:36.703 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-18T20:41:36.704 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-18T20:41:36.705 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-18T20:41:36.705 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-18T20:41:36.705 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-11-18T20:41:36.705 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-18T20:41:36.705 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.11-1.el9.x86_64 2023-11-18T20:41:36.705 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-18T20:41:36.705 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-18T20:41:36.705 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-18T20:41:36.705 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-18T20:41:36.705 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-18T20:41:36.706 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-18T20:41:36.707 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-18T20:41:36.708 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-18T20:41:36.708 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-18T20:41:36.708 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-18T20:41:36.708 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-18T20:41:36.708 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-18T20:41:36.708 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-18T20:41:36.708 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-18T20:41:36.708 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2023-11-18T20:41:36.708 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-18T20:41:36.708 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-18T20:41:36.709 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.710 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.711 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:18.0.0-7418.gb0f45a51.el9.x86_64 2023-11-18T20:41:36.711 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2023-11-18T20:41:36.711 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-18T20:41:36.711 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-18T20:41:36.711 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-18T20:41:36.711 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-18T20:41:36.711 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-18T20:41:36.711 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-18T20:41:36.712 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:36.712 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2023-11-18T20:41:37.115 DEBUG:teuthology.parallel:result is None 2023-11-18T20:41:37.116 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe 2023-11-18T20:41:37.268 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-18T20:41:37.295 INFO:teuthology.orchestra.run.smithi060.stdout:18.0.0-7418.gb0f45a51.el9 2023-11-18T20:41:37.296 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7418.gb0f45a51.el9 2023-11-18T20:41:37.296 INFO:teuthology.task.install:The correct ceph version 18.0.0-7418.gb0f45a51 is installed. 2023-11-18T20:41:37.296 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe 2023-11-18T20:41:37.436 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-18T20:41:37.458 INFO:teuthology.orchestra.run.smithi100.stdout:18.0.0-7418.gb0f45a51.el9 2023-11-18T20:41:37.459 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7418.gb0f45a51.el9 2023-11-18T20:41:37.459 INFO:teuthology.task.install:The correct ceph version 18.0.0-7418.gb0f45a51 is installed. 2023-11-18T20:41:37.459 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-18T20:41:37.459 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:37.459 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-18T20:41:37.498 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:41:37.499 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-18T20:41:37.529 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-18T20:41:37.529 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:37.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2023-11-18T20:41:37.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-18T20:41:37.646 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:41:37.646 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2023-11-18T20:41:37.673 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-18T20:41:37.740 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-18T20:41:37.740 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:37.740 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-18T20:41:37.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-18T20:41:37.848 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:41:37.848 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-18T20:41:37.874 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-18T20:41:37.942 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-18T20:41:37.942 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:37.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2023-11-18T20:41:37.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-18T20:41:38.049 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:41:38.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2023-11-18T20:41:38.076 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-18T20:41:38.143 INFO:teuthology.run_tasks:Running task ceph... 2023-11-18T20:41:38.206 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-18T20:41:38.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /var/log/ceph 2023-11-18T20:41:38.208 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2023-11-18T20:41:38.239 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-18T20:41:38.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-18T20:41:38.281 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-18T20:41:38.311 INFO:tasks.ceph:Creating extra log directories... 2023-11-18T20:41:38.312 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-18T20:41:38.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-18T20:41:38.390 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-18T20:41:38.390 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-18T20:41:38.390 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-18T20:41:38.390 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:38.391 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-18T20:41:38.441 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-18T20:41:38.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-18T20:41:38.603 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:41:38.603 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-18T20:41:38.618 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-18T20:41:38.734 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-18T20:41:38.766 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-18T20:41:38.766 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-18T20:41:38.766 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7461690', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1491, 'sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi060.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP/jp6CjnY96YaqBnfvGbW0BxolDYOlg13z8a86sb9oBZzGAkKns6lahWcxSXV8W7JKoP3jW9dkMLLOaAzH47nc=', 'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfh9p0ioMlgydSxkwEIUrNH/H9D8ZcH6UBaugbWq0YsmL7fhRTHJJ9GD2Z6oh0HuGRq1O9TFZp+IzykXO+C/J8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-18_13:51:50', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-18T20:41:38.767 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-18T20:41:38.769 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-18T20:41:38.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/run/ceph 2023-11-18T20:41:38.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2023-11-18T20:41:38.883 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:38.883 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2023-11-18T20:41:38.909 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-18T20:41:38.909 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2023-11-18T20:41:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-18T20:41:38.964 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-18T20:41:38.964 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 665 Links: 1 2023-11-18T20:41:38.964 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-18T20:41:38.965 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-11-18T20:41:38.965 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-11-18 20:41:25.097915637 +0000 2023-11-18T20:41:38.965 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-11-18 20:39:27.576074906 +0000 2023-11-18T20:41:38.965 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-11-18 20:39:27.576074906 +0000 2023-11-18T20:41:38.965 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-11-18 20:39:27.576074906 +0000 2023-11-18T20:41:38.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-18T20:41:39.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-11-18T20:41:39.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-11-18T20:41:39.034 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 9.9909e-05 s, 5.1 MB/s 2023-11-18T20:41:39.035 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-18T20:41:39.091 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2023-11-18T20:41:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-18T20:41:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-18T20:41:39.147 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 683 Links: 1 2023-11-18T20:41:39.147 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-18T20:41:39.147 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-11-18T20:41:39.147 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-11-18 20:41:25.098915628 +0000 2023-11-18T20:41:39.147 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-11-18 20:39:28.072070191 +0000 2023-11-18T20:41:39.147 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-11-18 20:39:28.072070191 +0000 2023-11-18T20:41:39.147 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-11-18 20:39:28.072070191 +0000 2023-11-18T20:41:39.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-18T20:41:39.216 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-11-18T20:41:39.216 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-11-18T20:41:39.216 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000100732 s, 5.1 MB/s 2023-11-18T20:41:39.217 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-18T20:41:39.274 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2023-11-18T20:41:39.329 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-18T20:41:39.329 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-18T20:41:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-11-18T20:41:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-18T20:41:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-11-18T20:41:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-11-18 20:41:25.098915628 +0000 2023-11-18T20:41:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-11-18 20:39:28.560065553 +0000 2023-11-18T20:41:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-11-18 20:39:28.560065553 +0000 2023-11-18T20:41:39.329 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-11-18 20:39:28.560065553 +0000 2023-11-18T20:41:39.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-18T20:41:39.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-11-18T20:41:39.398 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-11-18T20:41:39.398 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 7.5975e-05 s, 6.7 MB/s 2023-11-18T20:41:39.399 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-18T20:41:39.454 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2023-11-18T20:41:39.510 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-18T20:41:39.510 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-18T20:41:39.510 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 721 Links: 1 2023-11-18T20:41:39.510 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-18T20:41:39.510 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-11-18T20:41:39.510 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-11-18 20:41:25.098915628 +0000 2023-11-18T20:41:39.510 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-11-18 20:39:29.054060857 +0000 2023-11-18T20:41:39.510 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-11-18 20:39:29.054060857 +0000 2023-11-18T20:41:39.510 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2023-11-18 20:39:29.054060857 +0000 2023-11-18T20:41:39.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-18T20:41:39.578 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-11-18T20:41:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-11-18T20:41:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 9.4075e-05 s, 5.4 MB/s 2023-11-18T20:41:39.580 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-18T20:41:39.636 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-18T20:41:39.636 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:41:39.636 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2023-11-18T20:41:39.653 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-18T20:41:39.653 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2023-11-18T20:41:39.709 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-18T20:41:39.709 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-18T20:41:39.709 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 663 Links: 1 2023-11-18T20:41:39.709 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-18T20:41:39.709 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2023-11-18T20:41:39.709 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2023-11-18 20:41:29.672561179 +0000 2023-11-18T20:41:39.709 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2023-11-18 20:39:17.906920842 +0000 2023-11-18T20:41:39.709 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2023-11-18 20:39:17.906920842 +0000 2023-11-18T20:41:39.710 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2023-11-18 20:39:17.906920842 +0000 2023-11-18T20:41:39.710 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-18T20:41:39.776 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2023-11-18T20:41:39.777 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2023-11-18T20:41:39.777 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000209175 s, 2.4 MB/s 2023-11-18T20:41:39.778 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-18T20:41:39.835 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2023-11-18T20:41:39.890 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-18T20:41:39.891 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-18T20:41:39.891 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-11-18T20:41:39.891 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-18T20:41:39.891 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2023-11-18T20:41:39.891 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2023-11-18 20:41:29.673561154 +0000 2023-11-18T20:41:39.891 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2023-11-18 20:39:18.405907986 +0000 2023-11-18T20:41:39.891 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2023-11-18 20:39:18.405907986 +0000 2023-11-18T20:41:39.891 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2023-11-18 20:39:18.405907986 +0000 2023-11-18T20:41:39.891 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-18T20:41:39.957 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2023-11-18T20:41:39.957 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2023-11-18T20:41:39.957 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000180378 s, 2.8 MB/s 2023-11-18T20:41:39.958 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-18T20:41:40.014 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2023-11-18T20:41:40.070 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-18T20:41:40.070 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-18T20:41:40.070 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-11-18T20:41:40.070 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-18T20:41:40.070 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2023-11-18T20:41:40.070 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2023-11-18 20:41:29.673561154 +0000 2023-11-18T20:41:40.070 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2023-11-18 20:39:18.886895593 +0000 2023-11-18T20:41:40.070 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2023-11-18 20:39:18.886895593 +0000 2023-11-18T20:41:40.070 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2023-11-18 20:39:18.886895593 +0000 2023-11-18T20:41:40.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-18T20:41:40.136 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2023-11-18T20:41:40.136 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2023-11-18T20:41:40.136 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000253385 s, 2.0 MB/s 2023-11-18T20:41:40.137 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-18T20:41:40.193 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2023-11-18T20:41:40.250 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-18T20:41:40.250 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-18T20:41:40.250 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 719 Links: 1 2023-11-18T20:41:40.250 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-18T20:41:40.251 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2023-11-18T20:41:40.251 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2023-11-18 20:41:29.674561128 +0000 2023-11-18T20:41:40.251 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2023-11-18 20:39:19.373883046 +0000 2023-11-18T20:41:40.251 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2023-11-18 20:39:19.373883046 +0000 2023-11-18T20:41:40.251 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2023-11-18 20:39:19.373883046 +0000 2023-11-18T20:41:40.251 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-18T20:41:40.316 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2023-11-18T20:41:40.316 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2023-11-18T20:41:40.316 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000194403 s, 2.6 MB/s 2023-11-18T20:41:40.317 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-18T20:41:40.373 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-18T20:41:40.373 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-18T20:41:40.373 INFO:tasks.ceph:Generating config... 2023-11-18T20:41:40.374 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-18T20:41:40.375 INFO:tasks.ceph:[client] debug client = 20 2023-11-18T20:41:40.375 INFO:tasks.ceph:[client] debug ms = 1 2023-11-18T20:41:40.375 INFO:tasks.ceph:[client] pid file = 2023-11-18T20:41:40.375 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-18T20:41:40.375 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-18T20:41:40.375 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-18T20:41:40.375 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-18T20:41:40.375 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-18T20:41:40.375 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-18T20:41:40.375 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-18T20:41:40.375 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-18T20:41:40.376 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-18T20:41:40.376 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-18T20:41:40.376 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-18T20:41:40.376 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-18T20:41:40.376 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-18T20:41:40.376 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-18T20:41:40.376 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-18T20:41:40.376 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-18T20:41:40.376 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-18T20:41:40.376 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-18T20:41:40.376 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-18T20:41:40.376 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-18T20:41:40.376 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-18T20:41:40.377 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-18T20:41:40.378 INFO:tasks.ceph:Setting up mon.a... 2023-11-18T20:41:40.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-18T20:41:40.432 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.keyring 2023-11-18T20:41:40.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-18T20:41:40.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-18T20:41:40.518 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.60'), ('mon.b', '172.21.15.100'), ('mon.c', '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]')] 2023-11-18T20:41:40.519 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-18T20:41:40.519 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:40.520 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-18T20:41:40.573 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.60 --add b 172.21.15.100 --addv c '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:41:40.658 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:41:40.658 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: generated fsid ae4682cb-f89a-4edd-b6d6-55eae03fd1b8 2023-11-18T20:41:40.658 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = pacific 2023-11-18T20:41:40.658 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 0 2023-11-18T20:41:40.659 INFO:teuthology.orchestra.run.smithi060.stdout:fsid ae4682cb-f89a-4edd-b6d6-55eae03fd1b8 2023-11-18T20:41:40.659 INFO:teuthology.orchestra.run.smithi060.stdout:last_changed 2023-11-18T20:41:40.658100+0000 2023-11-18T20:41:40.659 INFO:teuthology.orchestra.run.smithi060.stdout:created 2023-11-18T20:41:40.658100+0000 2023-11-18T20:41:40.659 INFO:teuthology.orchestra.run.smithi060.stdout:min_mon_release 16 (pacific) 2023-11-18T20:41:40.659 INFO:teuthology.orchestra.run.smithi060.stdout:election_strategy: 1 2023-11-18T20:41:40.659 INFO:teuthology.orchestra.run.smithi060.stdout:0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2023-11-18T20:41:40.659 INFO:teuthology.orchestra.run.smithi060.stdout:1: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.b 2023-11-18T20:41:40.659 INFO:teuthology.orchestra.run.smithi060.stdout:2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2023-11-18T20:41:40.659 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-18T20:41:40.660 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-18T20:41:40.675 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ae4682cb-f89a-4edd-b6d6-55eae03fd1b8... 2023-11-18T20:41:40.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-18T20:41:40.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-18T20:41:40.768 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2023-11-18T20:41:40.769 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2023-11-18T20:41:40.770 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.771 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = ae4682cb-f89a-4edd-b6d6-55eae03fd1b8 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2023-11-18T20:41:40.772 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.773 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2023-11-18T20:41:40.774 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2023-11-18T20:41:40.775 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2023-11-18T20:41:40.776 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.777 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2023-11-18T20:41:40.778 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2023-11-18T20:41:40.779 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = True 2023-11-18T20:41:40.780 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2023-11-18T20:41:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = True 2023-11-18T20:41:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2023-11-18T20:41:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2023-11-18T20:41:40.781 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2023-11-18T20:41:40.781 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.782 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2023-11-18T20:41:40.783 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = ae4682cb-f89a-4edd-b6d6-55eae03fd1b8 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2023-11-18T20:41:40.785 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2023-11-18T20:41:40.786 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug trim objects = true 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2023-11-18T20:41:40.787 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2023-11-18T20:41:40.788 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-11-18T20:41:40.789 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2023-11-18T20:41:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2023-11-18T20:41:40.791 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2023-11-18T20:41:40.792 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2023-11-18T20:41:40.793 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2023-11-18T20:41:40.794 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2023-11-18T20:41:40.796 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-18T20:41:40.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-18T20:41:40.855 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-18T20:41:40.855 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:40.855 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-18T20:41:40.870 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:40.870 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-18T20:41:40.927 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:41:40.927 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:40.927 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-18T20:41:40.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-18T20:41:41.013 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:41:41.013 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:41:41.068 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:41:41.068 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:41:41.068 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-18T20:41:41.068 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-18T20:41:41.107 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:41:41.107 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:41:41.162 INFO:tasks.ceph:Setting up mon nodes... 2023-11-18T20:41:41.162 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-18T20:41:41.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-18T20:41:41.224 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-18T20:41:41.227 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-18T20:41:41.306 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-18T20:41:41.308 INFO:tasks.ceph:Setting up mds nodes... 2023-11-18T20:41:41.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-18T20:41:41.371 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-18T20:41:41.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-18T20:41:41.404 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-18T20:41:41.507 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-18T20:41:41.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-18T20:41:41.541 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-18T20:41:41.596 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-18T20:41:41.598 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-18T20:41:41.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-18T20:41:41.719 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-18T20:41:41.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-18T20:41:41.747 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-18T20:41:41.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-18T20:41:41.796 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-18T20:41:41.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-18T20:41:41.861 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-18T20:41:41.880 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-18T20:41:41.922 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-18T20:41:41.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-18T20:41:42.019 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-18T20:41:42.034 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-18T20:41:42.034 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-18T20:41:42.035 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-18T20:41:42.066 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-18T20:41:42.066 INFO:tasks.ceph:role: osd.0 2023-11-18T20:41:42.066 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:41:42.066 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-18T20:41:42.136 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-18T20:41:42.136 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-18T20:41:42.136 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-18T20:41:42.137 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-18T20:41:42.137 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-18T20:41:42.137 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2023-11-18T20:41:42.137 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-18T20:41:42.137 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-18T20:41:42.137 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-18T20:41:42.137 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-18T20:41:42.172 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2023-11-18T20:41:42.173 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2023-11-18T20:41:42.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-18T20:41:42.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-18T20:41:42.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-18T20:41:42.400 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-18T20:41:42.400 INFO:tasks.ceph:role: osd.1 2023-11-18T20:41:42.400 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:41:42.401 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-18T20:41:42.471 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-18T20:41:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-18T20:41:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-18T20:41:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-18T20:41:42.471 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-18T20:41:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2023-11-18T20:41:42.472 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-18T20:41:42.472 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-18T20:41:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-18T20:41:42.472 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-18T20:41:42.507 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2023-11-18T20:41:42.509 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2023-11-18T20:41:42.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-18T20:41:42.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-18T20:41:42.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-18T20:41:42.734 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-18T20:41:42.735 INFO:tasks.ceph:role: osd.2 2023-11-18T20:41:42.735 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:41:42.735 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-18T20:41:42.805 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-18T20:41:42.805 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-18T20:41:42.805 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-18T20:41:42.805 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-18T20:41:42.805 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-18T20:41:42.805 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2023-11-18T20:41:42.805 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-18T20:41:42.806 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-18T20:41:42.806 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-18T20:41:42.806 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-18T20:41:42.840 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2023-11-18T20:41:42.842 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2023-11-18T20:41:42.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-18T20:41:42.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-18T20:41:42.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-18T20:41:43.062 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-18T20:41:43.063 INFO:tasks.ceph:role: osd.3 2023-11-18T20:41:43.063 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:41:43.063 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-18T20:41:43.133 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-18T20:41:43.133 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-18T20:41:43.133 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-18T20:41:43.133 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-18T20:41:43.133 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-18T20:41:43.133 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2023-11-18T20:41:43.133 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-18T20:41:43.133 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-18T20:41:43.133 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-18T20:41:43.133 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-18T20:41:43.168 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2023-11-18T20:41:43.170 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2023-11-18T20:41:43.170 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-18T20:41:43.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-18T20:41:43.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:41:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:43.414+0000 7f9df1b24500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-18T20:41:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:43.414+0000 7f9df1b24500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-18T20:41:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:43.414+0000 7f9df1b24500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-18T20:41:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:43.414+0000 7f9df1b24500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-18T20:41:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:43.414+0000 7f9df1b24500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-18T20:41:43.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:43.414+0000 7f9df1b24500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-18T20:41:46.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-18T20:41:46.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:41:46.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:46.392+0000 7f56e921a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-18T20:41:46.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:46.392+0000 7f56e921a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-18T20:41:46.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:46.392+0000 7f56e921a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-18T20:41:46.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:46.392+0000 7f56e921a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-18T20:41:46.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:46.392+0000 7f56e921a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-18T20:41:46.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:46.392+0000 7f56e921a500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-18T20:41:49.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-18T20:41:49.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:41:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:49.379+0000 7f54af016500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-18T20:41:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:49.379+0000 7f54af016500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-18T20:41:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:49.379+0000 7f54af016500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-18T20:41:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:49.379+0000 7f54af016500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-18T20:41:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:49.379+0000 7f54af016500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-18T20:41:49.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:49.380+0000 7f54af016500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-18T20:41:52.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-18T20:41:52.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:41:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:52.355+0000 7f9b30a0e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-18T20:41:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:52.356+0000 7f9b30a0e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-18T20:41:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:52.356+0000 7f9b30a0e500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-18T20:41:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:52.356+0000 7f9b30a0e500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-18T20:41:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:52.356+0000 7f9b30a0e500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-18T20:41:52.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:41:52.356+0000 7f9b30a0e500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-18T20:41:55.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-18T20:41:55.242 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-18T20:41:55.273 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-18T20:41:55.274 INFO:tasks.ceph:role: osd.4 2023-11-18T20:41:55.274 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:41:55.274 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-18T20:41:55.352 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-18T20:41:55.352 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-18T20:41:55.352 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-18T20:41:55.352 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-18T20:41:55.352 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-18T20:41:55.352 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2023-11-18T20:41:55.352 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-18T20:41:55.352 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-18T20:41:55.352 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-18T20:41:55.352 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-18T20:41:55.832 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2023-11-18T20:41:55.833 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2023-11-18T20:41:55.834 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-18T20:41:55.932 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-18T20:41:55.968 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-18T20:41:56.036 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-18T20:41:56.036 INFO:tasks.ceph:role: osd.5 2023-11-18T20:41:56.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:41:56.036 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-18T20:41:56.111 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-18T20:41:56.112 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-18T20:41:56.112 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-18T20:41:56.112 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-18T20:41:56.112 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-18T20:41:56.112 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2023-11-18T20:41:56.112 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-18T20:41:56.113 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-18T20:41:56.113 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-18T20:41:56.113 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-18T20:41:56.569 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2023-11-18T20:41:56.571 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2023-11-18T20:41:56.571 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-18T20:41:56.645 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-18T20:41:56.715 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-18T20:41:56.782 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-18T20:41:56.782 INFO:tasks.ceph:role: osd.6 2023-11-18T20:41:56.782 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:41:56.783 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-18T20:41:56.856 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-18T20:41:56.856 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-18T20:41:56.856 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-18T20:41:56.856 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-18T20:41:56.856 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-18T20:41:56.857 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2023-11-18T20:41:56.857 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-18T20:41:56.857 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-18T20:41:56.857 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-18T20:41:56.857 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-18T20:41:57.341 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2023-11-18T20:41:57.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2023-11-18T20:41:57.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-18T20:41:57.441 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-18T20:41:57.474 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-18T20:41:57.542 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-18T20:41:57.542 INFO:tasks.ceph:role: osd.7 2023-11-18T20:41:57.542 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:41:57.542 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-18T20:41:57.617 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-18T20:41:57.617 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-18T20:41:57.617 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-18T20:41:57.617 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-18T20:41:57.617 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-18T20:41:57.617 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2023-11-18T20:41:57.617 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-18T20:41:57.617 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-18T20:41:57.617 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-18T20:41:57.618 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-18T20:41:58.112 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2023-11-18T20:41:58.114 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2023-11-18T20:41:58.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-18T20:41:58.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-18T20:41:58.286 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:41:58.387 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:41:58.384+0000 7fdd1fc2b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-18T20:41:58.387 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:41:58.385+0000 7fdd1fc2b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-18T20:41:58.387 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:41:58.385+0000 7fdd1fc2b500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-18T20:41:58.387 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:41:58.385+0000 7fdd1fc2b500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-18T20:41:58.387 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:41:58.385+0000 7fdd1fc2b500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-18T20:41:58.391 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:41:58.390+0000 7fdd1fc2b500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-18T20:42:01.242 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-18T20:42:01.273 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:42:01.364 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:01.363+0000 7f9734414500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-18T20:42:01.365 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:01.363+0000 7f9734414500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-18T20:42:01.365 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:01.363+0000 7f9734414500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-18T20:42:01.365 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:01.363+0000 7f9734414500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-18T20:42:01.365 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:01.363+0000 7f9734414500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-18T20:42:01.365 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:01.364+0000 7f9734414500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-18T20:42:04.223 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-18T20:42:04.253 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:42:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:04.343+0000 7f9a55939500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-18T20:42:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:04.343+0000 7f9a55939500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-18T20:42:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:04.343+0000 7f9a55939500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-18T20:42:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:04.343+0000 7f9a55939500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-18T20:42:04.344 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:04.343+0000 7f9a55939500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-18T20:42:04.345 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:04.343+0000 7f9a55939500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-18T20:42:07.196 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-18T20:42:07.226 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:42:07.317 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:07.315+0000 7fcc7aa25500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-18T20:42:07.317 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:07.316+0000 7fcc7aa25500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-18T20:42:07.317 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:07.316+0000 7fcc7aa25500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-18T20:42:07.317 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:07.316+0000 7fcc7aa25500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-18T20:42:07.317 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:07.316+0000 7fcc7aa25500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-18T20:42:07.317 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:07.316+0000 7fcc7aa25500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-18T20:42:08.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:42:08.770 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:42:10.184 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-18T20:42:10.217 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-18T20:42:10.217 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:10.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-18T20:42:10.253 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:10.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-18T20:42:10.329 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:10.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-18T20:42:10.402 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:10.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-18T20:42:10.476 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:10.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-18T20:42:10.549 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:10.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-18T20:42:10.621 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:10.621 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-18T20:42:10.692 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:10.692 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-18T20:42:10.723 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:10.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-18T20:42:10.794 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:10.794 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-18T20:42:10.865 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:10.865 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-18T20:42:10.934 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:10.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-18T20:42:11.001 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:11.002 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-18T20:42:11.071 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:11.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-18T20:42:11.139 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:11.139 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-18T20:42:11.157 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:11.157 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-18T20:42:11.215 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:11.215 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-18T20:42:11.230 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:11.230 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-18T20:42:11.287 INFO:tasks.ceph:Adding keys to all mons... 2023-11-18T20:42:11.288 DEBUG:teuthology.orchestra.run.smithi060:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-18T20:42:11.290 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-18T20:42:11.330 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.y] 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllkJ9gDRAAiqXLdSkKHV4SjIPBOA85YA== 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllC9UnFhAACm0q2kkeuVd7s/HDJ4HxsQ== 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllaDI6HhAARs+hK3PNr6Pw7YXSHrks1g== 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.0] 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCHIVlli+6+GBAAF9u74MMa50Abf4XBRswFpQ== 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.1] 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCKIVlln59wFxAAiqokbWoD901iRvF5ESqEbw== 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.2] 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCNIVll3XquFhAA1zESpR80pRgZBgAC9U3aDw== 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.3] 2023-11-18T20:42:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCQIVllMbZDFRAAVEOBFdVCedzteYM1w/enOA== 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.x] 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVlljlFBEhAAQDo8Xz2m+fCZpazdjrBOfQ== 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVlluYuEIxAAQk5LaZvlSTcwf9UNsO5a9Q== 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllFO7dKhAAXNdgA3EauG4YWKi99qLLRg== 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.4] 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCWIVllnl4LFxAAbiNZODS0LQROAz7MfPIo8A== 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.5] 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCZIVllTPeyFRAAB/IGUS8d7KCVaEqyKA6e3g== 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.6] 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCcIVllMIJ/FBAA7Wn5sn/6BBl+68DOSpAR5A== 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.7] 2023-11-18T20:42:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCfIVllIAfgEhAAcOZ3Z5ZmYzIzqao5jmGb7A== 2023-11-18T20:42:11.333 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-11-18T20:42:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA== 2023-11-18T20:42:11.333 INFO:teuthology.orchestra.run.smithi060.stdout:[client.2] 2023-11-18T20:42:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVlltRRZMxAALjGvfOl1LnaZj/6RaATH4w== 2023-11-18T20:42:11.333 INFO:teuthology.orchestra.run.smithi060.stdout:[client.1] 2023-11-18T20:42:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllyxjzNhAAYt+5RXeMr8fnEZeiWX3b/w== 2023-11-18T20:42:11.333 INFO:teuthology.orchestra.run.smithi060.stdout:[client.3] 2023-11-18T20:42:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCGIVllI90eARAAOKYZObsxr760sxu2J+xy0Q== 2023-11-18T20:42:11.353 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2023-11-18T20:42:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCFIVllkJ9gDRAAiqXLdSkKHV4SjIPBOA85YA== 2023-11-18T20:42:11.353 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2023-11-18T20:42:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCFIVllC9UnFhAACm0q2kkeuVd7s/HDJ4HxsQ== 2023-11-18T20:42:11.353 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2023-11-18T20:42:11.353 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCFIVllaDI6HhAARs+hK3PNr6Pw7YXSHrks1g== 2023-11-18T20:42:11.353 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCHIVlli+6+GBAAF9u74MMa50Abf4XBRswFpQ== 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCKIVlln59wFxAAiqokbWoD901iRvF5ESqEbw== 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCNIVll3XquFhAA1zESpR80pRgZBgAC9U3aDw== 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCQIVllMbZDFRAAVEOBFdVCedzteYM1w/enOA== 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCFIVlljlFBEhAAQDo8Xz2m+fCZpazdjrBOfQ== 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCFIVlluYuEIxAAQk5LaZvlSTcwf9UNsO5a9Q== 2023-11-18T20:42:11.354 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCFIVllFO7dKhAAXNdgA3EauG4YWKi99qLLRg== 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCWIVllnl4LFxAAbiNZODS0LQROAz7MfPIo8A== 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCZIVllTPeyFRAAB/IGUS8d7KCVaEqyKA6e3g== 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCcIVllMIJ/FBAA7Wn5sn/6BBl+68DOSpAR5A== 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCfIVllIAfgEhAAcOZ3Z5ZmYzIzqao5jmGb7A== 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA== 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout:[client.2] 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCFIVlltRRZMxAALjGvfOl1LnaZj/6RaATH4w== 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout:[client.1] 2023-11-18T20:42:11.355 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCFIVllyxjzNhAAYt+5RXeMr8fnEZeiWX3b/w== 2023-11-18T20:42:11.356 INFO:teuthology.orchestra.run.smithi100.stdout:[client.3] 2023-11-18T20:42:11.356 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCGIVllI90eARAAOKYZObsxr760sxu2J+xy0Q== 2023-11-18T20:42:11.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-18T20:42:11.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-18T20:42:11.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-18T20:42:11.451 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-18T20:42:11.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-18T20:42:11.511 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-18T20:42:11.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:11.570 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:11.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:11.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:11.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:11.681 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:11.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:11.738 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:11.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-18T20:42:11.793 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-18T20:42:11.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-18T20:42:11.850 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-18T20:42:11.907 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-18T20:42:11.909 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-18T20:42:11.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:11.968 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:12.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:12.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:12.078 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:12.080 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:12.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:12.136 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-18T20:42:12.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-18T20:42:12.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-18T20:42:12.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-18T20:42:12.251 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-18T20:42:12.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-18T20:42:12.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-18T20:42:12.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-18T20:42:12.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-18T20:42:12.426 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-18T20:42:12.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-18T20:42:12.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-18T20:42:12.557 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-18T20:42:12.587 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-18T20:42:12.614 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-18T20:42:12.711 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-18T20:42:12.737 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-18T20:42:12.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-18T20:42:12.900 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-18T20:42:12.927 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:42:12.930 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-18T20:42:12.981 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-18T20:42:12.981 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-18T20:42:12.982 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-18T20:42:12.985 INFO:tasks.ceph.mon.a:Started 2023-11-18T20:42:12.985 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-18T20:42:12.985 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-18T20:42:13.023 INFO:tasks.ceph.mon.b:Started 2023-11-18T20:42:13.023 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-18T20:42:13.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-18T20:42:13.025 INFO:tasks.ceph.mon.c:Started 2023-11-18T20:42:13.026 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-18T20:42:13.026 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-18T20:42:13.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-18T20:42:13.028 INFO:tasks.ceph.mgr.y:Started 2023-11-18T20:42:13.028 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-18T20:42:13.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-18T20:42:13.030 INFO:tasks.ceph.mgr.x:Started 2023-11-18T20:42:13.030 INFO:tasks.ceph:Setting crush tunables to default 2023-11-18T20:42:13.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd crush tunables default 2023-11-18T20:42:13.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.136+0000 7f535ec34640 1 Processor -- start 2023-11-18T20:42:13.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.136+0000 7f535ec34640 1 -- start start 2023-11-18T20:42:13.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.137+0000 7f535ec34640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f535806abe0 0x7f535815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:13.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.137+0000 7f535ec34640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5358143680 0x7f53581409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:13.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.137+0000 7f535ec34640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535806e960 con 0x7f53581432a0 2023-11-18T20:42:13.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.137+0000 7f535ec34640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535815dec0 con 0x7f5358143680 2023-11-18T20:42:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.137+0000 7f535ec34640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535815e1a0 con 0x7f535806abe0 2023-11-18T20:42:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.137+0000 7f535d1d1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5358143680 0x7f53581409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.137+0000 7f535d1d1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5358143680 0x7f53581409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49136/0 (socket says 172.21.15.60:49136) 2023-11-18T20:42:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.137+0000 7f535d1d1640 1 -- 172.21.15.60:0/3850923674 learned_addr learned my addr 172.21.15.60:0/3850923674 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.138+0000 7f535d1d1640 1 -- 172.21.15.60:0/3850923674 >> v2:172.21.15.60:3300/0 conn(0x7f5358143680 msgr2=0x7f53581409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-18T20:42:13.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.138+0000 7f535d1d1640 1 -- 172.21.15.60:0/3850923674 >> v2:172.21.15.60:3300/0 conn(0x7f5358143680 msgr2=0x7f53581409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-18T20:42:13.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.138+0000 7f535d1d1640 1 --2- 172.21.15.60:0/3850923674 >> v2:172.21.15.60:3300/0 conn(0x7f5358143680 0x7f53581409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-18T20:42:13.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.138+0000 7f535d1d1640 1 --2- 172.21.15.60:0/3850923674 >> v2:172.21.15.60:3300/0 conn(0x7f5358143680 0x7f53581409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-18T20:42:13.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.138+0000 7f5357fff640 1 --2- 172.21.15.60:0/3850923674 >> v2:172.21.15.100:3300/0 conn(0x7f535806abe0 0x7f535815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:13.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.138+0000 7f5357fff640 1 -- 172.21.15.60:0/3850923674 >> v2:172.21.15.100:3300/0 conn(0x7f535806abe0 msgr2=0x7f535815d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-18T20:42:13.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.138+0000 7f5357fff640 1 -- 172.21.15.60:0/3850923674 >> v2:172.21.15.100:3300/0 conn(0x7f535806abe0 msgr2=0x7f535815d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-18T20:42:13.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.138+0000 7f5357fff640 1 --2- 172.21.15.60:0/3850923674 >> v2:172.21.15.100:3300/0 conn(0x7f535806abe0 0x7f535815d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-18T20:42:13.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.138+0000 7f5357fff640 1 --2- 172.21.15.60:0/3850923674 >> v2:172.21.15.100:3300/0 conn(0x7f535806abe0 0x7f535815d8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-18T20:42:13.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.331+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878669803 0 0) 0x7f535806e960 con 0x7f53581432a0 2023-11-18T20:42:13.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.331+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5340003520 con 0x7f53581432a0 2023-11-18T20:42:13.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2364478401 0 0) 0x7f5340003520 con 0x7f53581432a0 2023-11-18T20:42:13.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 >> v2:172.21.15.100:3300/0 conn(0x7f535806abe0 msgr2=0x7f535815d8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-18T20:42:13.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 --2- 172.21.15.60:0/3850923674 >> v2:172.21.15.100:3300/0 conn(0x7f535806abe0 0x7f535815d8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 >> v2:172.21.15.60:3300/0 conn(0x7f5358143680 msgr2=0x7f53581409d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 --2- 172.21.15.60:0/3850923674 >> v2:172.21.15.60:3300/0 conn(0x7f5358143680 0x7f53581409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535815e480 con 0x7f53581432a0 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f5348001670 con 0x7f53581432a0 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 >> v1:172.21.15.100:6789/0 conn(0x7f53581432a0 legacy=0x7f535806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5340003970 0x7f5340003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53400047c0 0x7f5340004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 0x7f534000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535815dec0 con 0x7f5340005180 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535815e1a0 con 0x7f53400047c0 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534000c130 con 0x7f5340003970 2023-11-18T20:42:13.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f535d1d1640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 0x7f534000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:13.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f5357fff640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53400047c0 0x7f5340004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:13.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f535c9d0640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5340003970 0x7f5340003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:13.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f535d1d1640 1 -- 172.21.15.60:0/3850923674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5340003970 msgr2=0x7f5340003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:13.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f535d1d1640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5340003970 0x7f5340003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f535d1d1640 1 -- 172.21.15.60:0/3850923674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53400047c0 msgr2=0x7f5340004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:13.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f535d1d1640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53400047c0 0x7f5340004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.332+0000 7f535d1d1640 1 -- 172.21.15.60:0/3850923674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5340003520 con 0x7f5340005180 2023-11-18T20:42:13.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.334+0000 7f535d1d1640 1 -- 172.21.15.60:0/3850923674 reap_dead start 2023-11-18T20:42:13.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.334+0000 7f535d1d1640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 0x7f534000b980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5350009cc0 tx=0x7f53400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890096c2e7547a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:13.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.334+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535002cb10 con 0x7f5340005180 2023-11-18T20:42:13.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.334+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350030070 con 0x7f5340005180 2023-11-18T20:42:13.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.334+0000 7f53577fe640 1 -- 172.21.15.60:0/3850923674 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350036410 con 0x7f5340005180 2023-11-18T20:42:13.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.335+0000 7f535ec34640 1 -- 172.21.15.60:0/3850923674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 msgr2=0x7f534000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:13.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.335+0000 7f535ec34640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 0x7f534000b980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5350009cc0 tx=0x7f53400057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.335+0000 7f535ec34640 1 -- 172.21.15.60:0/3850923674 shutdown_connections 2023-11-18T20:42:13.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.335+0000 7f535ec34640 1 --2- 172.21.15.60:0/3850923674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 0x7f534000b980 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.335+0000 7f535ec34640 1 -- 172.21.15.60:0/3850923674 >> 172.21.15.60:0/3850923674 conn(0x7f53580b1de0 msgr2=0x7f53580b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:13.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.335+0000 7f535ec34640 1 -- 172.21.15.60:0/3850923674 shutdown_connections 2023-11-18T20:42:13.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.335+0000 7f535ec34640 1 -- 172.21.15.60:0/3850923674 wait complete. 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.336+0000 7f535ec34640 1 Processor -- start 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.336+0000 7f535ec34640 1 -- start start 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f535ec34640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 0x7f5358132160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f535ec34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5358141c70 0x7f5358155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f535ec34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53581420f0 0x7f535815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f535ec34640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358155e60 con 0x7f5340005180 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f535ec34640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358156140 con 0x7f53581420f0 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f535ec34640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5358156420 con 0x7f5358141c70 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f5357fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5358141c70 0x7f5358155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f535d1d1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53581420f0 0x7f535815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:13.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f535c9d0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 0x7f5358132160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f5357fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5358141c70 0x7f5358155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59344/0 (socket says 172.21.15.60:59344) 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f5357fff640 1 -- 172.21.15.60:0/851280829 learned_addr learned my addr 172.21.15.60:0/851280829 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f5357fff640 1 -- 172.21.15.60:0/851280829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53581420f0 msgr2=0x7f535815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.337+0000 7f5357fff640 1 --2- 172.21.15.60:0/851280829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53581420f0 0x7f535815c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.338+0000 7f5357fff640 1 -- 172.21.15.60:0/851280829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 msgr2=0x7f5358132160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.338+0000 7f5357fff640 1 --2- 172.21.15.60:0/851280829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 0x7f5358132160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.338+0000 7f5357fff640 1 -- 172.21.15.60:0/851280829 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5358156700 con 0x7f5358141c70 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.338+0000 7f535c9d0640 1 --2- 172.21.15.60:0/851280829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 0x7f5358132160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.338+0000 7f5357fff640 1 --2- 172.21.15.60:0/851280829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5358141c70 0x7f5358155370 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f534c0108c0 tx=0x7f534c010d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.339+0000 7f53557fa640 1 -- 172.21.15.60:0/851280829 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c00e9c0 con 0x7f5358141c70 2023-11-18T20:42:13.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.339+0000 7f53557fa640 1 -- 172.21.15.60:0/851280829 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c00eb20 con 0x7f5358141c70 2023-11-18T20:42:13.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.339+0000 7f535ec34640 1 -- 172.21.15.60:0/851280829 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53581569e0 con 0x7f5358141c70 2023-11-18T20:42:13.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.339+0000 7f53557fa640 1 -- 172.21.15.60:0/851280829 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c00eed0 con 0x7f5358141c70 2023-11-18T20:42:13.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.339+0000 7f535ec34640 1 -- 172.21.15.60:0/851280829 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53581326a0 con 0x7f5358141c70 2023-11-18T20:42:13.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.339+0000 7f53557fa640 1 -- 172.21.15.60:0/851280829 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f534c016780 con 0x7f5358141c70 2023-11-18T20:42:13.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.340+0000 7f53557fa640 1 -- 172.21.15.60:0/851280829 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f534c014870 con 0x7f5358141c70 2023-11-18T20:42:13.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.340+0000 7f535ec34640 1 -- 172.21.15.60:0/851280829 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5358067420 con 0x7f5358141c70 2023-11-18T20:42:13.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.348+0000 7f53557fa640 1 -- 172.21.15.60:0/851280829 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f534c03ece0 con 0x7f5358141c70 2023-11-18T20:42:13.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.415+0000 7f535ec34640 1 -- 172.21.15.60:0/851280829 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5358156bd0 con 0x7f5358141c70 2023-11-18T20:42:13.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.423+0000 7f53557fa640 1 -- 172.21.15.60:0/851280829 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f534c02d020 con 0x7f5358141c70 2023-11-18T20:42:13.424 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2023-11-18T20:42:13.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.424+0000 7f535ec34640 1 -- 172.21.15.60:0/851280829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5358141c70 msgr2=0x7f5358155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:13.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.424+0000 7f535ec34640 1 --2- 172.21.15.60:0/851280829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5358141c70 0x7f5358155370 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f534c0108c0 tx=0x7f534c010d80 comp rx=0 tx=0).stop 2023-11-18T20:42:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.424+0000 7f535ec34640 1 -- 172.21.15.60:0/851280829 shutdown_connections 2023-11-18T20:42:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.424+0000 7f535ec34640 1 --2- 172.21.15.60:0/851280829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53581420f0 0x7f535815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.424+0000 7f535ec34640 1 --2- 172.21.15.60:0/851280829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5358141c70 0x7f5358155370 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.424+0000 7f535ec34640 1 --2- 172.21.15.60:0/851280829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5340005180 0x7f5358132160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.424+0000 7f535ec34640 1 -- 172.21.15.60:0/851280829 >> 172.21.15.60:0/851280829 conn(0x7f53580b1de0 msgr2=0x7f5358153e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.424+0000 7f535ec34640 1 -- 172.21.15.60:0/851280829 shutdown_connections 2023-11-18T20:42:13.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:13.424+0000 7f535ec34640 1 -- 172.21.15.60:0/851280829 wait complete. 2023-11-18T20:42:13.436 INFO:tasks.ceph:check_enable_crimson: False 2023-11-18T20:42:13.436 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-18T20:42:13.436 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:13.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-18T20:42:13.467 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:13.467 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-18T20:42:13.537 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:13.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-18T20:42:13.608 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:42:13.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-18T20:42:13.678 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:13.679 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-18T20:42:13.705 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:13.705 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-18T20:42:13.772 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:13.772 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-18T20:42:13.798 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:13.797+0000 7f5528c32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-18T20:42:13.824 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:42:13.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-18T20:42:13.848 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:13.847+0000 7f34bfa2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-18T20:42:13.852 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:13.851+0000 7f5528c32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-18T20:42:13.878 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new c9a2cddd-47db-4b41-a027-3a80d0dd460f 0 2023-11-18T20:42:13.902 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:13.901+0000 7f34bfa2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-18T20:42:14.018 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.017+0000 7ff7cae21640 1 Processor -- start 2023-11-18T20:42:14.018 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.017+0000 7ff7cae21640 1 -- start start 2023-11-18T20:42:14.019 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7cae21640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c406abe0 0x7ff7c4161940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7cae21640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff7c40ad740 0x7ff7c40b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7cae21640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c4071b00 con 0x7ff7c4146d40 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7cae21640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c4147120 con 0x7ff7c40ad740 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7cae21640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4161e80 con 0x7ff7c406abe0 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7c8bbd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff7c4146d40 0x7ff7c406a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35182/0 (socket says 172.21.15.100:35182) 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7c93be640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff7c40ad740 0x7ff7c40b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7c3fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c406abe0 0x7ff7c4161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7c8bbd640 1 -- 172.21.15.100:0/1509974939 learned_addr learned my addr 172.21.15.100:0/1509974939 (peer_addr_for_me v1:172.21.15.100:0/0) 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7c93be640 1 -- 172.21.15.100:0/1509974939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c406abe0 msgr2=0x7ff7c4161940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7c93be640 1 --2- 172.21.15.100:0/1509974939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c406abe0 0x7ff7c4161940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.020 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7c93be640 1 -- 172.21.15.100:0/1509974939 >> v1:172.21.15.100:6789/0 conn(0x7ff7c4146d40 legacy=0x7ff7c406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.021 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7c93be640 1 -- 172.21.15.100:0/1509974939 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c4071de0 con 0x7ff7c40ad740 2023-11-18T20:42:14.021 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7c37fe640 1 -- 172.21.15.100:0/1509974939 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845595567 0 0) 0x7ff7c4071b00 con 0x7ff7c4146d40 2023-11-18T20:42:14.021 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.018+0000 7ff7c3fff640 1 --2- 172.21.15.100:0/1509974939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c406abe0 0x7ff7c4161940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:14.024 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.019+0000 7ff7c93be640 1 --2- 172.21.15.100:0/1509974939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c40ad740 0x7ff7c40b3330 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009d90 tx=0x7ff7bc02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3d8566710c36a9f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:14.024 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.019+0000 7ff7c37fe640 1 -- 172.21.15.100:0/1509974939 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc02cbc0 con 0x7ff7c40ad740 2023-11-18T20:42:14.024 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.019+0000 7ff7c37fe640 1 -- 172.21.15.100:0/1509974939 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc02cd20 con 0x7ff7c40ad740 2023-11-18T20:42:14.024 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.019+0000 7ff7c37fe640 1 -- 172.21.15.100:0/1509974939 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc034660 con 0x7ff7c40ad740 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.019+0000 7ff7cae21640 1 -- 172.21.15.100:0/1509974939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c40ad740 msgr2=0x7ff7c40b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.020+0000 7ff7cae21640 1 --2- 172.21.15.100:0/1509974939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c40ad740 0x7ff7c40b3330 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009d90 tx=0x7ff7bc02be10 comp rx=0 tx=0).stop 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.020+0000 7ff7cae21640 1 -- 172.21.15.100:0/1509974939 shutdown_connections 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.020+0000 7ff7cae21640 1 --2- 172.21.15.100:0/1509974939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c40ad740 0x7ff7c40b3330 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.020+0000 7ff7cae21640 1 --2- 172.21.15.100:0/1509974939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c406abe0 0x7ff7c4161940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.020+0000 7ff7cae21640 1 -- 172.21.15.100:0/1509974939 >> 172.21.15.100:0/1509974939 conn(0x7ff7c4066270 msgr2=0x7ff7c4066680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.020+0000 7ff7cae21640 1 -- 172.21.15.100:0/1509974939 shutdown_connections 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.020+0000 7ff7cae21640 1 -- 172.21.15.100:0/1509974939 wait complete. 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.021+0000 7ff7cae21640 1 Processor -- start 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.021+0000 7ff7cae21640 1 -- start start 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7cae21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c406abe0 0x7ff7c40ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.025 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7cae21640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7c40ad740 0x7ff7c4132be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7cae21640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c4146d40 0x7ff7c415cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7cae21640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4133120 con 0x7ff7c40ad740 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7cae21640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4130160 con 0x7ff7c406abe0 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7cae21640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40cf020 con 0x7ff7c4146d40 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c8bbd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c406abe0 0x7ff7c40ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c93be640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c4146d40 0x7ff7c415cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c8bbd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c406abe0 0x7ff7c40ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37234/0 (socket says 172.21.15.100:37234) 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c8bbd640 1 -- 172.21.15.100:0/3245908724 learned_addr learned my addr 172.21.15.100:0/3245908724 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c3fff640 1 --2- 172.21.15.100:0/3245908724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7c40ad740 0x7ff7c4132be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c93be640 1 -- 172.21.15.100:0/3245908724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c406abe0 msgr2=0x7ff7c40ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.026 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c93be640 1 --2- 172.21.15.100:0/3245908724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c406abe0 0x7ff7c40ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c93be640 1 -- 172.21.15.100:0/3245908724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7c40ad740 msgr2=0x7ff7c4132be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c93be640 1 --2- 172.21.15.100:0/3245908724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7c40ad740 0x7ff7c4132be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c93be640 1 -- 172.21.15.100:0/3245908724 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b0009400 con 0x7ff7c4146d40 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c8bbd640 1 --2- 172.21.15.100:0/3245908724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c406abe0 0x7ff7c40ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c93be640 1 --2- 172.21.15.100:0/3245908724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c4146d40 0x7ff7c415cc40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc033000 tx=0x7ff7bc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c17fa640 1 -- 172.21.15.100:0/3245908724 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc040650 con 0x7ff7c4146d40 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.022+0000 7ff7c17fa640 1 -- 172.21.15.100:0/3245908724 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc02ce90 con 0x7ff7c4146d40 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.023+0000 7ff7cae21640 1 -- 172.21.15.100:0/3245908724 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc009a70 con 0x7ff7c4146d40 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.023+0000 7ff7c17fa640 1 -- 172.21.15.100:0/3245908724 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0409f0 con 0x7ff7c4146d40 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.023+0000 7ff7cae21640 1 -- 172.21.15.100:0/3245908724 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c40cf5e0 con 0x7ff7c4146d40 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.023+0000 7ff7c17fa640 1 -- 172.21.15.100:0/3245908724 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff7bc049400 con 0x7ff7c4146d40 2023-11-18T20:42:14.027 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.023+0000 7ff7c17fa640 1 -- 172.21.15.100:0/3245908724 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff7bc03e400 con 0x7ff7c4146d40 2023-11-18T20:42:14.028 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.024+0000 7ff7cae21640 1 -- 172.21.15.100:0/3245908724 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c41302d0 con 0x7ff7c4146d40 2023-11-18T20:42:14.028 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.026+0000 7ff7c17fa640 1 -- 172.21.15.100:0/3245908724 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff7bc044020 con 0x7ff7c4146d40 2023-11-18T20:42:14.098 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.097+0000 7ff7cae21640 1 -- 172.21.15.100:0/3245908724 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c9a2cddd-47db-4b41-a027-3a80d0dd460f", "id": 0} v 0) v1 -- 0x7ff7c4067420 con 0x7ff7c4146d40 2023-11-18T20:42:14.101 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.100+0000 7ff7c17fa640 1 -- 172.21.15.100:0/3245908724 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c9a2cddd-47db-4b41-a027-3a80d0dd460f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff7bc0570b0 con 0x7ff7c4146d40 2023-11-18T20:42:14.102 INFO:teuthology.orchestra.run.smithi100.stdout:0 2023-11-18T20:42:14.103 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.101+0000 7ff7cae21640 1 -- 172.21.15.100:0/3245908724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c4146d40 msgr2=0x7ff7c415cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.103 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.101+0000 7ff7cae21640 1 --2- 172.21.15.100:0/3245908724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c4146d40 0x7ff7c415cc40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc033000 tx=0x7ff7bc004a40 comp rx=0 tx=0).stop 2023-11-18T20:42:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.102+0000 7ff7cae21640 1 -- 172.21.15.100:0/3245908724 shutdown_connections 2023-11-18T20:42:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.102+0000 7ff7cae21640 1 --2- 172.21.15.100:0/3245908724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7c4146d40 0x7ff7c415cc40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.102+0000 7ff7cae21640 1 --2- 172.21.15.100:0/3245908724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7c40ad740 0x7ff7c4132be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.102+0000 7ff7cae21640 1 --2- 172.21.15.100:0/3245908724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c406abe0 0x7ff7c40ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.102+0000 7ff7cae21640 1 -- 172.21.15.100:0/3245908724 >> 172.21.15.100:0/3245908724 conn(0x7ff7c4066270 msgr2=0x7ff7c40aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.102+0000 7ff7cae21640 1 -- 172.21.15.100:0/3245908724 shutdown_connections 2023-11-18T20:42:14.105 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.102+0000 7ff7cae21640 1 -- 172.21.15.100:0/3245908724 wait complete. 2023-11-18T20:42:14.117 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 3ac66b30-eb29-41c3-bbf5-c9d2463d00b2 1 2023-11-18T20:42:14.217 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.216+0000 7f7705ee7640 1 Processor -- start 2023-11-18T20:42:14.217 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.216+0000 7f7705ee7640 1 -- start start 2023-11-18T20:42:14.222 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f7705ee7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f77000cc520 0x7f770006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.222 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f7705ee7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770013e5d0 con 0x7f77000cc0c0 2023-11-18T20:42:14.222 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f7705ee7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77000ce3c0 con 0x7f770013e1f0 2023-11-18T20:42:14.222 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f7705ee7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000ce6a0 con 0x7f77000cc520 2023-11-18T20:42:14.222 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76ff7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f770013e1f0 0x7f77000d0010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35184/0 (socket says 172.21.15.100:35184) 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76ff7fe640 1 -- 172.21.15.100:0/1879101825 learned_addr learned my addr 172.21.15.100:0/1879101825 (peer_addr_for_me v1:172.21.15.100:0/0) 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76fffff640 1 --2- 172.21.15.100:0/1879101825 >> v2:172.21.15.60:3300/0 conn(0x7f77000cc520 0x7f770006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787067072 0 0) 0x7f77000ce3c0 con 0x7f770013e1f0 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76ec003520 con 0x7f770013e1f0 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398466972 0 0) 0x7f770013e5d0 con 0x7f77000cc0c0 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77000ce3c0 con 0x7f77000cc0c0 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 753432656 0 0) 0x7f76ec003520 con 0x7f770013e1f0 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 >> v2:172.21.15.60:3300/0 conn(0x7f77000cc520 msgr2=0x7f770006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76feffd640 1 --2- 172.21.15.100:0/1879101825 >> v2:172.21.15.60:3300/0 conn(0x7f77000cc520 0x7f770006abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.217+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 >> v1:172.21.15.60:6789/0 conn(0x7f77000cc0c0 legacy=0x7f770006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77000ce980 con 0x7f770013e1f0 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f76f0001670 con 0x7f770013e1f0 2023-11-18T20:42:14.223 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 >> v1:172.21.15.100:6789/0 conn(0x7f770013e1f0 legacy=0x7f77000d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76feffd640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76ec003b50 0x7f76ec003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76feffd640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76ec0047c0 0x7f76ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76feffd640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec005180 0x7f76ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000ce6a0 con 0x7f76ec0047c0 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec00c050 con 0x7f76ec003b50 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec00c210 con 0x7f76ec005180 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76fffff640 1 --2- 172.21.15.100:0/1879101825 >> v2:172.21.15.60:3300/0 conn(0x7f77000cc520 0x7f770006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76ff7fe640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76ec003b50 0x7f76ec003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76f6ffd640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76ec0047c0 0x7f76ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76fffff640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec005180 0x7f76ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76ff7fe640 1 -- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec005180 msgr2=0x7f76ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76ff7fe640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec005180 0x7f76ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.224 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76ff7fe640 1 -- 172.21.15.100:0/1879101825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76ec0047c0 msgr2=0x7f76ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76ff7fe640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76ec0047c0 0x7f76ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76ff7fe640 1 -- 172.21.15.100:0/1879101825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec003520 con 0x7f76ec003b50 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76fffff640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec005180 0x7f76ec00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.218+0000 7f76fffff640 1 -- 172.21.15.100:0/1879101825 reap_dead start 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.219+0000 7f76ff7fe640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76ec003b50 0x7f76ec003f50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f76f0007f90 tx=0x7f76ec0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d941d4cdfcf05ce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.219+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0005ad0 con 0x7f76ec003b50 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.219+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f0005c30 con 0x7f76ec003b50 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.219+0000 7f76feffd640 1 -- 172.21.15.100:0/1879101825 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f002c9b0 con 0x7f76ec003b50 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.219+0000 7f7705ee7640 1 -- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76ec003b50 msgr2=0x7f76ec003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.219+0000 7f7705ee7640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76ec003b50 0x7f76ec003f50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f76f0007f90 tx=0x7f76ec0057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.219+0000 7f7705ee7640 1 -- 172.21.15.100:0/1879101825 shutdown_connections 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.219+0000 7f7705ee7640 1 --2- 172.21.15.100:0/1879101825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76ec003b50 0x7f76ec003f50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.225 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.219+0000 7f7705ee7640 1 -- 172.21.15.100:0/1879101825 >> 172.21.15.100:0/1879101825 conn(0x7f77000b1a50 msgr2=0x7f77000cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:14.226 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.219+0000 7f7705ee7640 1 -- 172.21.15.100:0/1879101825 shutdown_connections 2023-11-18T20:42:14.226 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.220+0000 7f7705ee7640 1 -- 172.21.15.100:0/1879101825 wait complete. 2023-11-18T20:42:14.226 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.221+0000 7f7705ee7640 1 Processor -- start 2023-11-18T20:42:14.226 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.221+0000 7f7705ee7640 1 -- start start 2023-11-18T20:42:14.228 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.221+0000 7f7705ee7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec003b50 0x7f770013c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.228 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.221+0000 7f7705ee7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f770013c940 0x7f7700152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.228 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.221+0000 7f7705ee7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7700154620 0x7f7700154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.228 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.221+0000 7f7705ee7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7700154ff0 con 0x7f770013c940 2023-11-18T20:42:14.228 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.221+0000 7f7705ee7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770013c710 con 0x7f7700154620 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f7705ee7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7700152b90 con 0x7f76ec003b50 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76fffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7700154620 0x7f7700154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76ff7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec003b50 0x7f770013c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76ff7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec003b50 0x7f770013c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:57694/0 (socket says 172.21.15.100:57694) 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76ff7fe640 1 -- 172.21.15.100:0/773838701 learned_addr learned my addr 172.21.15.100:0/773838701 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76f6ffd640 1 --2- 172.21.15.100:0/773838701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f770013c940 0x7f7700152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76fffff640 1 -- 172.21.15.100:0/773838701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec003b50 msgr2=0x7f770013c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76fffff640 1 --2- 172.21.15.100:0/773838701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec003b50 0x7f770013c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76fffff640 1 -- 172.21.15.100:0/773838701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f770013c940 msgr2=0x7f7700152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76fffff640 1 --2- 172.21.15.100:0/773838701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f770013c940 0x7f7700152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76fffff640 1 -- 172.21.15.100:0/773838701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77001e3660 con 0x7f7700154620 2023-11-18T20:42:14.229 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76ff7fe640 1 --2- 172.21.15.100:0/773838701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec003b50 0x7f770013c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76f6ffd640 1 --2- 172.21.15.100:0/773838701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f770013c940 0x7f7700152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76fffff640 1 --2- 172.21.15.100:0/773838701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7700154620 0x7f7700154ab0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f77000b4e10 tx=0x7f76f8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76fcff9640 1 -- 172.21.15.100:0/773838701 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800e030 con 0x7f7700154620 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76fcff9640 1 -- 172.21.15.100:0/773838701 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8011e60 con 0x7f7700154620 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f7705ee7640 1 -- 172.21.15.100:0/773838701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77001e3940 con 0x7f7700154620 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f76fcff9640 1 -- 172.21.15.100:0/773838701 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8016690 con 0x7f7700154620 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.222+0000 7f7705ee7640 1 -- 172.21.15.100:0/773838701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001e3f00 con 0x7f7700154620 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.223+0000 7f76fcff9640 1 -- 172.21.15.100:0/773838701 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f76f8016af0 con 0x7f7700154620 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.223+0000 7f76fcff9640 1 -- 172.21.15.100:0/773838701 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f76f800f070 con 0x7f7700154620 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.223+0000 7f7705ee7640 1 -- 172.21.15.100:0/773838701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77001e40f0 con 0x7f7700154620 2023-11-18T20:42:14.230 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.225+0000 7f76fcff9640 1 -- 172.21.15.100:0/773838701 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f76f8021460 con 0x7f7700154620 2023-11-18T20:42:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.298+0000 7f7705ee7640 1 -- 172.21.15.100:0/773838701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3ac66b30-eb29-41c3-bbf5-c9d2463d00b2", "id": 1} v 0) v1 -- 0x7f7700067420 con 0x7f7700154620 2023-11-18T20:42:14.301 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.300+0000 7f76fcff9640 1 -- 172.21.15.100:0/773838701 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ac66b30-eb29-41c3-bbf5-c9d2463d00b2", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f76f8021600 con 0x7f7700154620 2023-11-18T20:42:14.302 INFO:teuthology.orchestra.run.smithi100.stdout:1 2023-11-18T20:42:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.301+0000 7f7705ee7640 1 -- 172.21.15.100:0/773838701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7700154620 msgr2=0x7f7700154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.301+0000 7f7705ee7640 1 --2- 172.21.15.100:0/773838701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7700154620 0x7f7700154ab0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f77000b4e10 tx=0x7f76f8009f90 comp rx=0 tx=0).stop 2023-11-18T20:42:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.301+0000 7f7705ee7640 1 -- 172.21.15.100:0/773838701 shutdown_connections 2023-11-18T20:42:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.301+0000 7f7705ee7640 1 --2- 172.21.15.100:0/773838701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7700154620 0x7f7700154ab0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.301+0000 7f7705ee7640 1 --2- 172.21.15.100:0/773838701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f770013c940 0x7f7700152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.301+0000 7f7705ee7640 1 --2- 172.21.15.100:0/773838701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76ec003b50 0x7f770013c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.301+0000 7f7705ee7640 1 -- 172.21.15.100:0/773838701 >> 172.21.15.100:0/773838701 conn(0x7f77000b1a50 msgr2=0x7f7700070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.302+0000 7f7705ee7640 1 -- 172.21.15.100:0/773838701 shutdown_connections 2023-11-18T20:42:14.303 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.302+0000 7f7705ee7640 1 -- 172.21.15.100:0/773838701 wait complete. 2023-11-18T20:42:14.313 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 16e394a5-b7ad-4465-85af-63a9107958f1 2 2023-11-18T20:42:14.405 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:14.404+0000 7f5528c32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-18T20:42:14.413 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.412+0000 7fc1316b4640 1 Processor -- start 2023-11-18T20:42:14.413 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.412+0000 7fc1316b4640 1 -- start start 2023-11-18T20:42:14.415 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc1316b4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc12c15ac90 0x7fc12c161740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.415 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc1316b4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc12c0b3f10 0x7fc12c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.415 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc1316b4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12c1612b0 con 0x7fc12c0b3000 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc1316b4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12c15e190 con 0x7fc12c0b3f10 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc1316b4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12c15e470 con 0x7fc12c15ac90 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc12affd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc12c15ac90 0x7fc12c161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc12b7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc12c0b3000 0x7fc12c06baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35190/0 (socket says 172.21.15.100:35190) 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc12b7fe640 1 -- 172.21.15.100:0/621482839 learned_addr learned my addr 172.21.15.100:0/621482839 (peer_addr_for_me v1:172.21.15.100:0/0) 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc12a7fc640 1 --2- 172.21.15.100:0/621482839 >> v2:172.21.15.60:3300/0 conn(0x7fc12c0b3f10 0x7fc12c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc12affd640 1 -- 172.21.15.100:0/621482839 >> v2:172.21.15.60:3300/0 conn(0x7fc12c0b3f10 msgr2=0x7fc12c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc12affd640 1 --2- 172.21.15.100:0/621482839 >> v2:172.21.15.60:3300/0 conn(0x7fc12c0b3f10 0x7fc12c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc12affd640 1 -- 172.21.15.100:0/621482839 >> v1:172.21.15.100:6789/0 conn(0x7fc12c0b3000 legacy=0x7fc12c06baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc12affd640 1 -- 172.21.15.100:0/621482839 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c15e750 con 0x7fc12c15ac90 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc129ffb640 1 -- 172.21.15.100:0/621482839 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458602973 0 0) 0x7fc12c1612b0 con 0x7fc12c0b3000 2023-11-18T20:42:14.416 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.413+0000 7fc12a7fc640 1 --2- 172.21.15.100:0/621482839 >> v2:172.21.15.60:3300/0 conn(0x7fc12c0b3f10 0x7fc12c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:14.417 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.414+0000 7fc12affd640 1 --2- 172.21.15.100:0/621482839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 0x7fc12c161740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc12c070520 tx=0x7fc12002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3582be10df0e5e1a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:14.417 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.414+0000 7fc129ffb640 1 -- 172.21.15.100:0/621482839 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12002e070 con 0x7fc12c15ac90 2023-11-18T20:42:14.417 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.414+0000 7fc129ffb640 1 -- 172.21.15.100:0/621482839 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1200315a0 con 0x7fc12c15ac90 2023-11-18T20:42:14.417 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.414+0000 7fc129ffb640 1 -- 172.21.15.100:0/621482839 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120031930 con 0x7fc12c15ac90 2023-11-18T20:42:14.421 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.415+0000 7fc1316b4640 1 -- 172.21.15.100:0/621482839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 msgr2=0x7fc12c161740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.421 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.415+0000 7fc1316b4640 1 --2- 172.21.15.100:0/621482839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 0x7fc12c161740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc12c070520 tx=0x7fc12002b840 comp rx=0 tx=0).stop 2023-11-18T20:42:14.421 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.415+0000 7fc1316b4640 1 -- 172.21.15.100:0/621482839 shutdown_connections 2023-11-18T20:42:14.421 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.415+0000 7fc1316b4640 1 --2- 172.21.15.100:0/621482839 >> v2:172.21.15.60:3300/0 conn(0x7fc12c0b3f10 0x7fc12c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.421 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.415+0000 7fc1316b4640 1 --2- 172.21.15.100:0/621482839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 0x7fc12c161740 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.415+0000 7fc1316b4640 1 -- 172.21.15.100:0/621482839 >> 172.21.15.100:0/621482839 conn(0x7fc12c06a8c0 msgr2=0x7fc12c06acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.415+0000 7fc1316b4640 1 -- 172.21.15.100:0/621482839 shutdown_connections 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.415+0000 7fc1316b4640 1 -- 172.21.15.100:0/621482839 wait complete. 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.416+0000 7fc1316b4640 1 Processor -- start 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.416+0000 7fc1316b4640 1 -- start start 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc1316b4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc12c0b3000 0x7fc12c1f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc1316b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc12c0b3f10 0x7fc12c1f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc1316b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 0x7fc12c0cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc1316b4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c1f6a80 con 0x7fc12c0b3f10 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc1316b4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c1f6d60 con 0x7fc12c15ac90 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc1316b4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0ca330 con 0x7fc12c0b3000 2023-11-18T20:42:14.422 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 0x7fc12c0cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12affd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc12c0b3000 0x7fc12c1f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 0x7fc12c0cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37286/0 (socket says 172.21.15.100:37286) 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12b7fe640 1 -- 172.21.15.100:0/4248602646 learned_addr learned my addr 172.21.15.100:0/4248602646 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12a7fc640 1 --2- 172.21.15.100:0/4248602646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc12c0b3f10 0x7fc12c1f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12b7fe640 1 -- 172.21.15.100:0/4248602646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc12c0b3000 msgr2=0x7fc12c1f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12b7fe640 1 --2- 172.21.15.100:0/4248602646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc12c0b3000 0x7fc12c1f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12b7fe640 1 -- 172.21.15.100:0/4248602646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc12c0b3f10 msgr2=0x7fc12c1f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12b7fe640 1 --2- 172.21.15.100:0/4248602646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc12c0b3f10 0x7fc12c1f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12b7fe640 1 -- 172.21.15.100:0/4248602646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc120009400 con 0x7fc12c15ac90 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.417+0000 7fc12b7fe640 1 --2- 172.21.15.100:0/4248602646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 0x7fc12c0cab30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c7e0 tx=0x7fc12400cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:14.423 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.418+0000 7fc1137fe640 1 -- 172.21.15.100:0/4248602646 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12400a640 con 0x7fc12c15ac90 2023-11-18T20:42:14.424 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.418+0000 7fc1137fe640 1 -- 172.21.15.100:0/4248602646 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12400a7c0 con 0x7fc12c15ac90 2023-11-18T20:42:14.424 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.418+0000 7fc1137fe640 1 -- 172.21.15.100:0/4248602646 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12400ab70 con 0x7fc12c15ac90 2023-11-18T20:42:14.424 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.418+0000 7fc1316b4640 1 -- 172.21.15.100:0/4248602646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12c0ca610 con 0x7fc12c15ac90 2023-11-18T20:42:14.424 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.418+0000 7fc1316b4640 1 -- 172.21.15.100:0/4248602646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c1f66e0 con 0x7fc12c15ac90 2023-11-18T20:42:14.424 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.418+0000 7fc1137fe640 1 -- 172.21.15.100:0/4248602646 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc1240070d0 con 0x7fc12c15ac90 2023-11-18T20:42:14.424 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.418+0000 7fc1137fe640 1 -- 172.21.15.100:0/4248602646 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fc1240129a0 con 0x7fc12c15ac90 2023-11-18T20:42:14.424 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.419+0000 7fc1316b4640 1 -- 172.21.15.100:0/4248602646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c067420 con 0x7fc12c15ac90 2023-11-18T20:42:14.425 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.421+0000 7fc1137fe640 1 -- 172.21.15.100:0/4248602646 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc124018070 con 0x7fc12c15ac90 2023-11-18T20:42:14.461 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:14.460+0000 7f5528c32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-18T20:42:14.483 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:14.482+0000 7f34bfa2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-18T20:42:14.494 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.493+0000 7fc1316b4640 1 -- 172.21.15.100:0/4248602646 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "16e394a5-b7ad-4465-85af-63a9107958f1", "id": 2} v 0) v1 -- 0x7fc12c0b6210 con 0x7fc12c15ac90 2023-11-18T20:42:14.497 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.497+0000 7fc1137fe640 1 -- 172.21.15.100:0/4248602646 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "16e394a5-b7ad-4465-85af-63a9107958f1", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc124012610 con 0x7fc12c15ac90 2023-11-18T20:42:14.498 INFO:teuthology.orchestra.run.smithi100.stdout:2 2023-11-18T20:42:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.497+0000 7fc1316b4640 1 -- 172.21.15.100:0/4248602646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 msgr2=0x7fc12c0cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.497+0000 7fc1316b4640 1 --2- 172.21.15.100:0/4248602646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 0x7fc12c0cab30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c7e0 tx=0x7fc12400cca0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.497+0000 7fc1316b4640 1 -- 172.21.15.100:0/4248602646 shutdown_connections 2023-11-18T20:42:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.497+0000 7fc1316b4640 1 --2- 172.21.15.100:0/4248602646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc12c15ac90 0x7fc12c0cab30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.497+0000 7fc1316b4640 1 --2- 172.21.15.100:0/4248602646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc12c0b3f10 0x7fc12c1f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.497+0000 7fc1316b4640 1 --2- 172.21.15.100:0/4248602646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc12c0b3000 0x7fc12c1f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.497+0000 7fc1316b4640 1 -- 172.21.15.100:0/4248602646 >> 172.21.15.100:0/4248602646 conn(0x7fc12c06a8c0 msgr2=0x7fc12c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.498+0000 7fc1316b4640 1 -- 172.21.15.100:0/4248602646 shutdown_connections 2023-11-18T20:42:14.499 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.498+0000 7fc1316b4640 1 -- 172.21.15.100:0/4248602646 wait complete. 2023-11-18T20:42:14.507 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:14.506+0000 7f5528c32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-18T20:42:14.509 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new babf5c8f-7384-4916-8d52-382f76dcd263 3 2023-11-18T20:42:14.539 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:14.538+0000 7f34bfa2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-18T20:42:14.563 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:14.562+0000 7f5528c32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-18T20:42:14.585 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:14.584+0000 7f34bfa2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-18T20:42:14.607 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:14.606+0000 7f5528c32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-18T20:42:14.608 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.608+0000 7f3ad1d3e640 1 Processor -- start 2023-11-18T20:42:14.609 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.608+0000 7f3ad1d3e640 1 -- start start 2023-11-18T20:42:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.608+0000 7f3ad1d3e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3acc06f630 0x7f3acc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.608+0000 7f3ad1d3e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3acc06ee30 0x7f3acc0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.608+0000 7f3ad1d3e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3acc06f280 con 0x7f3acc143b50 2023-11-18T20:42:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.608+0000 7f3ad1d3e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3acc0cfde0 con 0x7f3acc06f630 2023-11-18T20:42:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.608+0000 7f3ad1d3e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3acc0cf000 con 0x7f3acc06ee30 2023-11-18T20:42:14.611 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.609+0000 7f3acbfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3acc06ee30 0x7f3acc0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.612 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.609+0000 7f3acbfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3acc06ee30 0x7f3acc0b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37302/0 (socket says 172.21.15.100:37302) 2023-11-18T20:42:14.612 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.609+0000 7f3acbfff640 1 -- 172.21.15.100:0/1818462464 learned_addr learned my addr 172.21.15.100:0/1818462464 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:14.612 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.609+0000 7f3acaffd640 1 --2- 172.21.15.100:0/1818462464 >> v2:172.21.15.60:3300/0 conn(0x7f3acc06f630 0x7f3acc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.612 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.609+0000 7f3acbfff640 1 -- 172.21.15.100:0/1818462464 >> v2:172.21.15.60:3300/0 conn(0x7f3acc06f630 msgr2=0x7f3acc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:14.612 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.609+0000 7f3acbfff640 1 --2- 172.21.15.100:0/1818462464 >> v2:172.21.15.60:3300/0 conn(0x7f3acc06f630 0x7f3acc06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.612 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.609+0000 7f3acbfff640 1 -- 172.21.15.100:0/1818462464 >> v1:172.21.15.60:6789/0 conn(0x7f3acc143b50 legacy=0x7f3acc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.613 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.609+0000 7f3acbfff640 1 -- 172.21.15.100:0/1818462464 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc0cf260 con 0x7f3acc06ee30 2023-11-18T20:42:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.610+0000 7f3acbfff640 1 --2- 172.21.15.100:0/1818462464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc06ee30 0x7f3acc0b33c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3acc15eba0 tx=0x7f3ac402e0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a60f2e79c52f9b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.610+0000 7f3aca7fc640 1 -- 172.21.15.100:0/1818462464 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac402ea90 con 0x7f3acc06ee30 2023-11-18T20:42:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.610+0000 7f3aca7fc640 1 -- 172.21.15.100:0/1818462464 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac402ebf0 con 0x7f3acc06ee30 2023-11-18T20:42:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.610+0000 7f3aca7fc640 1 -- 172.21.15.100:0/1818462464 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4009600 con 0x7f3acc06ee30 2023-11-18T20:42:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.611+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1818462464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc06ee30 msgr2=0x7f3acc0b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.611+0000 7f3ad1d3e640 1 --2- 172.21.15.100:0/1818462464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc06ee30 0x7f3acc0b33c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3acc15eba0 tx=0x7f3ac402e0f0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.611+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1818462464 shutdown_connections 2023-11-18T20:42:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.611+0000 7f3ad1d3e640 1 --2- 172.21.15.100:0/1818462464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc06ee30 0x7f3acc0b33c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.611+0000 7f3ad1d3e640 1 --2- 172.21.15.100:0/1818462464 >> v2:172.21.15.60:3300/0 conn(0x7f3acc06f630 0x7f3acc06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.617 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.611+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1818462464 >> 172.21.15.100:0/1818462464 conn(0x7f3acc06d3b0 msgr2=0x7f3acc06d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.611+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1818462464 shutdown_connections 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.611+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1818462464 wait complete. 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.612+0000 7f3ad1d3e640 1 Processor -- start 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.612+0000 7f3ad1d3e640 1 -- start start 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3ad1d3e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc06ee30 0x7f3acc1edf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3ad1d3e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc06f630 0x7f3acc156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3ad1d3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc143b50 0x7f3acc156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3ad1d3e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc142cb0 con 0x7f3acc06f630 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3ad1d3e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc1420a0 con 0x7f3acc143b50 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3ad1d3e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc142380 con 0x7f3acc06ee30 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc06ee30 0x7f3acc1edf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.618 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc143b50 0x7f3acc156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc143b50 0x7f3acc156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37308/0 (socket says 172.21.15.100:37308) 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc06ee30 0x7f3acc1edf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:57714/0 (socket says 172.21.15.100:57714) 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acbfff640 1 -- 172.21.15.100:0/1071773370 learned_addr learned my addr 172.21.15.100:0/1071773370 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acaffd640 1 --2- 172.21.15.100:0/1071773370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc06f630 0x7f3acc156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acb7fe640 1 -- 172.21.15.100:0/1071773370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc143b50 msgr2=0x7f3acc156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acb7fe640 1 --2- 172.21.15.100:0/1071773370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc143b50 0x7f3acc156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acb7fe640 1 -- 172.21.15.100:0/1071773370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc06f630 msgr2=0x7f3acc156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acb7fe640 1 --2- 172.21.15.100:0/1071773370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc06f630 0x7f3acc156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acb7fe640 1 -- 172.21.15.100:0/1071773370 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac400bd40 con 0x7f3acc06ee30 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.613+0000 7f3acbfff640 1 --2- 172.21.15.100:0/1071773370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc143b50 0x7f3acc156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.614+0000 7f3acb7fe640 1 --2- 172.21.15.100:0/1071773370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc06ee30 0x7f3acc1edf10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c390 tx=0x7f3abc00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.614+0000 7f3ab3fff640 1 -- 172.21.15.100:0/1071773370 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0052d0 con 0x7f3acc06ee30 2023-11-18T20:42:14.619 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.614+0000 7f3ab3fff640 1 -- 172.21.15.100:0/1071773370 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc005430 con 0x7f3acc06ee30 2023-11-18T20:42:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.614+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1071773370 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3acc142660 con 0x7f3acc06ee30 2023-11-18T20:42:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.614+0000 7f3ab3fff640 1 -- 172.21.15.100:0/1071773370 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc003830 con 0x7f3acc06ee30 2023-11-18T20:42:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.614+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1071773370 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3acc133af0 con 0x7f3acc06ee30 2023-11-18T20:42:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.614+0000 7f3ab3fff640 1 -- 172.21.15.100:0/1071773370 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3abc003c90 con 0x7f3acc06ee30 2023-11-18T20:42:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.614+0000 7f3ab3fff640 1 -- 172.21.15.100:0/1071773370 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f3abc011a20 con 0x7f3acc06ee30 2023-11-18T20:42:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.615+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1071773370 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3acc067420 con 0x7f3acc06ee30 2023-11-18T20:42:14.620 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.617+0000 7f3ab3fff640 1 -- 172.21.15.100:0/1071773370 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f3abc017070 con 0x7f3acc06ee30 2023-11-18T20:42:14.643 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:14.641+0000 7f34bfa2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-18T20:42:14.686 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:14.685+0000 7f34bfa2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-18T20:42:14.688 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.687+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1071773370 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "babf5c8f-7384-4916-8d52-382f76dcd263", "id": 3} v 0) v1 -- 0x7f3acc15aa20 con 0x7f3acc06ee30 2023-11-18T20:42:14.691 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.690+0000 7f3ab3fff640 1 -- 172.21.15.100:0/1071773370 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "babf5c8f-7384-4916-8d52-382f76dcd263", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3acc06f280 con 0x7f3acc06ee30 2023-11-18T20:42:14.692 INFO:teuthology.orchestra.run.smithi100.stdout:3 2023-11-18T20:42:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.691+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1071773370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc06ee30 msgr2=0x7f3acc1edf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.691+0000 7f3ad1d3e640 1 --2- 172.21.15.100:0/1071773370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc06ee30 0x7f3acc1edf10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c390 tx=0x7f3abc00c850 comp rx=0 tx=0).stop 2023-11-18T20:42:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.691+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1071773370 shutdown_connections 2023-11-18T20:42:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.691+0000 7f3ad1d3e640 1 --2- 172.21.15.100:0/1071773370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc143b50 0x7f3acc156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.691+0000 7f3ad1d3e640 1 --2- 172.21.15.100:0/1071773370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc06f630 0x7f3acc156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.691+0000 7f3ad1d3e640 1 --2- 172.21.15.100:0/1071773370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc06ee30 0x7f3acc1edf10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.691+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1071773370 >> 172.21.15.100:0/1071773370 conn(0x7f3acc06d3b0 msgr2=0x7f3acc15c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.691+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1071773370 shutdown_connections 2023-11-18T20:42:14.693 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.692+0000 7f3ad1d3e640 1 -- 172.21.15.100:0/1071773370 wait complete. 2023-11-18T20:42:14.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new c814556f-cab3-4263-9038-9b6dec6dda55 4 2023-11-18T20:42:14.712 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:14.711+0000 7f5528c32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-18T20:42:14.787 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:14.786+0000 7f5528c32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-18T20:42:14.792 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:14.791+0000 7f34bfa2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-18T20:42:14.802 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.801+0000 7f631cd5e640 1 Processor -- start 2023-11-18T20:42:14.803 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.801+0000 7f631cd5e640 1 -- start start 2023-11-18T20:42:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.802+0000 7f631cd5e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63180714c0 0x7f6318071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.802+0000 7f631cd5e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63180b4180 con 0x7f63180ce8a0 2023-11-18T20:42:14.804 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.802+0000 7f631cd5e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63180cd1d0 con 0x7f6318143820 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.802+0000 7f631cd5e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6318071eb0 con 0x7f63180714c0 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.802+0000 7f631659c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6318143820 0x7f63180ccac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35196/0 (socket says 172.21.15.100:35196) 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.802+0000 7f631659c640 1 -- 172.21.15.100:0/666243539 learned_addr learned my addr 172.21.15.100:0/666243539 (peer_addr_for_me v1:172.21.15.100:0/0) 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.803+0000 7f6315d9b640 1 --2- 172.21.15.100:0/666243539 >> v2:172.21.15.60:3300/0 conn(0x7f63180714c0 0x7f6318071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.803+0000 7f631559a640 1 -- 172.21.15.100:0/666243539 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745138989 0 0) 0x7f63180cd1d0 con 0x7f6318143820 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.803+0000 7f631559a640 1 -- 172.21.15.100:0/666243539 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6300003520 con 0x7f6318143820 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.803+0000 7f631559a640 1 -- 172.21.15.100:0/666243539 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439113074 0 0) 0x7f63180b4180 con 0x7f63180ce8a0 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.803+0000 7f631559a640 1 -- 172.21.15.100:0/666243539 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63180cd1d0 con 0x7f63180ce8a0 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.803+0000 7f6315d9b640 1 -- 172.21.15.100:0/666243539 >> v1:172.21.15.100:6789/0 conn(0x7f6318143820 legacy=0x7f63180ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.803+0000 7f6315d9b640 1 -- 172.21.15.100:0/666243539 >> v1:172.21.15.60:6789/0 conn(0x7f63180ce8a0 legacy=0x7f63180ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.805 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.803+0000 7f6315d9b640 1 -- 172.21.15.100:0/666243539 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63180cd430 con 0x7f63180714c0 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.804+0000 7f6315d9b640 1 --2- 172.21.15.100:0/666243539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63180714c0 0x7f6318071970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f630c0097a0 tx=0x7f630c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ceb708036a15a7f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.804+0000 7f631559a640 1 -- 172.21.15.100:0/666243539 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c030610 con 0x7f63180714c0 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.804+0000 7f631559a640 1 -- 172.21.15.100:0/666243539 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c030770 con 0x7f63180714c0 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.804+0000 7f631559a640 1 -- 172.21.15.100:0/666243539 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c030b00 con 0x7f63180714c0 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.805+0000 7f631cd5e640 1 -- 172.21.15.100:0/666243539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63180714c0 msgr2=0x7f6318071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.805+0000 7f631cd5e640 1 --2- 172.21.15.100:0/666243539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63180714c0 0x7f6318071970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f630c0097a0 tx=0x7f630c02b960 comp rx=0 tx=0).stop 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.805+0000 7f631cd5e640 1 -- 172.21.15.100:0/666243539 shutdown_connections 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.805+0000 7f631cd5e640 1 --2- 172.21.15.100:0/666243539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63180714c0 0x7f6318071970 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.805+0000 7f631cd5e640 1 -- 172.21.15.100:0/666243539 >> 172.21.15.100:0/666243539 conn(0x7f63180b1de0 msgr2=0x7f63180b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.805+0000 7f631cd5e640 1 -- 172.21.15.100:0/666243539 shutdown_connections 2023-11-18T20:42:14.807 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.805+0000 7f631cd5e640 1 -- 172.21.15.100:0/666243539 wait complete. 2023-11-18T20:42:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.806+0000 7f631cd5e640 1 Processor -- start 2023-11-18T20:42:14.810 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.806+0000 7f631cd5e640 1 -- start start 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f631cd5e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63180714c0 0x7f631814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f631cd5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63180ce8a0 0x7f63181e2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f631cd5e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6318143820 0x7f631806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f631cd5e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63181e3d50 con 0x7f63180714c0 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f631cd5e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63181e3140 con 0x7f63180ce8a0 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f631cd5e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63181e3420 con 0x7f6318143820 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f6315d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63180ce8a0 0x7f63181e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f6316d9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6318143820 0x7f631806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f6315d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63180ce8a0 0x7f63181e2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37314/0 (socket says 172.21.15.100:37314) 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f6315d9b640 1 -- 172.21.15.100:0/2261086909 learned_addr learned my addr 172.21.15.100:0/2261086909 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.807+0000 7f631659c640 1 --2- 172.21.15.100:0/2261086909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63180714c0 0x7f631814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6315d9b640 1 -- 172.21.15.100:0/2261086909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6318143820 msgr2=0x7f631806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6315d9b640 1 --2- 172.21.15.100:0/2261086909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6318143820 0x7f631806a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6315d9b640 1 -- 172.21.15.100:0/2261086909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63180714c0 msgr2=0x7f631814ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6315d9b640 1 --2- 172.21.15.100:0/2261086909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63180714c0 0x7f631814ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6315d9b640 1 -- 172.21.15.100:0/2261086909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c009400 con 0x7f63180ce8a0 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6315d9b640 1 --2- 172.21.15.100:0/2261086909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63180ce8a0 0x7f63181e2760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f630c02bf70 tx=0x7f630c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6306ffd640 1 -- 172.21.15.100:0/2261086909 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c03e040 con 0x7f63180ce8a0 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6306ffd640 1 -- 172.21.15.100:0/2261086909 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c039a40 con 0x7f63180ce8a0 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f631cd5e640 1 -- 172.21.15.100:0/2261086909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63181e3700 con 0x7f63180ce8a0 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6306ffd640 1 -- 172.21.15.100:0/2261086909 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c039df0 con 0x7f63180ce8a0 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f631cd5e640 1 -- 172.21.15.100:0/2261086909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63181e7010 con 0x7f63180ce8a0 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6306ffd640 1 -- 172.21.15.100:0/2261086909 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f630c04e740 con 0x7f63180ce8a0 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.808+0000 7f6306ffd640 1 -- 172.21.15.100:0/2261086909 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f630c044b30 con 0x7f63180ce8a0 2023-11-18T20:42:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.809+0000 7f631cd5e640 1 -- 172.21.15.100:0/2261086909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6318067420 con 0x7f63180ce8a0 2023-11-18T20:42:14.813 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.811+0000 7f6306ffd640 1 -- 172.21.15.100:0/2261086909 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f630c049070 con 0x7f63180ce8a0 2023-11-18T20:42:14.835 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:14.835+0000 7f5528c32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-18T20:42:14.865 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:14.864+0000 7f34bfa2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-18T20:42:14.884 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.883+0000 7f631cd5e640 1 -- 172.21.15.100:0/2261086909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c814556f-cab3-4263-9038-9b6dec6dda55", "id": 4} v 0) v1 -- 0x7f63180b0520 con 0x7f63180ce8a0 2023-11-18T20:42:14.887 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.886+0000 7f6306ffd640 1 -- 172.21.15.100:0/2261086909 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c814556f-cab3-4263-9038-9b6dec6dda55", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f630c034070 con 0x7f63180ce8a0 2023-11-18T20:42:14.888 INFO:teuthology.orchestra.run.smithi100.stdout:4 2023-11-18T20:42:14.889 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.887+0000 7f631cd5e640 1 -- 172.21.15.100:0/2261086909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63180ce8a0 msgr2=0x7f63181e2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.887+0000 7f631cd5e640 1 --2- 172.21.15.100:0/2261086909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63180ce8a0 0x7f63181e2760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f630c02bf70 tx=0x7f630c004730 comp rx=0 tx=0).stop 2023-11-18T20:42:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.887+0000 7f631cd5e640 1 -- 172.21.15.100:0/2261086909 shutdown_connections 2023-11-18T20:42:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.887+0000 7f631cd5e640 1 --2- 172.21.15.100:0/2261086909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6318143820 0x7f631806a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.887+0000 7f631cd5e640 1 --2- 172.21.15.100:0/2261086909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63180ce8a0 0x7f63181e2760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.887+0000 7f631cd5e640 1 --2- 172.21.15.100:0/2261086909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63180714c0 0x7f631814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:14.890 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.887+0000 7f631cd5e640 1 -- 172.21.15.100:0/2261086909 >> 172.21.15.100:0/2261086909 conn(0x7f63180b1de0 msgr2=0x7f631815aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.887+0000 7f631cd5e640 1 -- 172.21.15.100:0/2261086909 shutdown_connections 2023-11-18T20:42:14.891 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:14.887+0000 7f631cd5e640 1 -- 172.21.15.100:0/2261086909 wait complete. 2023-11-18T20:42:14.891 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:14.889+0000 7f5528c32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-18T20:42:14.898 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new bd5eb319-c72d-4451-a478-db504bd2fcf3 5 2023-11-18T20:42:14.914 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:14.912+0000 7f34bfa2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-18T20:42:14.938 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:14.937+0000 7f5528c32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-18T20:42:14.968 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:14.967+0000 7f34bfa2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-18T20:42:15.017 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:15.015+0000 7f34bfa2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-18T20:42:15.038 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.037+0000 7f8d5e683640 1 Processor -- start 2023-11-18T20:42:15.038 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.037+0000 7f8d5e683640 1 -- start start 2023-11-18T20:42:15.040 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.037+0000 7f8d5e683640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d581543c0 0x7f8d5815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.040 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.037+0000 7f8d5e683640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5806db80 con 0x7f8d5806bbf0 2023-11-18T20:42:15.040 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.037+0000 7f8d5e683640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d581547a0 con 0x7f8d580b2720 2023-11-18T20:42:15.040 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.037+0000 7f8d5e683640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d580b3860 con 0x7f8d581543c0 2023-11-18T20:42:15.040 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.038+0000 7f8d57fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d581543c0 0x7f8d5815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.040 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.038+0000 7f8d57fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d581543c0 0x7f8d5815e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37324/0 (socket says 172.21.15.100:37324) 2023-11-18T20:42:15.040 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.038+0000 7f8d57fff640 1 -- 172.21.15.100:0/3250283069 learned_addr learned my addr 172.21.15.100:0/3250283069 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:15.040 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.038+0000 7f8d577fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8d580b2720 0x7f8d5806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35202/0 (socket says 172.21.15.100:35202) 2023-11-18T20:42:15.040 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.038+0000 7f8d57fff640 1 -- 172.21.15.100:0/3250283069 >> v1:172.21.15.100:6789/0 conn(0x7f8d580b2720 legacy=0x7f8d5806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.040 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.038+0000 7f8d57fff640 1 -- 172.21.15.100:0/3250283069 >> v1:172.21.15.60:6789/0 conn(0x7f8d5806bbf0 legacy=0x7f8d5806e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.041 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.038+0000 7f8d57fff640 1 -- 172.21.15.100:0/3250283069 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d580b2ce0 con 0x7f8d581543c0 2023-11-18T20:42:15.041 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d57fff640 1 --2- 172.21.15.100:0/3250283069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d581543c0 0x7f8d5815e2f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009870 tx=0x7f8d4802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfd3f707eab15245 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:15.041 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d56ffd640 1 -- 172.21.15.100:0/3250283069 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48031870 con 0x7f8d581543c0 2023-11-18T20:42:15.041 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d56ffd640 1 -- 172.21.15.100:0/3250283069 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d480319d0 con 0x7f8d581543c0 2023-11-18T20:42:15.041 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d56ffd640 1 -- 172.21.15.100:0/3250283069 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48031d60 con 0x7f8d581543c0 2023-11-18T20:42:15.042 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d5e683640 1 -- 172.21.15.100:0/3250283069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d581543c0 msgr2=0x7f8d5815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.042 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d5e683640 1 --2- 172.21.15.100:0/3250283069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d581543c0 0x7f8d5815e2f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009870 tx=0x7f8d4802b910 comp rx=0 tx=0).stop 2023-11-18T20:42:15.042 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d5e683640 1 -- 172.21.15.100:0/3250283069 shutdown_connections 2023-11-18T20:42:15.042 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d5e683640 1 --2- 172.21.15.100:0/3250283069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d581543c0 0x7f8d5815e2f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.042 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d5e683640 1 -- 172.21.15.100:0/3250283069 >> 172.21.15.100:0/3250283069 conn(0x7f8d58070ec0 msgr2=0x7f8d580712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:15.042 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d5e683640 1 -- 172.21.15.100:0/3250283069 shutdown_connections 2023-11-18T20:42:15.042 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.039+0000 7f8d5e683640 1 -- 172.21.15.100:0/3250283069 wait complete. 2023-11-18T20:42:15.042 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.040+0000 7f8d5e683640 1 Processor -- start 2023-11-18T20:42:15.043 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.041+0000 7f8d5e683640 1 -- start start 2023-11-18T20:42:15.045 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.041+0000 7f8d5e683640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5806bbf0 0x7f8d58146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.045 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.041+0000 7f8d5e683640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d580b2720 0x7f8d5815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.045 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.041+0000 7f8d5e683640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d581543c0 0x7f8d5815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.045 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.041+0000 7f8d5e683640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d58155700 con 0x7f8d581543c0 2023-11-18T20:42:15.045 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.041+0000 7f8d5e683640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d58154b10 con 0x7f8d580b2720 2023-11-18T20:42:15.045 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.041+0000 7f8d5e683640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d58154dd0 con 0x7f8d5806bbf0 2023-11-18T20:42:15.045 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.041+0000 7f8d577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d580b2720 0x7f8d5815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.045 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.041+0000 7f8d577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d580b2720 0x7f8d5815fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37330/0 (socket says 172.21.15.100:37330) 2023-11-18T20:42:15.045 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.041+0000 7f8d577fe640 1 -- 172.21.15.100:0/1327654672 learned_addr learned my addr 172.21.15.100:0/1327654672 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:15.045 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d57fff640 1 --2- 172.21.15.100:0/1327654672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5806bbf0 0x7f8d58146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d5cc20640 1 --2- 172.21.15.100:0/1327654672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d581543c0 0x7f8d5815ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d577fe640 1 -- 172.21.15.100:0/1327654672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5806bbf0 msgr2=0x7f8d58146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d577fe640 1 --2- 172.21.15.100:0/1327654672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5806bbf0 0x7f8d58146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d577fe640 1 -- 172.21.15.100:0/1327654672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d581543c0 msgr2=0x7f8d5815ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d577fe640 1 --2- 172.21.15.100:0/1327654672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d581543c0 0x7f8d5815ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d577fe640 1 -- 172.21.15.100:0/1327654672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d480094d0 con 0x7f8d580b2720 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d577fe640 1 --2- 172.21.15.100:0/1327654672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d580b2720 0x7f8d5815fa60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00c4d0 tx=0x7f8d4c00c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d54ff9640 1 -- 172.21.15.100:0/1327654672 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c007480 con 0x7f8d580b2720 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d54ff9640 1 -- 172.21.15.100:0/1327654672 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c0075e0 con 0x7f8d580b2720 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d54ff9640 1 -- 172.21.15.100:0/1327654672 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c0049e0 con 0x7f8d580b2720 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d5e683640 1 -- 172.21.15.100:0/1327654672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d581550b0 con 0x7f8d580b2720 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d5e683640 1 -- 172.21.15.100:0/1327654672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d58146d70 con 0x7f8d580b2720 2023-11-18T20:42:15.046 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d54ff9640 1 -- 172.21.15.100:0/1327654672 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8d4c012440 con 0x7f8d580b2720 2023-11-18T20:42:15.047 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.042+0000 7f8d54ff9640 1 -- 172.21.15.100:0/1327654672 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f8d4c004e40 con 0x7f8d580b2720 2023-11-18T20:42:15.047 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.043+0000 7f8d5e683640 1 -- 172.21.15.100:0/1327654672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d58067420 con 0x7f8d580b2720 2023-11-18T20:42:15.048 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.047+0000 7f8d54ff9640 1 -- 172.21.15.100:0/1327654672 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8d4c021060 con 0x7f8d580b2720 2023-11-18T20:42:15.120 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.119+0000 7f8d5e683640 1 -- 172.21.15.100:0/1327654672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bd5eb319-c72d-4451-a478-db504bd2fcf3", "id": 5} v 0) v1 -- 0x7f8d58073b30 con 0x7f8d580b2720 2023-11-18T20:42:15.123 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.122+0000 7f8d54ff9640 1 -- 172.21.15.100:0/1327654672 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bd5eb319-c72d-4451-a478-db504bd2fcf3", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8d4c018070 con 0x7f8d580b2720 2023-11-18T20:42:15.124 INFO:teuthology.orchestra.run.smithi100.stdout:5 2023-11-18T20:42:15.125 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.123+0000 7f8d5e683640 1 -- 172.21.15.100:0/1327654672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d580b2720 msgr2=0x7f8d5815fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.125 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.123+0000 7f8d5e683640 1 --2- 172.21.15.100:0/1327654672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d580b2720 0x7f8d5815fa60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00c4d0 tx=0x7f8d4c00c990 comp rx=0 tx=0).stop 2023-11-18T20:42:15.125 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.123+0000 7f8d5e683640 1 -- 172.21.15.100:0/1327654672 shutdown_connections 2023-11-18T20:42:15.125 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.123+0000 7f8d5e683640 1 --2- 172.21.15.100:0/1327654672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d581543c0 0x7f8d5815ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.125 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.123+0000 7f8d5e683640 1 --2- 172.21.15.100:0/1327654672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d580b2720 0x7f8d5815fa60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.125 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.123+0000 7f8d5e683640 1 --2- 172.21.15.100:0/1327654672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5806bbf0 0x7f8d58146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.125 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.123+0000 7f8d5e683640 1 -- 172.21.15.100:0/1327654672 >> 172.21.15.100:0/1327654672 conn(0x7f8d58070ec0 msgr2=0x7f8d5814ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:15.125 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.124+0000 7f8d5e683640 1 -- 172.21.15.100:0/1327654672 shutdown_connections 2023-11-18T20:42:15.125 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.124+0000 7f8d5e683640 1 -- 172.21.15.100:0/1327654672 wait complete. 2023-11-18T20:42:15.135 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 83e97732-ae79-4373-9313-65910a99e0d6 6 2023-11-18T20:42:15.235 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.234+0000 7f168a2f2640 1 Processor -- start 2023-11-18T20:42:15.235 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.234+0000 7f168a2f2640 1 -- start start 2023-11-18T20:42:15.236 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f168a2f2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f168406dc70 0x7f168415dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.236 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f168a2f2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168415e220 0x7f16840ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f168a2f2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168413d440 con 0x7f168413d060 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f168a2f2640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168415eaa0 con 0x7f168406dc70 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f168a2f2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168415a700 con 0x7f168415e220 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f168888f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168415e220 0x7f16840ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f1683fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f168413d060 0x7f168406d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:35212/0 (socket says 172.21.15.100:35212) 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f1683fff640 1 -- 172.21.15.100:0/961839347 learned_addr learned my addr 172.21.15.100:0/961839347 (peer_addr_for_me v1:172.21.15.100:0/0) 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f16837fe640 1 --2- 172.21.15.100:0/961839347 >> v2:172.21.15.60:3300/0 conn(0x7f168406dc70 0x7f168415dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f168888f640 1 -- 172.21.15.100:0/961839347 >> v2:172.21.15.60:3300/0 conn(0x7f168406dc70 msgr2=0x7f168415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f1682ffd640 1 -- 172.21.15.100:0/961839347 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320413842 0 0) 0x7f168413d440 con 0x7f168413d060 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f168888f640 1 --2- 172.21.15.100:0/961839347 >> v2:172.21.15.60:3300/0 conn(0x7f168406dc70 0x7f168415dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f168888f640 1 -- 172.21.15.100:0/961839347 >> v1:172.21.15.100:6789/0 conn(0x7f168413d060 legacy=0x7f168406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.235+0000 7f168888f640 1 -- 172.21.15.100:0/961839347 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168415a9e0 con 0x7f168415e220 2023-11-18T20:42:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.236+0000 7f168888f640 1 --2- 172.21.15.100:0/961839347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168415e220 0x7f16840ceaa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f167c009d90 tx=0x7f167c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea34b55aac4a031 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.236+0000 7f1682ffd640 1 -- 172.21.15.100:0/961839347 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c02cc50 con 0x7f168415e220 2023-11-18T20:42:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.236+0000 7f1682ffd640 1 -- 172.21.15.100:0/961839347 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167c02cdb0 con 0x7f168415e220 2023-11-18T20:42:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.236+0000 7f1682ffd640 1 -- 172.21.15.100:0/961839347 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c034660 con 0x7f168415e220 2023-11-18T20:42:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.237+0000 7f168a2f2640 1 -- 172.21.15.100:0/961839347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168415e220 msgr2=0x7f16840ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.237+0000 7f168a2f2640 1 --2- 172.21.15.100:0/961839347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168415e220 0x7f16840ceaa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f167c009d90 tx=0x7f167c02bea0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.237+0000 7f168a2f2640 1 -- 172.21.15.100:0/961839347 shutdown_connections 2023-11-18T20:42:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.237+0000 7f168a2f2640 1 --2- 172.21.15.100:0/961839347 >> v2:172.21.15.60:3300/0 conn(0x7f168406dc70 0x7f168415dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.237+0000 7f168a2f2640 1 --2- 172.21.15.100:0/961839347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168415e220 0x7f16840ceaa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.237+0000 7f168a2f2640 1 -- 172.21.15.100:0/961839347 >> 172.21.15.100:0/961839347 conn(0x7f1684066270 msgr2=0x7f1684066650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.237+0000 7f168a2f2640 1 -- 172.21.15.100:0/961839347 shutdown_connections 2023-11-18T20:42:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.237+0000 7f168a2f2640 1 -- 172.21.15.100:0/961839347 wait complete. 2023-11-18T20:42:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.238+0000 7f168a2f2640 1 Processor -- start 2023-11-18T20:42:15.240 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.238+0000 7f168a2f2640 1 -- start start 2023-11-18T20:42:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f168a2f2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168406dc70 0x7f16841f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f168a2f2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168413d060 0x7f16841590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f168a2f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f168415e220 0x7f1684159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f168a2f2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16841f63f0 con 0x7f168413d060 2023-11-18T20:42:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f168a2f2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168413a980 con 0x7f168415e220 2023-11-18T20:42:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f168a2f2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168413ac20 con 0x7f168406dc70 2023-11-18T20:42:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f168888f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f168415e220 0x7f1684159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f1683fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168406dc70 0x7f16841f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.242 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f1683fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168406dc70 0x7f16841f5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:57752/0 (socket says 172.21.15.100:57752) 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f168888f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f168415e220 0x7f1684159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37340/0 (socket says 172.21.15.100:37340) 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f1683fff640 1 -- 172.21.15.100:0/1861709804 learned_addr learned my addr 172.21.15.100:0/1861709804 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f16837fe640 1 --2- 172.21.15.100:0/1861709804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168413d060 0x7f16841590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f1683fff640 1 -- 172.21.15.100:0/1861709804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f168415e220 msgr2=0x7f1684159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f1683fff640 1 --2- 172.21.15.100:0/1861709804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f168415e220 0x7f1684159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f1683fff640 1 -- 172.21.15.100:0/1861709804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168413d060 msgr2=0x7f16841590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f1683fff640 1 --2- 172.21.15.100:0/1861709804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168413d060 0x7f16841590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f1683fff640 1 -- 172.21.15.100:0/1861709804 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c009a70 con 0x7f168406dc70 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f1683fff640 1 --2- 172.21.15.100:0/1861709804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168406dc70 0x7f16841f5eb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f167800c4a0 tx=0x7f167800c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.239+0000 7f1680ff9640 1 -- 172.21.15.100:0/1861709804 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678005ce0 con 0x7f168406dc70 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.240+0000 7f1680ff9640 1 -- 172.21.15.100:0/1861709804 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1678005e40 con 0x7f168406dc70 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.240+0000 7f168a2f2640 1 -- 172.21.15.100:0/1861709804 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168413af00 con 0x7f168406dc70 2023-11-18T20:42:15.243 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.240+0000 7f1680ff9640 1 -- 172.21.15.100:0/1861709804 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16780072b0 con 0x7f168406dc70 2023-11-18T20:42:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.240+0000 7f168a2f2640 1 -- 172.21.15.100:0/1861709804 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1684150f40 con 0x7f168406dc70 2023-11-18T20:42:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.240+0000 7f1680ff9640 1 -- 172.21.15.100:0/1861709804 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f167801c440 con 0x7f168406dc70 2023-11-18T20:42:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.240+0000 7f1680ff9640 1 -- 172.21.15.100:0/1861709804 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f16780129f0 con 0x7f168406dc70 2023-11-18T20:42:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.240+0000 7f168a2f2640 1 -- 172.21.15.100:0/1861709804 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1684067420 con 0x7f168406dc70 2023-11-18T20:42:15.244 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.242+0000 7f1680ff9640 1 -- 172.21.15.100:0/1861709804 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f1678017070 con 0x7f168406dc70 2023-11-18T20:42:15.316 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.315+0000 7f168a2f2640 1 -- 172.21.15.100:0/1861709804 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "83e97732-ae79-4373-9313-65910a99e0d6", "id": 6} v 0) v1 -- 0x7f16840b2f70 con 0x7f168406dc70 2023-11-18T20:42:15.319 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.318+0000 7f1680ff9640 1 -- 172.21.15.100:0/1861709804 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "83e97732-ae79-4373-9313-65910a99e0d6", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1678012400 con 0x7f168406dc70 2023-11-18T20:42:15.319 INFO:teuthology.orchestra.run.smithi100.stdout:6 2023-11-18T20:42:15.320 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.319+0000 7f168a2f2640 1 -- 172.21.15.100:0/1861709804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168406dc70 msgr2=0x7f16841f5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.320 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.319+0000 7f168a2f2640 1 --2- 172.21.15.100:0/1861709804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168406dc70 0x7f16841f5eb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f167800c4a0 tx=0x7f167800c960 comp rx=0 tx=0).stop 2023-11-18T20:42:15.320 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.319+0000 7f168a2f2640 1 -- 172.21.15.100:0/1861709804 shutdown_connections 2023-11-18T20:42:15.320 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.319+0000 7f168a2f2640 1 --2- 172.21.15.100:0/1861709804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f168415e220 0x7f1684159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.320 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.319+0000 7f168a2f2640 1 --2- 172.21.15.100:0/1861709804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168413d060 0x7f16841590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.320 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.319+0000 7f168a2f2640 1 --2- 172.21.15.100:0/1861709804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f168406dc70 0x7f16841f5eb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.321 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.319+0000 7f168a2f2640 1 -- 172.21.15.100:0/1861709804 >> 172.21.15.100:0/1861709804 conn(0x7f1684066270 msgr2=0x7f16841523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:15.321 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.319+0000 7f168a2f2640 1 -- 172.21.15.100:0/1861709804 shutdown_connections 2023-11-18T20:42:15.321 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.319+0000 7f168a2f2640 1 -- 172.21.15.100:0/1861709804 wait complete. 2023-11-18T20:42:15.331 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 228baf66-5cfe-414c-aeea-92ebcd6cbcad 7 2023-11-18T20:42:15.339 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:15.339+0000 7f5528c32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-18T20:42:15.418 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:15.417+0000 7f34bfa2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-18T20:42:15.431 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.430+0000 7f64cbb90640 1 Processor -- start 2023-11-18T20:42:15.431 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.430+0000 7f64cbb90640 1 -- start start 2023-11-18T20:42:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64cbb90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c4155960 0x7f64c406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64cbb90640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f64c415c370 0x7f64c4070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.438 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64cbb90640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64c415ec90 0x7f64c415f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64cbb90640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c415fc60 con 0x7f64c415ec90 2023-11-18T20:42:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64cbb90640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c415b640 con 0x7f64c415c370 2023-11-18T20:42:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64cbb90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c415f980 con 0x7f64c4155960 2023-11-18T20:42:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c992c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c4155960 0x7f64c406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.439 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c912b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f64c415c370 0x7f64c4070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.440 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c992c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c4155960 0x7f64c406fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:57754/0 (socket says 172.21.15.100:57754) 2023-11-18T20:42:15.440 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c912b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f64c415c370 0x7f64c4070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37342/0 (socket says 172.21.15.100:37342) 2023-11-18T20:42:15.440 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c992c640 1 -- 172.21.15.100:0/964462264 learned_addr learned my addr 172.21.15.100:0/964462264 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:15.440 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64ca12d640 1 --2- 172.21.15.100:0/964462264 >> v2:172.21.15.60:3300/0 conn(0x7f64c415ec90 0x7f64c415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.440 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c912b640 1 -- 172.21.15.100:0/964462264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c4155960 msgr2=0x7f64c406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:15.440 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c912b640 1 --2- 172.21.15.100:0/964462264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c4155960 0x7f64c406fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.440 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c912b640 1 -- 172.21.15.100:0/964462264 >> v2:172.21.15.60:3300/0 conn(0x7f64c415ec90 msgr2=0x7f64c415f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c912b640 1 --2- 172.21.15.100:0/964462264 >> v2:172.21.15.60:3300/0 conn(0x7f64c415ec90 0x7f64c415f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c912b640 1 -- 172.21.15.100:0/964462264 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c415b890 con 0x7f64c415c370 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.431+0000 7f64c992c640 1 --2- 172.21.15.100:0/964462264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c4155960 0x7f64c406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.432+0000 7f64c912b640 1 --2- 172.21.15.100:0/964462264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64c415c370 0x7f64c4070370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f64c4066e00 tx=0x7f64b402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de36033fdf6d887 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.432+0000 7f64c892a640 1 -- 172.21.15.100:0/964462264 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b402e070 con 0x7f64c415c370 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.432+0000 7f64c892a640 1 -- 172.21.15.100:0/964462264 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b40315a0 con 0x7f64c415c370 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.432+0000 7f64c892a640 1 -- 172.21.15.100:0/964462264 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4031930 con 0x7f64c415c370 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.433+0000 7f64cbb90640 1 -- 172.21.15.100:0/964462264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64c415c370 msgr2=0x7f64c4070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.433+0000 7f64cbb90640 1 --2- 172.21.15.100:0/964462264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64c415c370 0x7f64c4070370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f64c4066e00 tx=0x7f64b402b840 comp rx=0 tx=0).stop 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.433+0000 7f64cbb90640 1 -- 172.21.15.100:0/964462264 shutdown_connections 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.433+0000 7f64cbb90640 1 --2- 172.21.15.100:0/964462264 >> v2:172.21.15.60:3300/0 conn(0x7f64c415ec90 0x7f64c415f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.433+0000 7f64cbb90640 1 --2- 172.21.15.100:0/964462264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64c415c370 0x7f64c4070370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.433+0000 7f64cbb90640 1 --2- 172.21.15.100:0/964462264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c4155960 0x7f64c406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.441 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.433+0000 7f64cbb90640 1 -- 172.21.15.100:0/964462264 >> 172.21.15.100:0/964462264 conn(0x7f64c406e7d0 msgr2=0x7f64c406ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.433+0000 7f64cbb90640 1 -- 172.21.15.100:0/964462264 shutdown_connections 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.433+0000 7f64cbb90640 1 -- 172.21.15.100:0/964462264 wait complete. 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.434+0000 7f64cbb90640 1 Processor -- start 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.434+0000 7f64cbb90640 1 -- start start 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.434+0000 7f64cbb90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64c4155960 0x7f64c415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.434+0000 7f64cbb90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64c415c370 0x7f64c413d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.434+0000 7f64cbb90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c415ec90 0x7f64c4156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64cbb90640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c415a810 con 0x7f64c4155960 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64cbb90640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c4157980 con 0x7f64c415c370 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64cbb90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c41598b0 con 0x7f64c415ec90 2023-11-18T20:42:15.442 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64c912b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64c415c370 0x7f64c413d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ca12d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c415ec90 0x7f64c4156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ca12d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c415ec90 0x7f64c4156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:57756/0 (socket says 172.21.15.100:57756) 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ca12d640 1 -- 172.21.15.100:0/4153712905 learned_addr learned my addr 172.21.15.100:0/4153712905 (peer_addr_for_me v2:172.21.15.100:0/0) 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64c912b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64c415c370 0x7f64c413d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37356/0 (socket says 172.21.15.100:37356) 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64c992c640 1 --2- 172.21.15.100:0/4153712905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64c4155960 0x7f64c415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ca12d640 1 -- 172.21.15.100:0/4153712905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64c415c370 msgr2=0x7f64c413d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ca12d640 1 --2- 172.21.15.100:0/4153712905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64c415c370 0x7f64c413d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ca12d640 1 -- 172.21.15.100:0/4153712905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64c4155960 msgr2=0x7f64c415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ca12d640 1 --2- 172.21.15.100:0/4153712905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64c4155960 0x7f64c415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ca12d640 1 -- 172.21.15.100:0/4153712905 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c00094d0 con 0x7f64c415ec90 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ca12d640 1 --2- 172.21.15.100:0/4153712905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c415ec90 0x7f64c4156770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00efc0 tx=0x7f64bc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ba7fc640 1 -- 172.21.15.100:0/4153712905 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00ccd0 con 0x7f64c415ec90 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ba7fc640 1 -- 172.21.15.100:0/4153712905 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc00f040 con 0x7f64c415ec90 2023-11-18T20:42:15.443 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ba7fc640 1 -- 172.21.15.100:0/4153712905 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc014480 con 0x7f64c415ec90 2023-11-18T20:42:15.444 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64cbb90640 1 -- 172.21.15.100:0/4153712905 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b4009400 con 0x7f64c415ec90 2023-11-18T20:42:15.444 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64cbb90640 1 -- 172.21.15.100:0/4153712905 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c4159e70 con 0x7f64c415ec90 2023-11-18T20:42:15.444 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ba7fc640 1 -- 172.21.15.100:0/4153712905 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f64bc0148e0 con 0x7f64c415ec90 2023-11-18T20:42:15.444 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.435+0000 7f64ba7fc640 1 -- 172.21.15.100:0/4153712905 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f64bc013a40 con 0x7f64c415ec90 2023-11-18T20:42:15.444 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.436+0000 7f64cbb90640 1 -- 172.21.15.100:0/4153712905 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c415a950 con 0x7f64c415ec90 2023-11-18T20:42:15.444 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.438+0000 7f64ba7fc640 1 -- 172.21.15.100:0/4153712905 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f64bc018020 con 0x7f64c415ec90 2023-11-18T20:42:15.481 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:15.480+0000 7f5528c32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-18T20:42:15.511 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.510+0000 7f64cbb90640 1 -- 172.21.15.100:0/4153712905 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "228baf66-5cfe-414c-aeea-92ebcd6cbcad", "id": 7} v 0) v1 -- 0x7f64c4067420 con 0x7f64c415ec90 2023-11-18T20:42:15.514 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.514+0000 7f64ba7fc640 1 -- 172.21.15.100:0/4153712905 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "228baf66-5cfe-414c-aeea-92ebcd6cbcad", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f64bc013ce0 con 0x7f64c415ec90 2023-11-18T20:42:15.515 INFO:teuthology.orchestra.run.smithi100.stdout:7 2023-11-18T20:42:15.516 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.515+0000 7f64cbb90640 1 -- 172.21.15.100:0/4153712905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c415ec90 msgr2=0x7f64c4156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.516 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.515+0000 7f64cbb90640 1 --2- 172.21.15.100:0/4153712905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c415ec90 0x7f64c4156770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00efc0 tx=0x7f64bc00c490 comp rx=0 tx=0).stop 2023-11-18T20:42:15.516 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.515+0000 7f64cbb90640 1 -- 172.21.15.100:0/4153712905 shutdown_connections 2023-11-18T20:42:15.516 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.515+0000 7f64cbb90640 1 --2- 172.21.15.100:0/4153712905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64c415ec90 0x7f64c4156770 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.516 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.515+0000 7f64cbb90640 1 --2- 172.21.15.100:0/4153712905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64c415c370 0x7f64c413d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.516 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.515+0000 7f64cbb90640 1 --2- 172.21.15.100:0/4153712905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64c4155960 0x7f64c415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.517 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.515+0000 7f64cbb90640 1 -- 172.21.15.100:0/4153712905 >> 172.21.15.100:0/4153712905 conn(0x7f64c406e7d0 msgr2=0x7f64c40b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:15.517 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.515+0000 7f64cbb90640 1 -- 172.21.15.100:0/4153712905 shutdown_connections 2023-11-18T20:42:15.517 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:15.515+0000 7f64cbb90640 1 -- 172.21.15.100:0/4153712905 wait complete. 2023-11-18T20:42:15.526 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-18T20:42:15.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-18T20:42:15.528 INFO:tasks.ceph.osd.0:Started 2023-11-18T20:42:15.529 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-18T20:42:15.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-18T20:42:15.531 INFO:tasks.ceph.osd.1:Started 2023-11-18T20:42:15.531 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-18T20:42:15.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-18T20:42:15.533 INFO:tasks.ceph.osd.2:Started 2023-11-18T20:42:15.533 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-18T20:42:15.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-18T20:42:15.536 INFO:tasks.ceph.osd.3:Started 2023-11-18T20:42:15.536 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-18T20:42:15.536 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-18T20:42:15.568 INFO:tasks.ceph.osd.4:Started 2023-11-18T20:42:15.569 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-18T20:42:15.569 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-18T20:42:15.570 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:15.568+0000 7f34bfa2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-18T20:42:15.571 INFO:tasks.ceph.osd.5:Started 2023-11-18T20:42:15.572 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-18T20:42:15.572 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-18T20:42:15.574 INFO:tasks.ceph.osd.6:Started 2023-11-18T20:42:15.574 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-18T20:42:15.574 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-18T20:42:15.576 INFO:tasks.ceph.osd.7:Started 2023-11-18T20:42:15.576 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:15.613 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:15.612+0000 7f5528c32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-18T20:42:15.678 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:15.677+0000 7f5528c32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-18T20:42:15.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.698+0000 7f1d2b62d640 1 Processor -- start 2023-11-18T20:42:15.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.698+0000 7f1d2b62d640 1 -- start start 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d2b62d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d2415d330 0x7f1d240b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d2b62d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1d240d3900 0x7f1d240b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d2b62d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d240b2b10 con 0x7f1d240d2910 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d2b62d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d240d1590 con 0x7f1d2415d330 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d2b62d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d240d1870 con 0x7f1d240d3900 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d293c9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d2415d330 0x7f1d240b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d293c9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d2415d330 0x7f1d240b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49198/0 (socket says 172.21.15.60:49198) 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d28bc8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1d240d2910 0x7f1d240d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40448/0 (socket says 172.21.15.60:40448) 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d293c9640 1 -- 172.21.15.60:0/3967909181 learned_addr learned my addr 172.21.15.60:0/3967909181 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d29bca640 1 --2- 172.21.15.60:0/3967909181 >> v2:172.21.15.100:3300/0 conn(0x7f1d240d3900 0x7f1d240b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d293c9640 1 -- 172.21.15.60:0/3967909181 >> v2:172.21.15.100:3300/0 conn(0x7f1d240d3900 msgr2=0x7f1d240b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:15.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d293c9640 1 --2- 172.21.15.60:0/3967909181 >> v2:172.21.15.100:3300/0 conn(0x7f1d240d3900 0x7f1d240b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d293c9640 1 -- 172.21.15.60:0/3967909181 >> v1:172.21.15.60:6789/0 conn(0x7f1d240d2910 legacy=0x7f1d240d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d293c9640 1 -- 172.21.15.60:0/3967909181 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d240d1b50 con 0x7f1d2415d330 2023-11-18T20:42:15.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.699+0000 7f1d17fff640 1 -- 172.21.15.60:0/3967909181 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239952653 0 0) 0x7f1d240b2b10 con 0x7f1d240d2910 2023-11-18T20:42:15.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d293c9640 1 --2- 172.21.15.60:0/3967909181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 0x7f1d240b8b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1d2406c500 tx=0x7f1d2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc591a5a8a72ff3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:15.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d17fff640 1 -- 172.21.15.60:0/3967909181 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20031870 con 0x7f1d2415d330 2023-11-18T20:42:15.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d17fff640 1 -- 172.21.15.60:0/3967909181 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d200319d0 con 0x7f1d2415d330 2023-11-18T20:42:15.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d17fff640 1 -- 172.21.15.60:0/3967909181 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20031d60 con 0x7f1d2415d330 2023-11-18T20:42:15.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d2b62d640 1 -- 172.21.15.60:0/3967909181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 msgr2=0x7f1d240b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d2b62d640 1 --2- 172.21.15.60:0/3967909181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 0x7f1d240b8b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1d2406c500 tx=0x7f1d2002b910 comp rx=0 tx=0).stop 2023-11-18T20:42:15.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d2b62d640 1 -- 172.21.15.60:0/3967909181 shutdown_connections 2023-11-18T20:42:15.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d2b62d640 1 --2- 172.21.15.60:0/3967909181 >> v2:172.21.15.100:3300/0 conn(0x7f1d240d3900 0x7f1d240b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d2b62d640 1 --2- 172.21.15.60:0/3967909181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 0x7f1d240b8b80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d2b62d640 1 -- 172.21.15.60:0/3967909181 >> 172.21.15.60:0/3967909181 conn(0x7f1d24072020 msgr2=0x7f1d24072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:15.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.701+0000 7f1d2b62d640 1 -- 172.21.15.60:0/3967909181 shutdown_connections 2023-11-18T20:42:15.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.702+0000 7f1d2b62d640 1 -- 172.21.15.60:0/3967909181 wait complete. 2023-11-18T20:42:15.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.703+0000 7f1d2b62d640 1 Processor -- start 2023-11-18T20:42:15.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.703+0000 7f1d2b62d640 1 -- start start 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.703+0000 7f1d2b62d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d240d2910 0x7f1d24154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.703+0000 7f1d2b62d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d240d3900 0x7f1d241f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.703+0000 7f1d2b62d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 0x7f1d2415f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.703+0000 7f1d2b62d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240c7630 con 0x7f1d2415d330 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.703+0000 7f1d2b62d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240c9b90 con 0x7f1d240d3900 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.703+0000 7f1d2b62d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240c6cd0 con 0x7f1d240d2910 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.703+0000 7f1d29bca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 0x7f1d2415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d28bc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d240d3900 0x7f1d241f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d293c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d240d2910 0x7f1d24154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d29bca640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 0x7f1d2415f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49200/0 (socket says 172.21.15.60:49200) 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d29bca640 1 -- 172.21.15.60:0/1896914084 learned_addr learned my addr 172.21.15.60:0/1896914084 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d29bca640 1 -- 172.21.15.60:0/1896914084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d240d2910 msgr2=0x7f1d24154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d29bca640 1 --2- 172.21.15.60:0/1896914084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d240d2910 0x7f1d24154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d29bca640 1 -- 172.21.15.60:0/1896914084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d240d3900 msgr2=0x7f1d241f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d29bca640 1 --2- 172.21.15.60:0/1896914084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d240d3900 0x7f1d241f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d29bca640 1 -- 172.21.15.60:0/1896914084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d200094d0 con 0x7f1d2415d330 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d28bc8640 1 --2- 172.21.15.60:0/1896914084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d240d3900 0x7f1d241f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d29bca640 1 --2- 172.21.15.60:0/1896914084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 0x7f1d2415f350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00efc0 tx=0x7f1d1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d15ffb640 1 -- 172.21.15.60:0/1896914084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c009ca0 con 0x7f1d2415d330 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d15ffb640 1 -- 172.21.15.60:0/1896914084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c00f040 con 0x7f1d2415d330 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d2b62d640 1 -- 172.21.15.60:0/1896914084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d24154d70 con 0x7f1d2415d330 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d15ffb640 1 -- 172.21.15.60:0/1896914084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0146c0 con 0x7f1d2415d330 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.704+0000 7f1d2b62d640 1 -- 172.21.15.60:0/1896914084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d240c7250 con 0x7f1d2415d330 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.705+0000 7f1d15ffb640 1 -- 172.21.15.60:0/1896914084 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1d1c014b20 con 0x7f1d2415d330 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.705+0000 7f1d15ffb640 1 -- 172.21.15.60:0/1896914084 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f1d1c01dd00 con 0x7f1d2415d330 2023-11-18T20:42:15.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.705+0000 7f1d2b62d640 1 -- 172.21.15.60:0/1896914084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d240c7770 con 0x7f1d2415d330 2023-11-18T20:42:15.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.708+0000 7f1d15ffb640 1 -- 172.21.15.60:0/1896914084 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f1d1c03bce0 con 0x7f1d2415d330 2023-11-18T20:42:15.710 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:15.709+0000 7f34bfa2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-18T20:42:15.758 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:15.757+0000 7f34bfa2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-18T20:42:15.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.770+0000 7f1d2b62d640 1 -- 172.21.15.60:0/1896914084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d24069400 con 0x7f1d2415d330 2023-11-18T20:42:15.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.770+0000 7f1d15ffb640 1 -- 172.21.15.60:0/1896914084 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1d1c02a070 con 0x7f1d2415d330 2023-11-18T20:42:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:15.771 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":10,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:15.511939+0000","last_up_change":"0.000000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:15.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.771+0000 7f1d2b62d640 1 -- 172.21.15.60:0/1896914084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 msgr2=0x7f1d2415f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:15.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.771+0000 7f1d2b62d640 1 --2- 172.21.15.60:0/1896914084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 0x7f1d2415f350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00efc0 tx=0x7f1d1c00c490 comp rx=0 tx=0).stop 2023-11-18T20:42:15.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.771+0000 7f1d2b62d640 1 -- 172.21.15.60:0/1896914084 shutdown_connections 2023-11-18T20:42:15.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.771+0000 7f1d2b62d640 1 --2- 172.21.15.60:0/1896914084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2415d330 0x7f1d2415f350 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.771+0000 7f1d2b62d640 1 --2- 172.21.15.60:0/1896914084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d240d3900 0x7f1d241f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.771+0000 7f1d2b62d640 1 --2- 172.21.15.60:0/1896914084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d240d2910 0x7f1d24154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:15.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.771+0000 7f1d2b62d640 1 -- 172.21.15.60:0/1896914084 >> 172.21.15.60:0/1896914084 conn(0x7f1d24072020 msgr2=0x7f1d240b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:15.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.771+0000 7f1d2b62d640 1 -- 172.21.15.60:0/1896914084 shutdown_connections 2023-11-18T20:42:15.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:15.771+0000 7f1d2b62d640 1 -- 172.21.15.60:0/1896914084 wait complete. 2023-11-18T20:42:15.775 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:15.774+0000 7f5528c32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-18T20:42:15.783 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-18T20:42:15.783 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-18T20:42:15.861 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:15.860+0000 7f34bfa2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-18T20:42:15.868 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:15.867+0000 7f5528c32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-18T20:42:15.960 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:15.959+0000 7f34bfa2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-18T20:42:16.009 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:16.008+0000 7f5528c32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-18T20:42:16.051 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:16.050+0000 7f5528c32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-18T20:42:16.106 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:16.105+0000 7f34bfa2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-18T20:42:16.137 INFO:tasks.ceph.osd.0.smithi060.stderr:2023-11-18T20:42:16.136+0000 7f40d080e500 -1 Falling back to public interface 2023-11-18T20:42:16.150 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:16.149+0000 7f34bfa2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-18T20:42:16.156 INFO:tasks.ceph.osd.1.smithi060.stderr:2023-11-18T20:42:16.155+0000 7f376d62a500 -1 Falling back to public interface 2023-11-18T20:42:16.157 INFO:tasks.ceph.osd.2.smithi060.stderr:2023-11-18T20:42:16.155+0000 7fede5c1a500 -1 Falling back to public interface 2023-11-18T20:42:16.157 INFO:tasks.ceph.osd.1.smithi060.stderr:2023-11-18T20:42:16.156+0000 7f376bb73640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6811/0: (98) Address already in use 2023-11-18T20:42:16.158 INFO:tasks.ceph.osd.1.smithi060.stderr:2023-11-18T20:42:16.157+0000 7f376bb73640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6819/0: (98) Address already in use 2023-11-18T20:42:16.164 INFO:tasks.ceph.osd.3.smithi060.stderr:2023-11-18T20:42:16.163+0000 7f3719cfa500 -1 Falling back to public interface 2023-11-18T20:42:16.175 INFO:tasks.ceph.osd.4.smithi100.stderr:2023-11-18T20:42:16.174+0000 7f310c023500 -1 Falling back to public interface 2023-11-18T20:42:16.176 INFO:tasks.ceph.osd.6.smithi100.stderr:2023-11-18T20:42:16.174+0000 7f688fd48500 -1 Falling back to public interface 2023-11-18T20:42:16.176 INFO:tasks.ceph.osd.6.smithi100.stderr:2023-11-18T20:42:16.174+0000 7f688ecef640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-11-18T20:42:16.176 INFO:tasks.ceph.osd.6.smithi100.stderr:2023-11-18T20:42:16.175+0000 7f688ecef640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6804/0: (98) Address already in use 2023-11-18T20:42:16.179 INFO:tasks.ceph.osd.5.smithi100.stderr:2023-11-18T20:42:16.178+0000 7f1033707500 -1 Falling back to public interface 2023-11-18T20:42:16.193 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:16.191+0000 7f5528c32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-18T20:42:16.200 INFO:tasks.ceph.osd.7.smithi100.stderr:2023-11-18T20:42:16.200+0000 7fb547cfa500 -1 Falling back to public interface 2023-11-18T20:42:16.300 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:16.299+0000 7f34bfa2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-18T20:42:16.589 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:16.682 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.681+0000 7f67dcc90640 1 Processor -- start 2023-11-18T20:42:16.682 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.681+0000 7f67dcc90640 1 -- start start 2023-11-18T20:42:16.685 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67dcc90640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67d80ad4c0 0x7f67d806f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:16.685 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67dcc90640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d815c580 con 0x7f67d806ab80 2023-11-18T20:42:16.685 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67dcc90640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d80ca6f0 con 0x7f67d806a7a0 2023-11-18T20:42:16.685 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67dcc90640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d815fac0 con 0x7f67d80ad4c0 2023-11-18T20:42:16.685 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67cffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f67d806ab80 0x7f67d8151a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:40464/0 (socket says 172.21.15.60:40464) 2023-11-18T20:42:16.685 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67cffff640 1 -- 172.21.15.60:0/270131977 learned_addr learned my addr 172.21.15.60:0/270131977 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:16.685 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67d6d9d640 1 --2- 172.21.15.60:0/270131977 >> v2:172.21.15.100:3300/0 conn(0x7f67d80ad4c0 0x7f67d806f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:16.685 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148116410 0 0) 0x7f67d815c580 con 0x7f67d806ab80 2023-11-18T20:42:16.685 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c4003520 con 0x7f67d806ab80 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 651376575 0 0) 0x7f67c4003520 con 0x7f67d806ab80 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 >> v2:172.21.15.100:3300/0 conn(0x7f67d80ad4c0 msgr2=0x7f67d806f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67d5d9b640 1 --2- 172.21.15.60:0/270131977 >> v2:172.21.15.100:3300/0 conn(0x7f67d80ad4c0 0x7f67d806f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 >> v1:172.21.15.100:6789/0 conn(0x7f67d806a7a0 legacy=0x7f67d80acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d815ed00 con 0x7f67d806ab80 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f67c0001670 con 0x7f67d806ab80 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.682+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 >> v1:172.21.15.60:6789/0 conn(0x7f67d806ab80 legacy=0x7f67d8151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d5d9b640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c4003970 0x7f67c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d5d9b640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c40047c0 0x7f67c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d5d9b640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4005180 0x7f67c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d815fac0 con 0x7f67c4005180 2023-11-18T20:42:16.686 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c400c020 con 0x7f67c4003970 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c400c1e0 con 0x7f67c40047c0 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d6d9d640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4005180 0x7f67c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67cffff640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c40047c0 0x7f67c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d659c640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c4003970 0x7f67c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d6d9d640 1 -- 172.21.15.60:0/270131977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c40047c0 msgr2=0x7f67c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d6d9d640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c40047c0 0x7f67c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d6d9d640 1 -- 172.21.15.60:0/270131977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c4003970 msgr2=0x7f67c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d6d9d640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67c4003970 0x7f67c4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.683+0000 7f67d6d9d640 1 -- 172.21.15.60:0/270131977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c4003520 con 0x7f67c4005180 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.684+0000 7f67d6d9d640 1 -- 172.21.15.60:0/270131977 reap_dead start 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.684+0000 7f67d6d9d640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4005180 0x7f67c400b980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009fc0 tx=0x7f67c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc1d137be18f0ff server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.684+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d002cc60 con 0x7f67c4005180 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.684+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d000f040 con 0x7f67c4005180 2023-11-18T20:42:16.687 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.684+0000 7f67d5d9b640 1 -- 172.21.15.60:0/270131977 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0003990 con 0x7f67c4005180 2023-11-18T20:42:16.690 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.685+0000 7f67dcc90640 1 -- 172.21.15.60:0/270131977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4005180 msgr2=0x7f67c400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:16.690 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.685+0000 7f67dcc90640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4005180 0x7f67c400b980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009fc0 tx=0x7f67c40057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:16.690 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.685+0000 7f67dcc90640 1 -- 172.21.15.60:0/270131977 shutdown_connections 2023-11-18T20:42:16.690 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.685+0000 7f67dcc90640 1 --2- 172.21.15.60:0/270131977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67c4005180 0x7f67c400b980 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:16.690 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.685+0000 7f67dcc90640 1 -- 172.21.15.60:0/270131977 >> 172.21.15.60:0/270131977 conn(0x7f67d80cc910 msgr2=0x7f67d80ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.685+0000 7f67dcc90640 1 -- 172.21.15.60:0/270131977 shutdown_connections 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.685+0000 7f67dcc90640 1 -- 172.21.15.60:0/270131977 wait complete. 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.686+0000 7f67dcc90640 1 Processor -- start 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.686+0000 7f67dcc90640 1 -- start start 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67dcc90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c4005180 0x7f67d80c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67dcc90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d80c5830 0x7f67d815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67dcc90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8147090 0x7f67d8143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67dcc90640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80c4440 con 0x7f67d80c5830 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67dcc90640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80c6110 con 0x7f67d8147090 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67dcc90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80c63f0 con 0x7f67c4005180 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67cffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d80c5830 0x7f67d815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67cffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d80c5830 0x7f67d815a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49220/0 (socket says 172.21.15.60:49220) 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67cffff640 1 -- 172.21.15.60:0/3732841326 learned_addr learned my addr 172.21.15.60:0/3732841326 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67d659c640 1 --2- 172.21.15.60:0/3732841326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c4005180 0x7f67d80c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:16.691 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67d6d9d640 1 --2- 172.21.15.60:0/3732841326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8147090 0x7f67d8143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67cffff640 1 -- 172.21.15.60:0/3732841326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c4005180 msgr2=0x7f67d80c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67cffff640 1 --2- 172.21.15.60:0/3732841326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c4005180 0x7f67d80c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67cffff640 1 -- 172.21.15.60:0/3732841326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8147090 msgr2=0x7f67d8143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67cffff640 1 --2- 172.21.15.60:0/3732841326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8147090 0x7f67d8143bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67cffff640 1 -- 172.21.15.60:0/3732841326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d80c66d0 con 0x7f67d80c5830 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.687+0000 7f67cffff640 1 --2- 172.21.15.60:0/3732841326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d80c5830 0x7f67d815a2b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007290 tx=0x7f67c0007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.688+0000 7f67ceffd640 1 -- 172.21.15.60:0/3732841326 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0001650 con 0x7f67d80c5830 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.688+0000 7f67ceffd640 1 -- 172.21.15.60:0/3732841326 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c000f940 con 0x7f67d80c5830 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.688+0000 7f67dcc90640 1 -- 172.21.15.60:0/3732841326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d8146890 con 0x7f67d80c5830 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.688+0000 7f67ceffd640 1 -- 172.21.15.60:0/3732841326 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c000fcf0 con 0x7f67d80c5830 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.688+0000 7f67dcc90640 1 -- 172.21.15.60:0/3732841326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d815a7f0 con 0x7f67d80c5830 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.688+0000 7f67ceffd640 1 -- 172.21.15.60:0/3732841326 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f67c001d760 con 0x7f67d80c5830 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.688+0000 7f67ceffd640 1 -- 172.21.15.60:0/3732841326 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f67c0013c00 con 0x7f67d80c5830 2023-11-18T20:42:16.692 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.689+0000 7f67dcc90640 1 -- 172.21.15.60:0/3732841326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d8067420 con 0x7f67d80c5830 2023-11-18T20:42:16.693 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.691+0000 7f67ceffd640 1 -- 172.21.15.60:0/3732841326 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f67c0013530 con 0x7f67d80c5830 2023-11-18T20:42:16.755 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.754+0000 7f67dcc90640 1 -- 172.21.15.60:0/3732841326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67d80c2e40 con 0x7f67d80c5830 2023-11-18T20:42:16.755 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.754+0000 7f67ceffd640 1 -- 172.21.15.60:0/3732841326 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f67c001db50 con 0x7f67d80c5830 2023-11-18T20:42:16.755 INFO:teuthology.misc.health.smithi060.stdout: 2023-11-18T20:42:16.756 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":10,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:15.511939+0000","last_up_change":"0.000000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:16.757 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.755+0000 7f67dcc90640 1 -- 172.21.15.60:0/3732841326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d80c5830 msgr2=0x7f67d815a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:16.757 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.755+0000 7f67dcc90640 1 --2- 172.21.15.60:0/3732841326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d80c5830 0x7f67d815a2b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007290 tx=0x7f67c0007750 comp rx=0 tx=0).stop 2023-11-18T20:42:16.757 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.755+0000 7f67dcc90640 1 -- 172.21.15.60:0/3732841326 shutdown_connections 2023-11-18T20:42:16.757 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.755+0000 7f67dcc90640 1 --2- 172.21.15.60:0/3732841326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d8147090 0x7f67d8143bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:16.757 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.755+0000 7f67dcc90640 1 --2- 172.21.15.60:0/3732841326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d80c5830 0x7f67d815a2b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:16.757 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.755+0000 7f67dcc90640 1 --2- 172.21.15.60:0/3732841326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67c4005180 0x7f67d80c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:16.757 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.755+0000 7f67dcc90640 1 -- 172.21.15.60:0/3732841326 >> 172.21.15.60:0/3732841326 conn(0x7f67d80cc910 msgr2=0x7f67d806d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:16.757 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.755+0000 7f67dcc90640 1 -- 172.21.15.60:0/3732841326 shutdown_connections 2023-11-18T20:42:16.757 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:16.755+0000 7f67dcc90640 1 -- 172.21.15.60:0/3732841326 wait complete. 2023-11-18T20:42:16.765 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-18T20:42:16.988 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:16.987+0000 7f5528c32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-18T20:42:17.146 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:17.145+0000 7f34bfa2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-18T20:42:17.256 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:17.255+0000 7f5528c32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-18T20:42:17.308 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:17.308+0000 7f5528c32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-18T20:42:17.408 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:17.407+0000 7f5528c32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-18T20:42:17.431 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:17.430+0000 7f34bfa2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-18T20:42:17.486 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:17.485+0000 7f34bfa2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-18T20:42:17.591 INFO:tasks.ceph.mgr.y.smithi060.stderr:2023-11-18T20:42:17.590+0000 7f34bfa2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-18T20:42:18.437 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:18.436+0000 7f54c98ab640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-18T20:42:18.437 INFO:tasks.ceph.mgr.x.smithi100.stderr:2023-11-18T20:42:18.436+0000 7f54c98ab640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-18T20:42:19.632 INFO:tasks.ceph.osd.4.smithi100.stderr:2023-11-18T20:42:19.631+0000 7f310c023500 -1 osd.4 0 log_to_monitors true 2023-11-18T20:42:19.638 INFO:tasks.ceph.osd.0.smithi060.stderr:2023-11-18T20:42:19.637+0000 7f40d080e500 -1 osd.0 0 log_to_monitors true 2023-11-18T20:42:19.656 INFO:tasks.ceph.osd.1.smithi060.stderr:2023-11-18T20:42:19.655+0000 7f376d62a500 -1 osd.1 0 log_to_monitors true 2023-11-18T20:42:19.670 INFO:tasks.ceph.osd.3.smithi060.stderr:2023-11-18T20:42:19.669+0000 7f3719cfa500 -1 osd.3 0 log_to_monitors true 2023-11-18T20:42:19.681 INFO:tasks.ceph.osd.2.smithi060.stderr:2023-11-18T20:42:19.679+0000 7fede5c1a500 -1 osd.2 0 log_to_monitors true 2023-11-18T20:42:19.682 INFO:tasks.ceph.osd.6.smithi100.stderr:2023-11-18T20:42:19.681+0000 7f688fd48500 -1 osd.6 0 log_to_monitors true 2023-11-18T20:42:19.696 INFO:tasks.ceph.osd.5.smithi100.stderr:2023-11-18T20:42:19.695+0000 7f1033707500 -1 osd.5 0 log_to_monitors true 2023-11-18T20:42:19.728 INFO:tasks.ceph.osd.7.smithi100.stderr:2023-11-18T20:42:19.727+0000 7fb547cfa500 -1 osd.7 0 log_to_monitors true 2023-11-18T20:42:22.952 INFO:tasks.ceph.osd.3.smithi060.stderr:2023-11-18T20:42:22.950+0000 7f3715465640 -1 osd.3 0 waiting for initial osdmap 2023-11-18T20:42:22.966 INFO:tasks.ceph.osd.3.smithi060.stderr:2023-11-18T20:42:22.965+0000 7f370da9e640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-18T20:42:23.001 INFO:tasks.ceph.osd.1.smithi060.stderr:2023-11-18T20:42:23.000+0000 7f3768337640 -1 osd.1 0 waiting for initial osdmap 2023-11-18T20:42:23.020 INFO:tasks.ceph.osd.2.smithi060.stderr:2023-11-18T20:42:23.019+0000 7fede0de2640 -1 osd.2 0 waiting for initial osdmap 2023-11-18T20:42:23.023 INFO:tasks.ceph.osd.1.smithi060.stderr:2023-11-18T20:42:23.022+0000 7f3760970640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-18T20:42:23.037 INFO:tasks.ceph.osd.2.smithi060.stderr:2023-11-18T20:42:23.036+0000 7fedd941b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-18T20:42:23.054 INFO:tasks.ceph.osd.0.smithi060.stderr:2023-11-18T20:42:23.053+0000 7f40cbf79640 -1 osd.0 0 waiting for initial osdmap 2023-11-18T20:42:23.069 INFO:tasks.ceph.osd.0.smithi060.stderr:2023-11-18T20:42:23.068+0000 7f40c45b2640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-18T20:42:23.473 INFO:tasks.ceph.osd.6.smithi100.stderr:2023-11-18T20:42:23.472+0000 7f688b4b3640 -1 osd.6 0 waiting for initial osdmap 2023-11-18T20:42:23.542 INFO:tasks.ceph.osd.6.smithi100.stderr:2023-11-18T20:42:23.488+0000 7f6883aec640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-18T20:42:23.543 INFO:tasks.ceph.osd.4.smithi100.stderr:2023-11-18T20:42:23.498+0000 7f31071f0640 -1 osd.4 0 waiting for initial osdmap 2023-11-18T20:42:23.543 INFO:tasks.ceph.osd.4.smithi100.stderr:2023-11-18T20:42:23.506+0000 7f30ff829640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-18T20:42:23.543 INFO:tasks.ceph.osd.7.smithi100.stderr:2023-11-18T20:42:23.526+0000 7fb544c9f640 -1 osd.7 0 waiting for initial osdmap 2023-11-18T20:42:23.546 INFO:tasks.ceph.osd.7.smithi100.stderr:2023-11-18T20:42:23.538+0000 7fb53baa0640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-18T20:42:23.563 INFO:tasks.ceph.osd.5.smithi100.stderr:2023-11-18T20:42:23.562+0000 7f10306aa640 -1 osd.5 0 waiting for initial osdmap 2023-11-18T20:42:23.571 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:23.579 INFO:tasks.ceph.osd.5.smithi100.stderr:2023-11-18T20:42:23.578+0000 7f10274ab640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-18T20:42:23.669 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.667+0000 7fe298075640 1 Processor -- start 2023-11-18T20:42:23.669 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.668+0000 7fe298075640 1 -- start start 2023-11-18T20:42:23.672 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.668+0000 7fe298075640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe290161b90 0x7fe2900d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:23.672 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.668+0000 7fe298075640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2900d2f50 0x7fe2900d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:23.672 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.668+0000 7fe298075640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2900ce0e0 con 0x7fe29015d4f0 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.668+0000 7fe298075640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29015d8d0 con 0x7fe290161b90 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.668+0000 7fe298075640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900cddb0 con 0x7fe2900d2f50 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.668+0000 7fe295610640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe290161b90 0x7fe2900d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.669+0000 7fe295610640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe290161b90 0x7fe2900d1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60676/0 (socket says 172.21.15.60:60676) 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.669+0000 7fe295e11640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe29015d4f0 0x7fe2900b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34164/0 (socket says 172.21.15.60:34164) 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.669+0000 7fe295610640 1 -- 172.21.15.60:0/4218859784 learned_addr learned my addr 172.21.15.60:0/4218859784 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.669+0000 7fe296612640 1 --2- 172.21.15.60:0/4218859784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2900d2f50 0x7fe2900d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.669+0000 7fe295610640 1 -- 172.21.15.60:0/4218859784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2900d2f50 msgr2=0x7fe2900d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.669+0000 7fe295610640 1 --2- 172.21.15.60:0/4218859784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2900d2f50 0x7fe2900d33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.669+0000 7fe295610640 1 -- 172.21.15.60:0/4218859784 >> v1:172.21.15.60:6789/0 conn(0x7fe29015d4f0 legacy=0x7fe2900b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.669+0000 7fe294e0f640 1 -- 172.21.15.60:0/4218859784 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858087326 0 0) 0x7fe2900ce0e0 con 0x7fe29015d4f0 2023-11-18T20:42:23.673 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.669+0000 7fe295610640 1 -- 172.21.15.60:0/4218859784 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900d3920 con 0x7fe290161b90 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe295610640 1 --2- 172.21.15.60:0/4218859784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161b90 0x7fe2900d1ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe280009870 tx=0x7fe28002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff71c49d0e8bfe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe294e0f640 1 -- 172.21.15.60:0/4218859784 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280030840 con 0x7fe290161b90 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe294e0f640 1 -- 172.21.15.60:0/4218859784 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2800309a0 con 0x7fe290161b90 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe294e0f640 1 -- 172.21.15.60:0/4218859784 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280030d30 con 0x7fe290161b90 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe298075640 1 -- 172.21.15.60:0/4218859784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161b90 msgr2=0x7fe2900d1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe298075640 1 --2- 172.21.15.60:0/4218859784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161b90 0x7fe2900d1ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe280009870 tx=0x7fe28002b980 comp rx=0 tx=0).stop 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe298075640 1 -- 172.21.15.60:0/4218859784 shutdown_connections 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe298075640 1 --2- 172.21.15.60:0/4218859784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2900d2f50 0x7fe2900d33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe298075640 1 --2- 172.21.15.60:0/4218859784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161b90 0x7fe2900d1ab0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe298075640 1 -- 172.21.15.60:0/4218859784 >> 172.21.15.60:0/4218859784 conn(0x7fe29008cbf0 msgr2=0x7fe29009c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe298075640 1 -- 172.21.15.60:0/4218859784 shutdown_connections 2023-11-18T20:42:23.674 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.670+0000 7fe298075640 1 -- 172.21.15.60:0/4218859784 wait complete. 2023-11-18T20:42:23.678 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.672+0000 7fe298075640 1 Processor -- start 2023-11-18T20:42:23.678 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.672+0000 7fe298075640 1 -- start start 2023-11-18T20:42:23.678 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.672+0000 7fe298075640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900d2f50 0x7fe29015b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:23.678 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.672+0000 7fe298075640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe29015d4f0 0x7fe2901e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:23.678 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.672+0000 7fe298075640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe290161b90 0x7fe2901e9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:23.678 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe298075640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29015fda0 con 0x7fe2900d2f50 2023-11-18T20:42:23.678 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe298075640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29015bbb0 con 0x7fe29015d4f0 2023-11-18T20:42:23.678 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe298075640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900cb5b0 con 0x7fe290161b90 2023-11-18T20:42:23.678 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe295e11640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900d2f50 0x7fe29015b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:23.679 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe295e11640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900d2f50 0x7fe29015b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60680/0 (socket says 172.21.15.60:60680) 2023-11-18T20:42:23.679 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe295e11640 1 -- 172.21.15.60:0/2926090361 learned_addr learned my addr 172.21.15.60:0/2926090361 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:23.679 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe296612640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe290161b90 0x7fe2901e9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:23.679 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe295610640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe29015d4f0 0x7fe2901e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:23.679 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe295e11640 1 -- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe290161b90 msgr2=0x7fe2901e9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:23.679 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe295e11640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe290161b90 0x7fe2901e9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:23.679 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe295e11640 1 -- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe29015d4f0 msgr2=0x7fe2901e8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:23.679 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe295e11640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe29015d4f0 0x7fe2901e8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:23.679 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe295e11640 1 -- 172.21.15.60:0/2926090361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2800094d0 con 0x7fe2900d2f50 2023-11-18T20:42:23.679 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe295e11640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900d2f50 0x7fe29015b670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00c4d0 tx=0x7fe28c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe2867fc640 1 -- 172.21.15.60:0/2926090361 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c0074e0 con 0x7fe2900d2f50 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe2867fc640 1 -- 172.21.15.60:0/2926090361 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c007660 con 0x7fe2900d2f50 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe298075640 1 -- 172.21.15.60:0/2926090361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29015f190 con 0x7fe2900d2f50 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe2867fc640 1 -- 172.21.15.60:0/2926090361 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c004a20 con 0x7fe2900d2f50 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.673+0000 7fe298075640 1 -- 172.21.15.60:0/2926090361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29015f750 con 0x7fe2900d2f50 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.674+0000 7fe298075640 1 -- 172.21.15.60:0/2926090361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe290069320 con 0x7fe2900d2f50 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.675+0000 7fe2867fc640 1 -- 172.21.15.60:0/2926090361 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fe28c004b80 con 0x7fe2900d2f50 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.675+0000 7fe2867fc640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe27406adb0 0x7fe27406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.675+0000 7fe2867fc640 1 -- 172.21.15.60:0/2926090361 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fe28c087860 con 0x7fe2900d2f50 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.675+0000 7fe295610640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe27406adb0 0x7fe27406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:23.680 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.676+0000 7fe295610640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe27406adb0 0x7fe27406d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe280036000 tx=0x7fe28003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:23.681 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.680+0000 7fe2867fc640 1 -- 172.21.15.60:0/2926090361 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe28c053830 con 0x7fe2900d2f50 2023-11-18T20:42:23.858 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.857+0000 7fe298075640 1 -- 172.21.15.60:0/2926090361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2900d4020 con 0x7fe2900d2f50 2023-11-18T20:42:23.858 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.857+0000 7fe2867fc640 1 -- 172.21.15.60:0/2926090361 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe28c018070 con 0x7fe2900d2f50 2023-11-18T20:42:23.858 INFO:teuthology.misc.health.smithi060.stdout: 2023-11-18T20:42:23.858 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":12,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:21.437748+0000","last_up_change":"0.000000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:23.860 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 -- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe27406adb0 msgr2=0x7fe27406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:23.860 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe27406adb0 0x7fe27406d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe280036000 tx=0x7fe28003b040 comp rx=0 tx=0).stop 2023-11-18T20:42:23.860 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 -- 172.21.15.60:0/2926090361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900d2f50 msgr2=0x7fe29015b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:23.860 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900d2f50 0x7fe29015b670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00c4d0 tx=0x7fe28c00c990 comp rx=0 tx=0).stop 2023-11-18T20:42:23.861 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 -- 172.21.15.60:0/2926090361 shutdown_connections 2023-11-18T20:42:23.861 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe27406adb0 0x7fe27406d270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:23.861 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe290161b90 0x7fe2901e9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:23.861 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe29015d4f0 0x7fe2901e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:23.861 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 --2- 172.21.15.60:0/2926090361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900d2f50 0x7fe29015b670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:23.861 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 -- 172.21.15.60:0/2926090361 >> 172.21.15.60:0/2926090361 conn(0x7fe29008cbf0 msgr2=0x7fe2900b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:23.861 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 -- 172.21.15.60:0/2926090361 shutdown_connections 2023-11-18T20:42:23.861 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:23.859+0000 7fe298075640 1 -- 172.21.15.60:0/2926090361 wait complete. 2023-11-18T20:42:23.870 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-18T20:42:30.675 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:30.779 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.778+0000 7f84cf4da640 1 Processor -- start 2023-11-18T20:42:30.779 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.778+0000 7f84cf4da640 1 -- start start 2023-11-18T20:42:30.780 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cf4da640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84c8143e50 0x7f84c815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cf4da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 0x7f84c80b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cf4da640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c815f450 con 0x7f84c80b31e0 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cf4da640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c80ccf50 con 0x7f84c8143e50 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cf4da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80b3670 con 0x7f84c80b1f60 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cd276640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84c8143e50 0x7f84c815ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cda77640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f84c80b31e0 0x7f84c806d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55140/0 (socket says 172.21.15.60:55140) 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cd276640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84c8143e50 0x7f84c815ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59760/0 (socket says 172.21.15.60:59760) 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cda77640 1 -- 172.21.15.60:0/1003198342 learned_addr learned my addr 172.21.15.60:0/1003198342 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cca75640 1 --2- 172.21.15.60:0/1003198342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 0x7f84c80b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cd276640 1 -- 172.21.15.60:0/1003198342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 msgr2=0x7f84c80b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.779+0000 7f84cd276640 1 --2- 172.21.15.60:0/1003198342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 0x7f84c80b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:30.781 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.780+0000 7f84cd276640 1 -- 172.21.15.60:0/1003198342 >> v1:172.21.15.60:6789/0 conn(0x7f84c80b31e0 legacy=0x7f84c806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:30.782 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.780+0000 7f84bffff640 1 -- 172.21.15.60:0/1003198342 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764799407 0 0) 0x7f84c815f450 con 0x7f84c80b31e0 2023-11-18T20:42:30.782 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.780+0000 7f84cd276640 1 -- 172.21.15.60:0/1003198342 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c815f730 con 0x7f84c8143e50 2023-11-18T20:42:30.783 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.780+0000 7f84cd276640 1 --2- 172.21.15.60:0/1003198342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8143e50 0x7f84c815ee70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f84c40097a0 tx=0x7f84c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa8c359e2c2eb83 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:30.783 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.780+0000 7f84bffff640 1 -- 172.21.15.60:0/1003198342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4031830 con 0x7f84c8143e50 2023-11-18T20:42:30.783 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.780+0000 7f84bffff640 1 -- 172.21.15.60:0/1003198342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c4031990 con 0x7f84c8143e50 2023-11-18T20:42:30.783 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.780+0000 7f84bffff640 1 -- 172.21.15.60:0/1003198342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4031d20 con 0x7f84c8143e50 2023-11-18T20:42:30.783 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.781+0000 7f84cf4da640 1 -- 172.21.15.60:0/1003198342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8143e50 msgr2=0x7f84c815ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:30.783 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.781+0000 7f84cf4da640 1 --2- 172.21.15.60:0/1003198342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8143e50 0x7f84c815ee70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f84c40097a0 tx=0x7f84c402b8d0 comp rx=0 tx=0).stop 2023-11-18T20:42:30.783 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.781+0000 7f84cf4da640 1 -- 172.21.15.60:0/1003198342 shutdown_connections 2023-11-18T20:42:30.783 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.781+0000 7f84cf4da640 1 --2- 172.21.15.60:0/1003198342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 0x7f84c80b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:30.783 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.781+0000 7f84cf4da640 1 --2- 172.21.15.60:0/1003198342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8143e50 0x7f84c815ee70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:30.784 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.781+0000 7f84cf4da640 1 -- 172.21.15.60:0/1003198342 >> 172.21.15.60:0/1003198342 conn(0x7f84c80b0c60 msgr2=0x7f84c80b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:30.784 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.781+0000 7f84cf4da640 1 -- 172.21.15.60:0/1003198342 shutdown_connections 2023-11-18T20:42:30.784 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.781+0000 7f84cf4da640 1 -- 172.21.15.60:0/1003198342 wait complete. 2023-11-18T20:42:30.786 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.782+0000 7f84cf4da640 1 Processor -- start 2023-11-18T20:42:30.786 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.782+0000 7f84cf4da640 1 -- start start 2023-11-18T20:42:30.786 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cf4da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 0x7f84c8070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:30.786 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cf4da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80b31e0 0x7f84c8158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:30.786 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cf4da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c8143e50 0x7f84c8159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:30.786 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cf4da640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c813c490 con 0x7f84c80b31e0 2023-11-18T20:42:30.786 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cf4da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80709b0 con 0x7f84c8143e50 2023-11-18T20:42:30.786 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cf4da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c815ac40 con 0x7f84c80b1f60 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cca75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80b31e0 0x7f84c8158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cca75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80b31e0 0x7f84c8158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59762/0 (socket says 172.21.15.60:59762) 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cca75640 1 -- 172.21.15.60:0/355660172 learned_addr learned my addr 172.21.15.60:0/355660172 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cd276640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 0x7f84c8070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.783+0000 7f84cda77640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c8143e50 0x7f84c8159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84cca75640 1 -- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 msgr2=0x7f84c8070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84cca75640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 0x7f84c8070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84cca75640 1 -- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c8143e50 msgr2=0x7f84c8159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84cca75640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c8143e50 0x7f84c8159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84cca75640 1 -- 172.21.15.60:0/355660172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4009400 con 0x7f84c80b31e0 2023-11-18T20:42:30.787 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84cd276640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 0x7f84c8070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:30.788 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84cca75640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80b31e0 0x7f84c8158bb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f84b800e9b0 tx=0x7f84b800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:30.788 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84bdffb640 1 -- 172.21.15.60:0/355660172 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8009670 con 0x7f84c80b31e0 2023-11-18T20:42:30.788 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84bdffb640 1 -- 172.21.15.60:0/355660172 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b800c9a0 con 0x7f84c80b31e0 2023-11-18T20:42:30.788 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84cf4da640 1 -- 172.21.15.60:0/355660172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c815a680 con 0x7f84c80b31e0 2023-11-18T20:42:30.788 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84bdffb640 1 -- 172.21.15.60:0/355660172 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b800cd50 con 0x7f84c80b31e0 2023-11-18T20:42:30.788 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.784+0000 7f84cf4da640 1 -- 172.21.15.60:0/355660172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c813c0b0 con 0x7f84c80b31e0 2023-11-18T20:42:30.789 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.785+0000 7f84cf4da640 1 -- 172.21.15.60:0/355660172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c80b3bb0 con 0x7f84c80b31e0 2023-11-18T20:42:30.789 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.786+0000 7f84bdffb640 1 -- 172.21.15.60:0/355660172 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84b8013470 con 0x7f84c80b31e0 2023-11-18T20:42:30.789 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.786+0000 7f84bdffb640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84ac06ae40 0x7f84ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:30.789 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.786+0000 7f84bdffb640 1 -- 172.21.15.60:0/355660172 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f84b80894e0 con 0x7f84c80b31e0 2023-11-18T20:42:30.789 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.786+0000 7f84cd276640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84ac06ae40 0x7f84ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:30.789 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.787+0000 7f84cd276640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84ac06ae40 0x7f84ac06d300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f84c4036000 tx=0x7f84c403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:30.791 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.791+0000 7f84bdffb640 1 -- 172.21.15.60:0/355660172 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84b8054970 con 0x7f84c80b31e0 2023-11-18T20:42:30.971 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.970+0000 7f84cf4da640 1 -- 172.21.15.60:0/355660172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84c8069320 con 0x7f84c80b31e0 2023-11-18T20:42:30.972 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.971+0000 7f84bdffb640 1 -- 172.21.15.60:0/355660172 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7f84b8057f90 con 0x7f84c80b31e0 2023-11-18T20:42:30.972 INFO:teuthology.misc.health.smithi060.stdout: 2023-11-18T20:42:30.972 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":17,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:28.457288+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:30.974 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 -- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84ac06ae40 msgr2=0x7f84ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:30.974 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84ac06ae40 0x7f84ac06d300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f84c4036000 tx=0x7f84c403c040 comp rx=0 tx=0).stop 2023-11-18T20:42:30.975 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 -- 172.21.15.60:0/355660172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80b31e0 msgr2=0x7f84c8158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:30.975 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80b31e0 0x7f84c8158bb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f84b800e9b0 tx=0x7f84b800ee70 comp rx=0 tx=0).stop 2023-11-18T20:42:30.975 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 -- 172.21.15.60:0/355660172 shutdown_connections 2023-11-18T20:42:30.975 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84ac06ae40 0x7f84ac06d300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:30.975 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84c8143e50 0x7f84c8159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:30.975 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c80b31e0 0x7f84c8158bb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:30.975 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 --2- 172.21.15.60:0/355660172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80b1f60 0x7f84c8070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:30.975 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 -- 172.21.15.60:0/355660172 >> 172.21.15.60:0/355660172 conn(0x7f84c80b0c60 msgr2=0x7f84c80b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:30.975 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 -- 172.21.15.60:0/355660172 shutdown_connections 2023-11-18T20:42:30.975 INFO:teuthology.misc.health.smithi060.stderr:2023-11-18T20:42:30.973+0000 7f84cf4da640 1 -- 172.21.15.60:0/355660172 wait complete. 2023-11-18T20:42:30.984 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-18T20:42:30.984 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-18T20:42:30.984 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-18T20:42:30.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-18T20:42:30.986 INFO:tasks.ceph.mds.a:Started 2023-11-18T20:42:30.986 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-18T20:42:30.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-18T20:42:30.988 INFO:tasks.ceph.mds.c:Started 2023-11-18T20:42:30.989 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-18T20:42:30.989 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-18T20:42:30.991 INFO:tasks.ceph.mds.b:Started 2023-11-18T20:42:30.991 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-18T20:42:30.992 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-18T20:42:30.994 INFO:tasks.ceph.mds.d:Started 2023-11-18T20:42:30.994 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-18T20:42:30.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:31.066 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-11-18T20:42:31.070 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-11-18T20:42:31.076 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2023-11-18T20:42:31.076 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2023-11-18T20:42:31.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.104+0000 7f5066347640 1 Processor -- start 2023-11-18T20:42:31.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.104+0000 7f5066347640 1 -- start start 2023-11-18T20:42:31.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.105+0000 7f5066347640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f506015d330 0x7f50600b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.105+0000 7f5066347640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50600b2b10 con 0x7f50600d2910 2023-11-18T20:42:31.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.105+0000 7f5066347640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50600d1590 con 0x7f50600d3900 2023-11-18T20:42:31.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.105+0000 7f5066347640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50600d1870 con 0x7f506015d330 2023-11-18T20:42:31.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.105+0000 7f505ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f506015d330 0x7f50600b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.105+0000 7f505f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f50600d2910 0x7f50600d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55154/0 (socket says 172.21.15.60:55154) 2023-11-18T20:42:31.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.105+0000 7f505f7fe640 1 -- 172.21.15.60:0/3664701321 learned_addr learned my addr 172.21.15.60:0/3664701321 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:31.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.106+0000 7f505ffff640 1 -- 172.21.15.60:0/3664701321 >> v1:172.21.15.100:6789/0 conn(0x7f50600d3900 legacy=0x7f50600b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.106+0000 7f505ffff640 1 -- 172.21.15.60:0/3664701321 >> v1:172.21.15.60:6789/0 conn(0x7f50600d2910 legacy=0x7f50600d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.106+0000 7f505effd640 1 -- 172.21.15.60:0/3664701321 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109879892 0 0) 0x7f50600d1590 con 0x7f50600d3900 2023-11-18T20:42:31.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.106+0000 7f505ffff640 1 -- 172.21.15.60:0/3664701321 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600d1b50 con 0x7f506015d330 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.106+0000 7f505ffff640 1 --2- 172.21.15.60:0/3664701321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506015d330 0x7f50600b8b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50540097a0 tx=0x7f505402b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36aadd55bca7b83c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.107+0000 7f505effd640 1 -- 172.21.15.60:0/3664701321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054030710 con 0x7f506015d330 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.107+0000 7f505effd640 1 -- 172.21.15.60:0/3664701321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505402e050 con 0x7f506015d330 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.107+0000 7f505effd640 1 -- 172.21.15.60:0/3664701321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054030a90 con 0x7f506015d330 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.107+0000 7f5066347640 1 -- 172.21.15.60:0/3664701321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506015d330 msgr2=0x7f50600b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.107+0000 7f5066347640 1 --2- 172.21.15.60:0/3664701321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506015d330 0x7f50600b8b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50540097a0 tx=0x7f505402b710 comp rx=0 tx=0).stop 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.107+0000 7f5066347640 1 -- 172.21.15.60:0/3664701321 shutdown_connections 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.107+0000 7f5066347640 1 --2- 172.21.15.60:0/3664701321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f506015d330 0x7f50600b8b80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.107+0000 7f5066347640 1 -- 172.21.15.60:0/3664701321 >> 172.21.15.60:0/3664701321 conn(0x7f5060072020 msgr2=0x7f5060072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.107+0000 7f5066347640 1 -- 172.21.15.60:0/3664701321 shutdown_connections 2023-11-18T20:42:31.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.107+0000 7f5066347640 1 -- 172.21.15.60:0/3664701321 wait complete. 2023-11-18T20:42:31.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.108+0000 7f5066347640 1 Processor -- start 2023-11-18T20:42:31.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.108+0000 7f5066347640 1 -- start start 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f5066347640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600d2910 0x7f50600c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f5066347640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600d3900 0x7f50600b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f5066347640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015d330 0x7f50600c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f5066347640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50600c6ba0 con 0x7f50600d2910 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f5066347640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50600c6e80 con 0x7f506015d330 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f5066347640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50600c7160 con 0x7f50600d3900 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f505ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600d2910 0x7f50600c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f505ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600d2910 0x7f50600c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59808/0 (socket says 172.21.15.60:59808) 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f505ffff640 1 -- 172.21.15.60:0/768640910 learned_addr learned my addr 172.21.15.60:0/768640910 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f50648e4640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015d330 0x7f50600c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f505f7fe640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600d3900 0x7f50600b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f505ffff640 1 -- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600d3900 msgr2=0x7f50600b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f505ffff640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600d3900 0x7f50600b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f505ffff640 1 -- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015d330 msgr2=0x7f50600c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f505ffff640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015d330 0x7f50600c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.109+0000 7f505ffff640 1 -- 172.21.15.60:0/768640910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5054009400 con 0x7f50600d2910 2023-11-18T20:42:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.110+0000 7f505ffff640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600d2910 0x7f50600c7c40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50540097a0 tx=0x7f50540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.110+0000 7f505cff9640 1 -- 172.21.15.60:0/768640910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054032910 con 0x7f50600d2910 2023-11-18T20:42:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.110+0000 7f505cff9640 1 -- 172.21.15.60:0/768640910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5054032a70 con 0x7f50600d2910 2023-11-18T20:42:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.110+0000 7f5066347640 1 -- 172.21.15.60:0/768640910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50600c7440 con 0x7f50600d2910 2023-11-18T20:42:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.110+0000 7f505cff9640 1 -- 172.21.15.60:0/768640910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054032e20 con 0x7f50600d2910 2023-11-18T20:42:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.110+0000 7f5066347640 1 -- 172.21.15.60:0/768640910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50600c8180 con 0x7f50600d2910 2023-11-18T20:42:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.111+0000 7f5066347640 1 -- 172.21.15.60:0/768640910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5060069400 con 0x7f50600d2910 2023-11-18T20:42:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.111+0000 7f505cff9640 1 -- 172.21.15.60:0/768640910 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5054039040 con 0x7f50600d2910 2023-11-18T20:42:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.112+0000 7f505cff9640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f504c06ae40 0x7f504c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.112+0000 7f505cff9640 1 -- 172.21.15.60:0/768640910 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f50540ab140 con 0x7f50600d2910 2023-11-18T20:42:31.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.112+0000 7f505f7fe640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f504c06ae40 0x7f504c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.113+0000 7f505f7fe640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f504c06ae40 0x7f504c06d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50601582d0 tx=0x7f5050007c60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:31.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.116+0000 7f505cff9640 1 -- 172.21.15.60:0/768640910 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50540764c0 con 0x7f50600d2910 2023-11-18T20:42:31.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.295+0000 7f5066347640 1 -- 172.21.15.60:0/768640910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50600d07d0 con 0x7f50600d2910 2023-11-18T20:42:31.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.296+0000 7f505cff9640 1 -- 172.21.15.60:0/768640910 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7f5054079ae0 con 0x7f50600d2910 2023-11-18T20:42:31.297 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:31.297 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:28.457288+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 -- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f504c06ae40 msgr2=0x7f504c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f504c06ae40 0x7f504c06d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50601582d0 tx=0x7f5050007c60 comp rx=0 tx=0).stop 2023-11-18T20:42:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 -- 172.21.15.60:0/768640910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600d2910 msgr2=0x7f50600c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600d2910 0x7f50600c7c40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50540097a0 tx=0x7f50540046c0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 -- 172.21.15.60:0/768640910 shutdown_connections 2023-11-18T20:42:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f504c06ae40 0x7f504c06d300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015d330 0x7f50600c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600d3900 0x7f50600b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 --2- 172.21.15.60:0/768640910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600d2910 0x7f50600c7c40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 -- 172.21.15.60:0/768640910 >> 172.21.15.60:0/768640910 conn(0x7f5060072020 msgr2=0x7f50600d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:31.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 -- 172.21.15.60:0/768640910 shutdown_connections 2023-11-18T20:42:31.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.298+0000 7f5066347640 1 -- 172.21.15.60:0/768640910 wait complete. 2023-11-18T20:42:31.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:42:31.311 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:42:31.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.418+0000 7f76cefb0640 1 Processor -- start 2023-11-18T20:42:31.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.419+0000 7f76cefb0640 1 -- start start 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.419+0000 7f76cefb0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76c8160fb0 0x7f76c8161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.419+0000 7f76cefb0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80d0ed0 0x7f76c80d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.419+0000 7f76cefb0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c8161d20 con 0x7f76c806ea50 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.419+0000 7f76cefb0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c815e760 con 0x7f76c8160fb0 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.419+0000 7f76cefb0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c815d9a0 con 0x7f76c80d0ed0 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.419+0000 7f76c7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76c8160fb0 0x7f76c8161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.420+0000 7f76c7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f76c8160fb0 0x7f76c8161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59810/0 (socket says 172.21.15.60:59810) 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.420+0000 7f76c7fff640 1 -- 172.21.15.60:0/2031655721 learned_addr learned my addr 172.21.15.60:0/2031655721 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.420+0000 7f76c7fff640 1 -- 172.21.15.60:0/2031655721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80d0ed0 msgr2=0x7f76c80d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.420+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2031655721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80d0ed0 0x7f76c80d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.420+0000 7f76c7fff640 1 -- 172.21.15.60:0/2031655721 >> v1:172.21.15.100:6789/0 conn(0x7f76c806ea50 legacy=0x7f76c806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.420+0000 7f76c7fff640 1 -- 172.21.15.60:0/2031655721 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c815dc80 con 0x7f76c8160fb0 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.420+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2031655721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8160fb0 0x7f76c8161440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009870 tx=0x7f76b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1dc10684617514 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76c77fe640 1 -- 172.21.15.60:0/2031655721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8031640 con 0x7f76c8160fb0 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76c77fe640 1 -- 172.21.15.60:0/2031655721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b80317a0 con 0x7f76c8160fb0 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76c77fe640 1 -- 172.21.15.60:0/2031655721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8031b30 con 0x7f76c8160fb0 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76cefb0640 1 -- 172.21.15.60:0/2031655721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8160fb0 msgr2=0x7f76c8161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76cefb0640 1 --2- 172.21.15.60:0/2031655721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8160fb0 0x7f76c8161440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009870 tx=0x7f76b802b8f0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76cefb0640 1 -- 172.21.15.60:0/2031655721 shutdown_connections 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76cefb0640 1 --2- 172.21.15.60:0/2031655721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8160fb0 0x7f76c8161440 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76cefb0640 1 --2- 172.21.15.60:0/2031655721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80d0ed0 0x7f76c80d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76cefb0640 1 -- 172.21.15.60:0/2031655721 >> 172.21.15.60:0/2031655721 conn(0x7f76c806a6b0 msgr2=0x7f76c806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76cefb0640 1 -- 172.21.15.60:0/2031655721 shutdown_connections 2023-11-18T20:42:31.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.421+0000 7f76cefb0640 1 -- 172.21.15.60:0/2031655721 wait complete. 2023-11-18T20:42:31.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.422+0000 7f76cefb0640 1 Processor -- start 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.422+0000 7f76cefb0640 1 -- start start 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.423+0000 7f76cefb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c806ea50 0x7f76c8159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.423+0000 7f76cefb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80d0ed0 0x7f76c8157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.423+0000 7f76cefb0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c8160fb0 0x7f76c80b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.423+0000 7f76cefb0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c813c380 con 0x7f76c806ea50 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.423+0000 7f76cefb0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c81588d0 con 0x7f76c80d0ed0 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.423+0000 7f76cefb0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c8158bb0 con 0x7f76c8160fb0 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.423+0000 7f76ccd4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c806ea50 0x7f76c8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.423+0000 7f76ccd4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c806ea50 0x7f76c8159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59826/0 (socket says 172.21.15.60:59826) 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.423+0000 7f76ccd4c640 1 -- 172.21.15.60:0/2409381320 learned_addr learned my addr 172.21.15.60:0/2409381320 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80d0ed0 0x7f76c8157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76cd54d640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c8160fb0 0x7f76c80b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76ccd4c640 1 -- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c8160fb0 msgr2=0x7f76c80b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76ccd4c640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c8160fb0 0x7f76c80b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76ccd4c640 1 -- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80d0ed0 msgr2=0x7f76c8157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76ccd4c640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80d0ed0 0x7f76c8157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76ccd4c640 1 -- 172.21.15.60:0/2409381320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b80094d0 con 0x7f76c806ea50 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80d0ed0 0x7f76c8157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76cd54d640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c8160fb0 0x7f76c80b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76ccd4c640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c806ea50 0x7f76c8159970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f76c000ba50 tx=0x7f76c000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76c57fa640 1 -- 172.21.15.60:0/2409381320 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c000cc10 con 0x7f76c806ea50 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76c57fa640 1 -- 172.21.15.60:0/2409381320 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c000cd70 con 0x7f76c806ea50 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76cefb0640 1 -- 172.21.15.60:0/2409381320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c8158e90 con 0x7f76c806ea50 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76c57fa640 1 -- 172.21.15.60:0/2409381320 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0014680 con 0x7f76c806ea50 2023-11-18T20:42:31.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.424+0000 7f76cefb0640 1 -- 172.21.15.60:0/2409381320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c8159450 con 0x7f76c806ea50 2023-11-18T20:42:31.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.425+0000 7f76cefb0640 1 -- 172.21.15.60:0/2409381320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c815b260 con 0x7f76c806ea50 2023-11-18T20:42:31.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.425+0000 7f76c57fa640 1 -- 172.21.15.60:0/2409381320 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76c0007370 con 0x7f76c806ea50 2023-11-18T20:42:31.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.426+0000 7f76c57fa640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76a806ae70 0x7f76a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.426+0000 7f76c57fa640 1 -- 172.21.15.60:0/2409381320 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f76c0088e30 con 0x7f76c806ea50 2023-11-18T20:42:31.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.426+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76a806ae70 0x7f76a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.427+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76a806ae70 0x7f76a806d330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f76b8036000 tx=0x7f76b8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:31.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.430+0000 7f76c57fa640 1 -- 172.21.15.60:0/2409381320 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76c0054230 con 0x7f76c806ea50 2023-11-18T20:42:31.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.605+0000 7f76cefb0640 1 -- 172.21.15.60:0/2409381320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76c8069400 con 0x7f76c806ea50 2023-11-18T20:42:31.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.605+0000 7f76c57fa640 1 -- 172.21.15.60:0/2409381320 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76c0057850 con 0x7f76c806ea50 2023-11-18T20:42:31.606 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:42:31.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 -- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76a806ae70 msgr2=0x7f76a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76a806ae70 0x7f76a806d330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f76b8036000 tx=0x7f76b8008030 comp rx=0 tx=0).stop 2023-11-18T20:42:31.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 -- 172.21.15.60:0/2409381320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c806ea50 msgr2=0x7f76c8159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c806ea50 0x7f76c8159970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f76c000ba50 tx=0x7f76c000bf10 comp rx=0 tx=0).stop 2023-11-18T20:42:31.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 -- 172.21.15.60:0/2409381320 shutdown_connections 2023-11-18T20:42:31.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76a806ae70 0x7f76a806d330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c8160fb0 0x7f76c80b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80d0ed0 0x7f76c8157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 --2- 172.21.15.60:0/2409381320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c806ea50 0x7f76c8159970 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 -- 172.21.15.60:0/2409381320 >> 172.21.15.60:0/2409381320 conn(0x7f76c806a6b0 msgr2=0x7f76c80b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:31.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 -- 172.21.15.60:0/2409381320 shutdown_connections 2023-11-18T20:42:31.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.607+0000 7f76cefb0640 1 -- 172.21.15.60:0/2409381320 wait complete. 2023-11-18T20:42:31.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:42:31.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.724+0000 7f58d9b2c640 1 Processor -- start 2023-11-18T20:42:31.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.724+0000 7f58d9b2c640 1 -- start start 2023-11-18T20:42:31.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d9b2c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58d4143490 0x7f58d40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d9b2c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f58d415e730 0x7f58d40b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d9b2c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415a4e0 0x7f58d415a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d9b2c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d415b520 con 0x7f58d4143490 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d9b2c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d4143870 con 0x7f58d415e730 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d9b2c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d415e2a0 con 0x7f58d415a4e0 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58d4143490 0x7f58d40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58d4143490 0x7f58d40b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59832/0 (socket says 172.21.15.60:59832) 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d37fe640 1 -- 172.21.15.60:0/776978923 learned_addr learned my addr 172.21.15.60:0/776978923 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d2ffd640 1 --2- 172.21.15.60:0/776978923 >> v2:172.21.15.100:3300/0 conn(0x7f58d415e730 0x7f58d40b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d37fe640 1 -- 172.21.15.60:0/776978923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415a4e0 msgr2=0x7f58d415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d3fff640 1 --2- 172.21.15.60:0/776978923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415a4e0 0x7f58d415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d37fe640 1 --2- 172.21.15.60:0/776978923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415a4e0 0x7f58d415a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d37fe640 1 -- 172.21.15.60:0/776978923 >> v2:172.21.15.100:3300/0 conn(0x7f58d415e730 msgr2=0x7f58d40b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:31.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d37fe640 1 --2- 172.21.15.60:0/776978923 >> v2:172.21.15.100:3300/0 conn(0x7f58d415e730 0x7f58d40b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d37fe640 1 -- 172.21.15.60:0/776978923 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d415b240 con 0x7f58d4143490 2023-11-18T20:42:31.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.725+0000 7f58d2ffd640 1 --2- 172.21.15.60:0/776978923 >> v2:172.21.15.100:3300/0 conn(0x7f58d415e730 0x7f58d40b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:31.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.726+0000 7f58d37fe640 1 --2- 172.21.15.60:0/776978923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d4143490 0x7f58d40b0cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58d40aeca0 tx=0x7f58c402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6099e944af14077 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:31.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.726+0000 7f58d27fc640 1 -- 172.21.15.60:0/776978923 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c4031860 con 0x7f58d4143490 2023-11-18T20:42:31.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.726+0000 7f58d27fc640 1 -- 172.21.15.60:0/776978923 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c40319c0 con 0x7f58d4143490 2023-11-18T20:42:31.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.726+0000 7f58d27fc640 1 -- 172.21.15.60:0/776978923 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c4031d50 con 0x7f58d4143490 2023-11-18T20:42:31.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.727+0000 7f58d9b2c640 1 -- 172.21.15.60:0/776978923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d4143490 msgr2=0x7f58d40b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.727+0000 7f58d9b2c640 1 --2- 172.21.15.60:0/776978923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d4143490 0x7f58d40b0cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58d40aeca0 tx=0x7f58c402b900 comp rx=0 tx=0).stop 2023-11-18T20:42:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.727+0000 7f58d9b2c640 1 -- 172.21.15.60:0/776978923 shutdown_connections 2023-11-18T20:42:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.727+0000 7f58d9b2c640 1 --2- 172.21.15.60:0/776978923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415a4e0 0x7f58d415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.727+0000 7f58d9b2c640 1 --2- 172.21.15.60:0/776978923 >> v2:172.21.15.100:3300/0 conn(0x7f58d415e730 0x7f58d40b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.727+0000 7f58d9b2c640 1 --2- 172.21.15.60:0/776978923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d4143490 0x7f58d40b0cf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.727+0000 7f58d9b2c640 1 -- 172.21.15.60:0/776978923 >> 172.21.15.60:0/776978923 conn(0x7f58d406a4d0 msgr2=0x7f58d406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.727+0000 7f58d9b2c640 1 -- 172.21.15.60:0/776978923 shutdown_connections 2023-11-18T20:42:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.727+0000 7f58d9b2c640 1 -- 172.21.15.60:0/776978923 wait complete. 2023-11-18T20:42:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.728+0000 7f58d9b2c640 1 Processor -- start 2023-11-18T20:42:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.728+0000 7f58d9b2c640 1 -- start start 2023-11-18T20:42:31.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d9b2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d4143490 0x7f58d413e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d9b2c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415a4e0 0x7f58d40b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d9b2c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415e730 0x7f58d4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d9b2c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d413eed0 con 0x7f58d415a4e0 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d9b2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d41f0e20 con 0x7f58d4143490 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d9b2c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d41f1100 con 0x7f58d415e730 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415a4e0 0x7f58d40b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415a4e0 0x7f58d40b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59844/0 (socket says 172.21.15.60:59844) 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d2ffd640 1 -- 172.21.15.60:0/4168167566 learned_addr learned my addr 172.21.15.60:0/4168167566 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d37fe640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d4143490 0x7f58d413e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d3fff640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415e730 0x7f58d4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d2ffd640 1 -- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415e730 msgr2=0x7f58d4070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d2ffd640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415e730 0x7f58d4070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d2ffd640 1 -- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d4143490 msgr2=0x7f58d413e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d2ffd640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d4143490 0x7f58d413e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d2ffd640 1 -- 172.21.15.60:0/4168167566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c40094d0 con 0x7f58d415a4e0 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.729+0000 7f58d2ffd640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415a4e0 0x7f58d40b0e50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c7f0 tx=0x7f58c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.730+0000 7f58bbfff640 1 -- 172.21.15.60:0/4168167566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8007820 con 0x7f58d415a4e0 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.730+0000 7f58bbfff640 1 -- 172.21.15.60:0/4168167566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8007980 con 0x7f58d415a4e0 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.730+0000 7f58d9b2c640 1 -- 172.21.15.60:0/4168167566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d41f13e0 con 0x7f58d415a4e0 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.730+0000 7f58bbfff640 1 -- 172.21.15.60:0/4168167566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8007cf0 con 0x7f58d415a4e0 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.730+0000 7f58d9b2c640 1 -- 172.21.15.60:0/4168167566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d40b1390 con 0x7f58d415a4e0 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.731+0000 7f58d9b2c640 1 -- 172.21.15.60:0/4168167566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d41f1900 con 0x7f58d415a4e0 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.731+0000 7f58bbfff640 1 -- 172.21.15.60:0/4168167566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58c80113f0 con 0x7f58d415a4e0 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.731+0000 7f58bbfff640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c006ae40 0x7f58c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.731+0000 7f58bbfff640 1 -- 172.21.15.60:0/4168167566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f58c8089480 con 0x7f58d415a4e0 2023-11-18T20:42:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.732+0000 7f58d37fe640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c006ae40 0x7f58c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:31.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.732+0000 7f58d37fe640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c006ae40 0x7f58c006d300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f58c4036000 tx=0x7f58c403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:31.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.736+0000 7f58bbfff640 1 -- 172.21.15.60:0/4168167566 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58d41f1900 con 0x7f58d415a4e0 2023-11-18T20:42:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.951+0000 7f58d9b2c640 1 -- 172.21.15.60:0/4168167566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58d41f1900 con 0x7f58d415a4e0 2023-11-18T20:42:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.952+0000 7f58bbfff640 1 -- 172.21.15.60:0/4168167566 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f58c801f540 con 0x7f58d415a4e0 2023-11-18T20:42:31.954 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:31.954 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4248907695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4248907695},{"type":"v1","addr":"172.21.15.100:6836","nonce":4248907695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 2 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.953+0000 7f58d9b2c640 1 -- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c006ae40 msgr2=0x7f58c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.953+0000 7f58d9b2c640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c006ae40 0x7f58c006d300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f58c4036000 tx=0x7f58c403c040 comp rx=0 tx=0).stop 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.953+0000 7f58d9b2c640 1 -- 172.21.15.60:0/4168167566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415a4e0 msgr2=0x7f58d40b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.953+0000 7f58d9b2c640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415a4e0 0x7f58d40b0e50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c7f0 tx=0x7f58c800ccb0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.953+0000 7f58d9b2c640 1 -- 172.21.15.60:0/4168167566 shutdown_connections 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.953+0000 7f58d9b2c640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c006ae40 0x7f58c006d300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.953+0000 7f58d9b2c640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415e730 0x7f58d4070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.953+0000 7f58d9b2c640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415a4e0 0x7f58d40b0e50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.953+0000 7f58d9b2c640 1 --2- 172.21.15.60:0/4168167566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d4143490 0x7f58d413e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.953+0000 7f58d9b2c640 1 -- 172.21.15.60:0/4168167566 >> 172.21.15.60:0/4168167566 conn(0x7f58d406a4d0 msgr2=0x7f58d40cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.954+0000 7f58d9b2c640 1 -- 172.21.15.60:0/4168167566 shutdown_connections 2023-11-18T20:42:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:31.954+0000 7f58d9b2c640 1 -- 172.21.15.60:0/4168167566 wait complete. 2023-11-18T20:42:33.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:42:34.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.083+0000 7f126cb6c640 1 Processor -- start 2023-11-18T20:42:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.083+0000 7f126cb6c640 1 -- start start 2023-11-18T20:42:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f126cb6c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f126815d2d0 0x7f12680b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f126cb6c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 0x7f12680b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f126cb6c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f12680cda40 0x7f12680cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f126cb6c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12680d1bb0 con 0x7f12680cda40 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f126cb6c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126815d6b0 con 0x7f126815d2d0 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f126cb6c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12680ce700 con 0x7f12680d2040 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f1266d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f12680cda40 0x7f12680cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f1266d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f12680cda40 0x7f12680cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59856/0 (socket says 172.21.15.60:59856) 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f1266d9d640 1 -- 172.21.15.60:0/616428656 learned_addr learned my addr 172.21.15.60:0/616428656 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f125ffff640 1 --2- 172.21.15.60:0/616428656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 0x7f12680b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.084+0000 7f126659c640 1 --2- 172.21.15.60:0/616428656 >> v2:172.21.15.100:3300/0 conn(0x7f126815d2d0 0x7f12680b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.085+0000 7f1266d9d640 1 -- 172.21.15.60:0/616428656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 msgr2=0x7f12680b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.085+0000 7f1266d9d640 1 --2- 172.21.15.60:0/616428656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 0x7f12680b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.085+0000 7f1266d9d640 1 -- 172.21.15.60:0/616428656 >> v2:172.21.15.100:3300/0 conn(0x7f126815d2d0 msgr2=0x7f12680b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.085+0000 7f1266d9d640 1 --2- 172.21.15.60:0/616428656 >> v2:172.21.15.100:3300/0 conn(0x7f126815d2d0 0x7f12680b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.085+0000 7f1266d9d640 1 -- 172.21.15.60:0/616428656 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680d1280 con 0x7f12680cda40 2023-11-18T20:42:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.085+0000 7f126659c640 1 --2- 172.21.15.60:0/616428656 >> v2:172.21.15.100:3300/0 conn(0x7f126815d2d0 0x7f12680b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.085+0000 7f125ffff640 1 --2- 172.21.15.60:0/616428656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 0x7f12680b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.085+0000 7f1266d9d640 1 --2- 172.21.15.60:0/616428656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12680cda40 0x7f12680cded0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f126806c5d0 tx=0x7f126002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939797e1bc8cf044 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.085+0000 7f1265d9b640 1 -- 172.21.15.60:0/616428656 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126002ca80 con 0x7f12680cda40 2023-11-18T20:42:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f1265d9b640 1 -- 172.21.15.60:0/616428656 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126002cbe0 con 0x7f12680cda40 2023-11-18T20:42:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f1265d9b640 1 -- 172.21.15.60:0/616428656 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12600343f0 con 0x7f12680cda40 2023-11-18T20:42:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f126cb6c640 1 -- 172.21.15.60:0/616428656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12680cda40 msgr2=0x7f12680cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f126cb6c640 1 --2- 172.21.15.60:0/616428656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12680cda40 0x7f12680cded0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f126806c5d0 tx=0x7f126002bee0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f126cb6c640 1 -- 172.21.15.60:0/616428656 shutdown_connections 2023-11-18T20:42:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f126cb6c640 1 --2- 172.21.15.60:0/616428656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12680cda40 0x7f12680cded0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f126cb6c640 1 --2- 172.21.15.60:0/616428656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 0x7f12680b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f126cb6c640 1 --2- 172.21.15.60:0/616428656 >> v2:172.21.15.100:3300/0 conn(0x7f126815d2d0 0x7f12680b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f126cb6c640 1 -- 172.21.15.60:0/616428656 >> 172.21.15.60:0/616428656 conn(0x7f1268072050 msgr2=0x7f1268072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f126cb6c640 1 -- 172.21.15.60:0/616428656 shutdown_connections 2023-11-18T20:42:34.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.086+0000 7f126cb6c640 1 -- 172.21.15.60:0/616428656 wait complete. 2023-11-18T20:42:34.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.088+0000 7f126cb6c640 1 Processor -- start 2023-11-18T20:42:34.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.088+0000 7f126cb6c640 1 -- start start 2023-11-18T20:42:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.088+0000 7f126cb6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12680cda40 0x7f12680c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.088+0000 7f126cb6c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 0x7f1268156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.088+0000 7f126cb6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f126815d2d0 0x7f12681f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.088+0000 7f126cb6c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12680c8380 con 0x7f126815d2d0 2023-11-18T20:42:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.088+0000 7f126cb6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12680b62d0 con 0x7f12680cda40 2023-11-18T20:42:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.088+0000 7f126cb6c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12681f3290 con 0x7f12680d2040 2023-11-18T20:42:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.088+0000 7f1266d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f126815d2d0 0x7f12681f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f1266d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f126815d2d0 0x7f12681f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59862/0 (socket says 172.21.15.60:59862) 2023-11-18T20:42:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f1266d9d640 1 -- 172.21.15.60:0/4161595790 learned_addr learned my addr 172.21.15.60:0/4161595790 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f126659c640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12680cda40 0x7f12680c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f125ffff640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 0x7f1268156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f1266d9d640 1 -- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 msgr2=0x7f1268156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f1266d9d640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 0x7f1268156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f1266d9d640 1 -- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12680cda40 msgr2=0x7f12680c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f1266d9d640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12680cda40 0x7f12680c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f1266d9d640 1 -- 172.21.15.60:0/4161595790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1260009b30 con 0x7f126815d2d0 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f1266d9d640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f126815d2d0 0x7f12681f29c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1260009f40 tx=0x7f1260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f125effd640 1 -- 172.21.15.60:0/4161595790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126003e450 con 0x7f126815d2d0 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f125effd640 1 -- 172.21.15.60:0/4161595790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126003e5d0 con 0x7f126815d2d0 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f125effd640 1 -- 172.21.15.60:0/4161595790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126003e980 con 0x7f126815d2d0 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f126cb6c640 1 -- 172.21.15.60:0/4161595790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12681f3570 con 0x7f126815d2d0 2023-11-18T20:42:34.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.089+0000 7f126cb6c640 1 -- 172.21.15.60:0/4161595790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1268156d80 con 0x7f126815d2d0 2023-11-18T20:42:34.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.090+0000 7f126cb6c640 1 -- 172.21.15.60:0/4161595790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1268069400 con 0x7f126815d2d0 2023-11-18T20:42:34.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.091+0000 7f125effd640 1 -- 172.21.15.60:0/4161595790 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f126002cd50 con 0x7f126815d2d0 2023-11-18T20:42:34.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.091+0000 7f125effd640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f124806ae40 0x7f124806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.091+0000 7f125effd640 1 -- 172.21.15.60:0/4161595790 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f12600abd50 con 0x7f126815d2d0 2023-11-18T20:42:34.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.092+0000 7f126659c640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f124806ae40 0x7f124806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.092+0000 7f126659c640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f124806ae40 0x7f124806d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f12680693d0 tx=0x7f1258008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:34.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.096+0000 7f125effd640 1 -- 172.21.15.60:0/4161595790 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12600af030 con 0x7f126815d2d0 2023-11-18T20:42:34.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.308+0000 7f126cb6c640 1 -- 172.21.15.60:0/4161595790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f126815cb80 con 0x7f126815d2d0 2023-11-18T20:42:34.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.308+0000 7f125effd640 1 -- 172.21.15.60:0/4161595790 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7f1260043e90 con 0x7f126815d2d0 2023-11-18T20:42:34.309 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:34.310 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1530496056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1530496056},{"type":"v1","addr":"172.21.15.60:6833","nonce":1530496056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3842389676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3842389676},{"type":"v1","addr":"172.21.15.60:6835","nonce":3842389676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4248907695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4248907695},{"type":"v1","addr":"172.21.15.100:6836","nonce":4248907695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4303,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1851816776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1851816776},{"type":"v1","addr":"172.21.15.100:6837","nonce":1851816776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-18T20:42:34.310 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 3 2023-11-18T20:42:34.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 -- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f124806ae40 msgr2=0x7f124806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f124806ae40 0x7f124806d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f12680693d0 tx=0x7f1258008040 comp rx=0 tx=0).stop 2023-11-18T20:42:34.311 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 -- 172.21.15.60:0/4161595790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f126815d2d0 msgr2=0x7f12681f29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f126815d2d0 0x7f12681f29c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1260009f40 tx=0x7f1260004a40 comp rx=0 tx=0).stop 2023-11-18T20:42:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 -- 172.21.15.60:0/4161595790 shutdown_connections 2023-11-18T20:42:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f124806ae40 0x7f124806d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f126815d2d0 0x7f12681f29c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12680d2040 0x7f1268156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 --2- 172.21.15.60:0/4161595790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12680cda40 0x7f12680c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 -- 172.21.15.60:0/4161595790 >> 172.21.15.60:0/4161595790 conn(0x7f1268072050 msgr2=0x7f126815dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 -- 172.21.15.60:0/4161595790 shutdown_connections 2023-11-18T20:42:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.310+0000 7f126cb6c640 1 -- 172.21.15.60:0/4161595790 wait complete. 2023-11-18T20:42:34.322 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.427+0000 7f673f4e0640 1 Processor -- start 2023-11-18T20:42:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.427+0000 7f673f4e0640 1 -- start start 2023-11-18T20:42:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.428+0000 7f673f4e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f673815d1d0 0x7f673806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.428+0000 7f673f4e0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6738161ce0 con 0x7f67380aeb80 2023-11-18T20:42:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.428+0000 7f673f4e0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67380adae0 con 0x7f67380ad640 2023-11-18T20:42:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.428+0000 7f673f4e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67380addc0 con 0x7f673815d1d0 2023-11-18T20:42:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.428+0000 7f673d27c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f673815d1d0 0x7f673806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.428+0000 7f673da7d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f67380aeb80 0x7f67381615d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55162/0 (socket says 172.21.15.60:55162) 2023-11-18T20:42:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.428+0000 7f673da7d640 1 -- 172.21.15.60:0/3471124068 learned_addr learned my addr 172.21.15.60:0/3471124068 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.428+0000 7f673d27c640 1 -- 172.21.15.60:0/3471124068 >> v1:172.21.15.100:6789/0 conn(0x7f67380ad640 legacy=0x7f6738160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.428+0000 7f673d27c640 1 -- 172.21.15.60:0/3471124068 >> v1:172.21.15.60:6789/0 conn(0x7f67380aeb80 legacy=0x7f67381615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.428+0000 7f673d27c640 1 -- 172.21.15.60:0/3471124068 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67380ae0a0 con 0x7f673815d1d0 2023-11-18T20:42:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.429+0000 7f673d27c640 1 --2- 172.21.15.60:0/3471124068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 0x7f673806b4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f67340097a0 tx=0x7f673402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec937f73dcff067 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.429+0000 7f672ffff640 1 -- 172.21.15.60:0/3471124068 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734030610 con 0x7f673815d1d0 2023-11-18T20:42:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.429+0000 7f672ffff640 1 -- 172.21.15.60:0/3471124068 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6734030770 con 0x7f673815d1d0 2023-11-18T20:42:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.429+0000 7f672ffff640 1 -- 172.21.15.60:0/3471124068 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734030b00 con 0x7f673815d1d0 2023-11-18T20:42:34.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.430+0000 7f673f4e0640 1 -- 172.21.15.60:0/3471124068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 msgr2=0x7f673806b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.430+0000 7f673f4e0640 1 --2- 172.21.15.60:0/3471124068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 0x7f673806b4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f67340097a0 tx=0x7f673402b960 comp rx=0 tx=0).stop 2023-11-18T20:42:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.430+0000 7f673f4e0640 1 -- 172.21.15.60:0/3471124068 shutdown_connections 2023-11-18T20:42:34.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.430+0000 7f673f4e0640 1 --2- 172.21.15.60:0/3471124068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 0x7f673806b4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.430+0000 7f673f4e0640 1 -- 172.21.15.60:0/3471124068 >> 172.21.15.60:0/3471124068 conn(0x7f6738070e70 msgr2=0x7f6738071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.430+0000 7f673f4e0640 1 -- 172.21.15.60:0/3471124068 shutdown_connections 2023-11-18T20:42:34.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.430+0000 7f673f4e0640 1 -- 172.21.15.60:0/3471124068 wait complete. 2023-11-18T20:42:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.431+0000 7f673f4e0640 1 Processor -- start 2023-11-18T20:42:34.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.431+0000 7f673f4e0640 1 -- start start 2023-11-18T20:42:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673f4e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67380ad640 0x7f67381514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673f4e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67380aeb80 0x7f67380c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673f4e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 0x7f67380c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673f4e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6738151a10 con 0x7f673815d1d0 2023-11-18T20:42:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673f4e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67380c4bb0 con 0x7f67380aeb80 2023-11-18T20:42:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673f4e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67380c4e90 con 0x7f67380ad640 2023-11-18T20:42:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673da7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 0x7f67380c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673da7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 0x7f67380c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59880/0 (socket says 172.21.15.60:59880) 2023-11-18T20:42:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673da7d640 1 -- 172.21.15.60:0/1054064797 learned_addr learned my addr 172.21.15.60:0/1054064797 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:34.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673d27c640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67380ad640 0x7f67381514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673ca7b640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67380aeb80 0x7f67380c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673da7d640 1 -- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67380ad640 msgr2=0x7f67381514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673da7d640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67380ad640 0x7f67381514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673da7d640 1 -- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67380aeb80 msgr2=0x7f67380c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673da7d640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67380aeb80 0x7f67380c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673da7d640 1 -- 172.21.15.60:0/1054064797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6734009400 con 0x7f673815d1d0 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.432+0000 7f673da7d640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 0x7f67380c6700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f673000cbc0 tx=0x7f673000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.433+0000 7f672dffb640 1 -- 172.21.15.60:0/1054064797 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673000ad40 con 0x7f673815d1d0 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.433+0000 7f672dffb640 1 -- 172.21.15.60:0/1054064797 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6730012400 con 0x7f673815d1d0 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.433+0000 7f673f4e0640 1 -- 172.21.15.60:0/1054064797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67380c5170 con 0x7f673815d1d0 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.433+0000 7f672dffb640 1 -- 172.21.15.60:0/1054064797 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673000aea0 con 0x7f673815d1d0 2023-11-18T20:42:34.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.433+0000 7f673f4e0640 1 -- 172.21.15.60:0/1054064797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67380c5730 con 0x7f673815d1d0 2023-11-18T20:42:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.434+0000 7f673f4e0640 1 -- 172.21.15.60:0/1054064797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67380c5920 con 0x7f673815d1d0 2023-11-18T20:42:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.434+0000 7f672dffb640 1 -- 172.21.15.60:0/1054064797 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6730007070 con 0x7f673815d1d0 2023-11-18T20:42:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.434+0000 7f672dffb640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f671c06ae40 0x7f671c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.434+0000 7f672dffb640 1 -- 172.21.15.60:0/1054064797 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6730089130 con 0x7f673815d1d0 2023-11-18T20:42:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.435+0000 7f673d27c640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f671c06ae40 0x7f671c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.435+0000 7f673d27c640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f671c06ae40 0x7f671c06d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6734036000 tx=0x7f673403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:34.440 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.439+0000 7f672dffb640 1 -- 172.21.15.60:0/1054064797 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67300545d0 con 0x7f673815d1d0 2023-11-18T20:42:34.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.615+0000 7f673f4e0640 1 -- 172.21.15.60:0/1054064797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6738069400 con 0x7f673815d1d0 2023-11-18T20:42:34.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.616+0000 7f672dffb640 1 -- 172.21.15.60:0/1054064797 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7f6730057bf0 con 0x7f673815d1d0 2023-11-18T20:42:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:34.618 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:28.457288+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:34.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 -- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f671c06ae40 msgr2=0x7f671c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f671c06ae40 0x7f671c06d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6734036000 tx=0x7f673403b040 comp rx=0 tx=0).stop 2023-11-18T20:42:34.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 -- 172.21.15.60:0/1054064797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 msgr2=0x7f67380c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 0x7f67380c6700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f673000cbc0 tx=0x7f673000a430 comp rx=0 tx=0).stop 2023-11-18T20:42:34.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 -- 172.21.15.60:0/1054064797 shutdown_connections 2023-11-18T20:42:34.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f671c06ae40 0x7f671c06d300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f673815d1d0 0x7f67380c6700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67380aeb80 0x7f67380c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 --2- 172.21.15.60:0/1054064797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67380ad640 0x7f67381514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 -- 172.21.15.60:0/1054064797 >> 172.21.15.60:0/1054064797 conn(0x7f6738070e70 msgr2=0x7f673806d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:34.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 -- 172.21.15.60:0/1054064797 shutdown_connections 2023-11-18T20:42:34.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.618+0000 7f673f4e0640 1 -- 172.21.15.60:0/1054064797 wait complete. 2023-11-18T20:42:34.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:42:34.630 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:42:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.738+0000 7f3793e46640 1 Processor -- start 2023-11-18T20:42:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.738+0000 7f3793e46640 1 -- start start 2023-11-18T20:42:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f3793e46640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c06ea50 0x7f378c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f3793e46640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f378c0d0ed0 0x7f378c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f3793e46640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c161d20 con 0x7f378c160fb0 2023-11-18T20:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f3793e46640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378c161b20 con 0x7f378c0d0ed0 2023-11-18T20:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f3793e46640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c06f370 con 0x7f378c06ea50 2023-11-18T20:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f3791be2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c06ea50 0x7f378c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f37923e3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f378c0d0ed0 0x7f378c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f3791be2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c06ea50 0x7f378c06ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:33144/0 (socket says 172.21.15.60:33144) 2023-11-18T20:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f37923e3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f378c0d0ed0 0x7f378c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45960/0 (socket says 172.21.15.60:45960) 2023-11-18T20:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f3791be2640 1 -- 172.21.15.60:0/1178884087 learned_addr learned my addr 172.21.15.60:0/1178884087 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.739+0000 7f37913e1640 1 --1- 172.21.15.60:0/1178884087 >> v1:172.21.15.100:6789/0 conn(0x7f378c160fb0 0x7f378c161410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:60130/0 (socket says 172.21.15.60:60130) 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.740+0000 7f3791be2640 1 -- 172.21.15.60:0/1178884087 >> v2:172.21.15.100:3300/0 conn(0x7f378c0d0ed0 msgr2=0x7f378c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.740+0000 7f3791be2640 1 --2- 172.21.15.60:0/1178884087 >> v2:172.21.15.100:3300/0 conn(0x7f378c0d0ed0 0x7f378c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.740+0000 7f3791be2640 1 -- 172.21.15.60:0/1178884087 >> v1:172.21.15.100:6789/0 conn(0x7f378c160fb0 legacy=0x7f378c161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.740+0000 7f3791be2640 1 -- 172.21.15.60:0/1178884087 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c15e760 con 0x7f378c06ea50 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3791be2640 1 --2- 172.21.15.60:0/1178884087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c06ea50 0x7f378c06ee30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3784009870 tx=0x7f378402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699dbe9840229484 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3790be0640 1 -- 172.21.15.60:0/1178884087 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784030840 con 0x7f378c06ea50 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3790be0640 1 -- 172.21.15.60:0/1178884087 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37840309a0 con 0x7f378c06ea50 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3790be0640 1 -- 172.21.15.60:0/1178884087 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784030d30 con 0x7f378c06ea50 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3793e46640 1 -- 172.21.15.60:0/1178884087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c06ea50 msgr2=0x7f378c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3793e46640 1 --2- 172.21.15.60:0/1178884087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c06ea50 0x7f378c06ee30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3784009870 tx=0x7f378402b980 comp rx=0 tx=0).stop 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3793e46640 1 -- 172.21.15.60:0/1178884087 shutdown_connections 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3793e46640 1 --2- 172.21.15.60:0/1178884087 >> v2:172.21.15.100:3300/0 conn(0x7f378c0d0ed0 0x7f378c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3793e46640 1 --2- 172.21.15.60:0/1178884087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c06ea50 0x7f378c06ee30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3793e46640 1 -- 172.21.15.60:0/1178884087 >> 172.21.15.60:0/1178884087 conn(0x7f378c06a6b0 msgr2=0x7f378c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:34.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3793e46640 1 -- 172.21.15.60:0/1178884087 shutdown_connections 2023-11-18T20:42:34.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.741+0000 7f3793e46640 1 -- 172.21.15.60:0/1178884087 wait complete. 2023-11-18T20:42:34.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3793e46640 1 Processor -- start 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3793e46640 1 -- start start 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3793e46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c06ea50 0x7f378c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3793e46640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0d0ed0 0x7f378c159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3793e46640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c160fb0 0x7f378c13f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3793e46640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c14cc90 con 0x7f378c06ea50 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3793e46640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c158eb0 con 0x7f378c0d0ed0 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3793e46640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c15ac40 con 0x7f378c160fb0 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3791be2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c06ea50 0x7f378c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3791be2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c06ea50 0x7f378c15a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59892/0 (socket says 172.21.15.60:59892) 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.743+0000 7f3791be2640 1 -- 172.21.15.60:0/1358556705 learned_addr learned my addr 172.21.15.60:0/1358556705 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f37923e3640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c160fb0 0x7f378c13f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f37913e1640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0d0ed0 0x7f378c159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f3791be2640 1 -- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c160fb0 msgr2=0x7f378c13f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f3791be2640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c160fb0 0x7f378c13f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f3791be2640 1 -- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0d0ed0 msgr2=0x7f378c159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f3791be2640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0d0ed0 0x7f378c159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f3791be2640 1 -- 172.21.15.60:0/1358556705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37840094d0 con 0x7f378c06ea50 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f3791be2640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c06ea50 0x7f378c15a700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3784000c00 tx=0x7f3784004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f378a7fc640 1 -- 172.21.15.60:0/1358556705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784039a90 con 0x7f378c06ea50 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f378a7fc640 1 -- 172.21.15.60:0/1358556705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3784039bf0 con 0x7f378c06ea50 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f3793e46640 1 -- 172.21.15.60:0/1358556705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c1582a0 con 0x7f378c06ea50 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f378a7fc640 1 -- 172.21.15.60:0/1358556705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784043410 con 0x7f378c06ea50 2023-11-18T20:42:34.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.744+0000 7f3793e46640 1 -- 172.21.15.60:0/1358556705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c158860 con 0x7f378c06ea50 2023-11-18T20:42:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.745+0000 7f3793e46640 1 -- 172.21.15.60:0/1358556705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378c0705f0 con 0x7f378c06ea50 2023-11-18T20:42:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.746+0000 7f378a7fc640 1 -- 172.21.15.60:0/1358556705 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f378404c3b0 con 0x7f378c06ea50 2023-11-18T20:42:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.746+0000 7f378a7fc640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f377806ae70 0x7f377806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:34.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.746+0000 7f378a7fc640 1 -- 172.21.15.60:0/1358556705 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f37840ab520 con 0x7f378c06ea50 2023-11-18T20:42:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.746+0000 7f37913e1640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f377806ae70 0x7f377806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:34.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.747+0000 7f37913e1640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f377806ae70 0x7f377806d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f378c156e50 tx=0x7f3780000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.750+0000 7f378a7fc640 1 -- 172.21.15.60:0/1358556705 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3784076a60 con 0x7f378c06ea50 2023-11-18T20:42:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.923+0000 7f3793e46640 1 -- 172.21.15.60:0/1358556705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f378c069400 con 0x7f378c06ea50 2023-11-18T20:42:34.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.923+0000 7f378a7fc640 1 -- 172.21.15.60:0/1358556705 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f378407a080 con 0x7f378c06ea50 2023-11-18T20:42:34.925 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 -- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f377806ae70 msgr2=0x7f377806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f377806ae70 0x7f377806d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f378c156e50 tx=0x7f3780000ed0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 -- 172.21.15.60:0/1358556705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c06ea50 msgr2=0x7f378c15a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c06ea50 0x7f378c15a700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3784000c00 tx=0x7f3784004730 comp rx=0 tx=0).stop 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 -- 172.21.15.60:0/1358556705 shutdown_connections 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f377806ae70 0x7f377806d330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c160fb0 0x7f378c13f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0d0ed0 0x7f378c159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 --2- 172.21.15.60:0/1358556705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c06ea50 0x7f378c15a700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 -- 172.21.15.60:0/1358556705 >> 172.21.15.60:0/1358556705 conn(0x7f378c06a6b0 msgr2=0x7f378c161410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 -- 172.21.15.60:0/1358556705 shutdown_connections 2023-11-18T20:42:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:34.925+0000 7f3793e46640 1 -- 172.21.15.60:0/1358556705 wait complete. 2023-11-18T20:42:34.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-18T20:42:35.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.042+0000 7fbf59533640 1 Processor -- start 2023-11-18T20:42:35.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.042+0000 7fbf59533640 1 -- start start 2023-11-18T20:42:35.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf59533640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbf5406ea50 0x7fbf5406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf59533640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf54161d20 con 0x7fbf540d0ed0 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf59533640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5406f370 con 0x7fbf54160fb0 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf59533640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5415e760 con 0x7fbf5406ea50 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf537fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbf540d0ed0 0x7fbf540d1320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55170/0 (socket says 172.21.15.60:55170) 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf537fe640 1 -- 172.21.15.60:0/1512687705 learned_addr learned my addr 172.21.15.60:0/1512687705 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf52ffd640 1 --2- 172.21.15.60:0/1512687705 >> v2:172.21.15.100:3300/0 conn(0x7fbf5406ea50 0x7fbf5406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639219326 0 0) 0x7fbf54161d20 con 0x7fbf540d0ed0 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf3c003520 con 0x7fbf540d0ed0 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3290037093 0 0) 0x7fbf3c003520 con 0x7fbf540d0ed0 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 >> v2:172.21.15.100:3300/0 conn(0x7fbf5406ea50 msgr2=0x7fbf5406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:35.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 --2- 172.21.15.60:0/1512687705 >> v2:172.21.15.100:3300/0 conn(0x7fbf5406ea50 0x7fbf5406ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 >> v1:172.21.15.100:6789/0 conn(0x7fbf54160fb0 legacy=0x7fbf54161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5415d9a0 con 0x7fbf540d0ed0 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fbf4c0019f0 con 0x7fbf540d0ed0 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 >> v1:172.21.15.60:6789/0 conn(0x7fbf540d0ed0 legacy=0x7fbf540d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c003970 0x7fbf3c003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf3c004370 0x7fbf3c004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf3c0050e0 0x7fbf3c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5415e760 con 0x7fbf3c003970 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c00c020 con 0x7fbf3c004370 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.043+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c00c1e0 con 0x7fbf3c0050e0 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.044+0000 7fbf52ffd640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c003970 0x7fbf3c003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.044+0000 7fbf527fc640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf3c004370 0x7fbf3c004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.044+0000 7fbf537fe640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf3c0050e0 0x7fbf3c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.044+0000 7fbf52ffd640 1 -- 172.21.15.60:0/1512687705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf3c0050e0 msgr2=0x7fbf3c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.044+0000 7fbf52ffd640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf3c0050e0 0x7fbf3c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.044+0000 7fbf52ffd640 1 -- 172.21.15.60:0/1512687705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf3c004370 msgr2=0x7fbf3c004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.044+0000 7fbf52ffd640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf3c004370 0x7fbf3c004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.044+0000 7fbf52ffd640 1 -- 172.21.15.60:0/1512687705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c003520 con 0x7fbf3c003970 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.044+0000 7fbf537fe640 1 -- 172.21.15.60:0/1512687705 reap_dead start 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf52ffd640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c003970 0x7fbf3c003d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009ac0 tx=0x7fbf3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df161b724c72fbeb server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4402c6b0 con 0x7fbf3c003970 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4402c810 con 0x7fbf3c003970 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf51ffb640 1 -- 172.21.15.60:0/1512687705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4402cba0 con 0x7fbf3c003970 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf59533640 1 -- 172.21.15.60:0/1512687705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c003970 msgr2=0x7fbf3c003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf59533640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c003970 0x7fbf3c003d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009ac0 tx=0x7fbf3c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf59533640 1 -- 172.21.15.60:0/1512687705 shutdown_connections 2023-11-18T20:42:35.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf59533640 1 --2- 172.21.15.60:0/1512687705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf3c003970 0x7fbf3c003d50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf59533640 1 -- 172.21.15.60:0/1512687705 >> 172.21.15.60:0/1512687705 conn(0x7fbf5406a6b0 msgr2=0x7fbf5406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:35.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf59533640 1 -- 172.21.15.60:0/1512687705 shutdown_connections 2023-11-18T20:42:35.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.045+0000 7fbf59533640 1 -- 172.21.15.60:0/1512687705 wait complete. 2023-11-18T20:42:35.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.046+0000 7fbf59533640 1 Processor -- start 2023-11-18T20:42:35.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.046+0000 7fbf59533640 1 -- start start 2023-11-18T20:42:35.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf59533640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf3c003970 0x7fbf54160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf59533640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf5413ba20 0x7fbf5413b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf59533640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf5415c010 0x7fbf5415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf59533640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf541608c0 con 0x7fbf5415c010 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf59533640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5415c9d0 con 0x7fbf3c003970 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf59533640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54157ec0 con 0x7fbf5413ba20 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf5415c010 0x7fbf5415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf5415c010 0x7fbf5415c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59912/0 (socket says 172.21.15.60:59912) 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf537fe640 1 -- 172.21.15.60:0/4287168101 learned_addr learned my addr 172.21.15.60:0/4287168101 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf527fc640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf5413ba20 0x7fbf5413b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.047+0000 7fbf52ffd640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf3c003970 0x7fbf54160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf537fe640 1 -- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf5413ba20 msgr2=0x7fbf5413b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf537fe640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf5413ba20 0x7fbf5413b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf537fe640 1 -- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf3c003970 msgr2=0x7fbf54160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf537fe640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf3c003970 0x7fbf54160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf537fe640 1 -- 172.21.15.60:0/4287168101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5406b670 con 0x7fbf5415c010 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf537fe640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf5415c010 0x7fbf5415c490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf541407c0 tx=0x7fbf4c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf3b7fe640 1 -- 172.21.15.60:0/4287168101 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00ea20 con 0x7fbf5415c010 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf3b7fe640 1 -- 172.21.15.60:0/4287168101 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c00eba0 con 0x7fbf5415c010 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf59533640 1 -- 172.21.15.60:0/4287168101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf54154a40 con 0x7fbf5415c010 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf3b7fe640 1 -- 172.21.15.60:0/4287168101 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c013420 con 0x7fbf5415c010 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.048+0000 7fbf59533640 1 -- 172.21.15.60:0/4287168101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf54155000 con 0x7fbf5415c010 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.049+0000 7fbf59533640 1 -- 172.21.15.60:0/4287168101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5406b800 con 0x7fbf5415c010 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.049+0000 7fbf3b7fe640 1 -- 172.21.15.60:0/4287168101 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf4c013580 con 0x7fbf5415c010 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.050+0000 7fbf3b7fe640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf4006ae70 0x7fbf4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.050+0000 7fbf3b7fe640 1 -- 172.21.15.60:0/4287168101 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbf4c089420 con 0x7fbf5415c010 2023-11-18T20:42:35.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.050+0000 7fbf52ffd640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf4006ae70 0x7fbf4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.051+0000 7fbf52ffd640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf4006ae70 0x7fbf4006d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf44036000 tx=0x7fbf44030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:35.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.054+0000 7fbf3b7fe640 1 -- 172.21.15.60:0/4287168101 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf4c0548d0 con 0x7fbf5415c010 2023-11-18T20:42:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.232+0000 7fbf59533640 1 -- 172.21.15.60:0/4287168101 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbf54069400 con 0x7fbf5415c010 2023-11-18T20:42:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.232+0000 7fbf3b7fe640 1 -- 172.21.15.60:0/4287168101 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbf4c057ef0 con 0x7fbf5415c010 2023-11-18T20:42:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:35.234 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-11-18T20:42:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:42:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-11-18T20:42:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-11-18T20:42:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:42:35.234 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-11-18T20:42:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 -- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf4006ae70 msgr2=0x7fbf4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf4006ae70 0x7fbf4006d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf44036000 tx=0x7fbf44030000 comp rx=0 tx=0).stop 2023-11-18T20:42:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 -- 172.21.15.60:0/4287168101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf5415c010 msgr2=0x7fbf5415c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf5415c010 0x7fbf5415c490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf541407c0 tx=0x7fbf4c007bc0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 -- 172.21.15.60:0/4287168101 shutdown_connections 2023-11-18T20:42:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf4006ae70 0x7fbf4006d330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf5415c010 0x7fbf5415c490 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf5413ba20 0x7fbf5413b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 --2- 172.21.15.60:0/4287168101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf3c003970 0x7fbf54160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 -- 172.21.15.60:0/4287168101 >> 172.21.15.60:0/4287168101 conn(0x7fbf5406a6b0 msgr2=0x7fbf540b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 -- 172.21.15.60:0/4287168101 shutdown_connections 2023-11-18T20:42:35.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.234+0000 7fbf59533640 1 -- 172.21.15.60:0/4287168101 wait complete. 2023-11-18T20:42:35.245 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-18T20:42:35.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.354+0000 7f49eeb16640 1 Processor -- start 2023-11-18T20:42:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.354+0000 7f49eeb16640 1 -- start start 2023-11-18T20:42:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.355+0000 7f49eeb16640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e815b130 0x7f49e80d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.355+0000 7f49eeb16640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49e815f760 0x7f49e80cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.355+0000 7f49eeb16640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f49e80cf150 0x7f49e806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.355+0000 7f49eeb16640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e815fd20 con 0x7f49e815f760 2023-11-18T20:42:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.355+0000 7f49eeb16640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e80d0730 con 0x7f49e80cf150 2023-11-18T20:42:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.355+0000 7f49eeb16640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80cf5c0 con 0x7f49e815b130 2023-11-18T20:42:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.355+0000 7f49e7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49e815f760 0x7f49e80cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.355+0000 7f49e7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f49e815f760 0x7f49e80cec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59920/0 (socket says 172.21.15.60:59920) 2023-11-18T20:42:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.355+0000 7f49e7fff640 1 -- 172.21.15.60:0/1678280695 learned_addr learned my addr 172.21.15.60:0/1678280695 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.356+0000 7f49ec8b2640 1 --2- 172.21.15.60:0/1678280695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e815b130 0x7f49e80d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.356+0000 7f49ed0b3640 1 --2- 172.21.15.60:0/1678280695 >> v2:172.21.15.100:3300/0 conn(0x7f49e80cf150 0x7f49e806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.356+0000 7f49e7fff640 1 -- 172.21.15.60:0/1678280695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e815b130 msgr2=0x7f49e80d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.356+0000 7f49e7fff640 1 --2- 172.21.15.60:0/1678280695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e815b130 0x7f49e80d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.356+0000 7f49e7fff640 1 -- 172.21.15.60:0/1678280695 >> v2:172.21.15.100:3300/0 conn(0x7f49e80cf150 msgr2=0x7f49e806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.356+0000 7f49e7fff640 1 --2- 172.21.15.60:0/1678280695 >> v2:172.21.15.100:3300/0 conn(0x7f49e80cf150 0x7f49e806ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.356+0000 7f49e7fff640 1 -- 172.21.15.60:0/1678280695 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e80b3f10 con 0x7f49e815f760 2023-11-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.356+0000 7f49e7fff640 1 --2- 172.21.15.60:0/1678280695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f760 0x7f49e80cec10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f49e80cce80 tx=0x7f49dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38294b1a5d5f0211 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49e77fe640 1 -- 172.21.15.60:0/1678280695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc031640 con 0x7f49e815f760 2023-11-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49e77fe640 1 -- 172.21.15.60:0/1678280695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49dc0317a0 con 0x7f49e815f760 2023-11-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49e77fe640 1 -- 172.21.15.60:0/1678280695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc031b30 con 0x7f49e815f760 2023-11-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49eeb16640 1 -- 172.21.15.60:0/1678280695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f760 msgr2=0x7f49e80cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49eeb16640 1 --2- 172.21.15.60:0/1678280695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f760 0x7f49e80cec10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f49e80cce80 tx=0x7f49dc02b8f0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49eeb16640 1 -- 172.21.15.60:0/1678280695 shutdown_connections 2023-11-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49eeb16640 1 --2- 172.21.15.60:0/1678280695 >> v2:172.21.15.100:3300/0 conn(0x7f49e80cf150 0x7f49e806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49eeb16640 1 --2- 172.21.15.60:0/1678280695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f760 0x7f49e80cec10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49eeb16640 1 --2- 172.21.15.60:0/1678280695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e815b130 0x7f49e80d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49eeb16640 1 -- 172.21.15.60:0/1678280695 >> 172.21.15.60:0/1678280695 conn(0x7f49e80b2c10 msgr2=0x7f49e80b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49eeb16640 1 -- 172.21.15.60:0/1678280695 shutdown_connections 2023-11-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.357+0000 7f49eeb16640 1 -- 172.21.15.60:0/1678280695 wait complete. 2023-11-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.358+0000 7f49eeb16640 1 Processor -- start 2023-11-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.358+0000 7f49eeb16640 1 -- start start 2023-11-18T20:42:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.359+0000 7f49eeb16640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e80cf150 0x7f49e8150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.359+0000 7f49eeb16640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815b130 0x7f49e81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.359+0000 7f49eeb16640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e815f760 0x7f49e815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.359+0000 7f49eeb16640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e8150ba0 con 0x7f49e815b130 2023-11-18T20:42:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.359+0000 7f49eeb16640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e8150da0 con 0x7f49e815f760 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.359+0000 7f49eeb16640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80ca570 con 0x7f49e80cf150 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.359+0000 7f49e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815b130 0x7f49e81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.359+0000 7f49e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815b130 0x7f49e81582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59930/0 (socket says 172.21.15.60:59930) 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.359+0000 7f49e7fff640 1 -- 172.21.15.60:0/1606220499 learned_addr learned my addr 172.21.15.60:0/1606220499 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49ec8b2640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e80cf150 0x7f49e8150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49ed0b3640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e815f760 0x7f49e815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49e7fff640 1 -- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e80cf150 msgr2=0x7f49e8150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49e7fff640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e80cf150 0x7f49e8150660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49e7fff640 1 -- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e815f760 msgr2=0x7f49e815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49e7fff640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e815f760 0x7f49e815cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49e7fff640 1 -- 172.21.15.60:0/1606220499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc0094d0 con 0x7f49e815b130 2023-11-18T20:42:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49e7fff640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815b130 0x7f49e81582b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49dc02b930 tx=0x7f49dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49e57fa640 1 -- 172.21.15.60:0/1606220499 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc03ac60 con 0x7f49e815b130 2023-11-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49e57fa640 1 -- 172.21.15.60:0/1606220499 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49dc03ade0 con 0x7f49e815b130 2023-11-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49e57fa640 1 -- 172.21.15.60:0/1606220499 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc044630 con 0x7f49e815b130 2023-11-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49eeb16640 1 -- 172.21.15.60:0/1606220499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e80ca850 con 0x7f49e815b130 2023-11-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.360+0000 7f49eeb16640 1 -- 172.21.15.60:0/1606220499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e80cae10 con 0x7f49e815b130 2023-11-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.361+0000 7f49eeb16640 1 -- 172.21.15.60:0/1606220499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e8067fc0 con 0x7f49e815b130 2023-11-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.362+0000 7f49e57fa640 1 -- 172.21.15.60:0/1606220499 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49dc02f050 con 0x7f49e815b130 2023-11-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.362+0000 7f49e57fa640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f49d406ae70 0x7f49d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.362+0000 7f49e57fa640 1 -- 172.21.15.60:0/1606220499 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f49dc0aafe0 con 0x7f49e815b130 2023-11-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.362+0000 7f49ec8b2640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f49d406ae70 0x7f49d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.363+0000 7f49ec8b2640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f49d406ae70 0x7f49d406d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f49d8006fd0 tx=0x7f49d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.366+0000 7f49e57fa640 1 -- 172.21.15.60:0/1606220499 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49dc076490 con 0x7f49e815b130 2023-11-18T20:42:35.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:35.542+0000 7f49eeb16640 1 -- 172.21.15.60:0/1606220499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f49e8067420 con 0x7f49e815b130 2023-11-18T20:42:36.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.084+0000 7f49e57fa640 1 -- 172.21.15.60:0/1606220499 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f49dc079ab0 con 0x7f49e815b130 2023-11-18T20:42:36.086 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-11-18T20:42:36.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.088+0000 7f49eeb16640 1 -- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f49d406ae70 msgr2=0x7f49d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:36.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.088+0000 7f49eeb16640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f49d406ae70 0x7f49d406d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f49d8006fd0 tx=0x7f49d8008040 comp rx=0 tx=0).stop 2023-11-18T20:42:36.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.088+0000 7f49eeb16640 1 -- 172.21.15.60:0/1606220499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815b130 msgr2=0x7f49e81582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:36.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.088+0000 7f49eeb16640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815b130 0x7f49e81582b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49dc02b930 tx=0x7f49dc0046c0 comp rx=0 tx=0).stop 2023-11-18T20:42:36.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.088+0000 7f49eeb16640 1 -- 172.21.15.60:0/1606220499 shutdown_connections 2023-11-18T20:42:36.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.088+0000 7f49eeb16640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f49d406ae70 0x7f49d406d330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:36.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.088+0000 7f49eeb16640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e815f760 0x7f49e815cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:36.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.088+0000 7f49eeb16640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815b130 0x7f49e81582b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:36.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.088+0000 7f49eeb16640 1 --2- 172.21.15.60:0/1606220499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e80cf150 0x7f49e8150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:36.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.088+0000 7f49eeb16640 1 -- 172.21.15.60:0/1606220499 >> 172.21.15.60:0/1606220499 conn(0x7f49e80b2c10 msgr2=0x7f49e80b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:36.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.089+0000 7f49eeb16640 1 -- 172.21.15.60:0/1606220499 shutdown_connections 2023-11-18T20:42:36.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.089+0000 7f49eeb16640 1 -- 172.21.15.60:0/1606220499 wait complete. 2023-11-18T20:42:36.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-18T20:42:36.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.224+0000 7f8da3cd4640 1 Processor -- start 2023-11-18T20:42:36.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.224+0000 7f8da3cd4640 1 -- start start 2023-11-18T20:42:36.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da3cd4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c146010 0x7f8d9c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:36.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da3cd4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d9c0b3400 0x7f8d9c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:36.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da3cd4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c161170 con 0x7f8d9c146d70 2023-11-18T20:42:36.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da3cd4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c161450 con 0x7f8d9c0b3400 2023-11-18T20:42:36.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da3cd4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c161730 con 0x7f8d9c146010 2023-11-18T20:42:36.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da2271640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d9c0b3400 0x7f8d9c06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:36.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da2271640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d9c0b3400 0x7f8d9c06c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59944/0 (socket says 172.21.15.60:59944) 2023-11-18T20:42:36.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da2271640 1 -- 172.21.15.60:0/2463914416 learned_addr learned my addr 172.21.15.60:0/2463914416 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:36.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da1a70640 1 --2- 172.21.15.60:0/2463914416 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c146010 0x7f8d9c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:36.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da2271640 1 -- 172.21.15.60:0/2463914416 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c146010 msgr2=0x7f8d9c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:36.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da2271640 1 --2- 172.21.15.60:0/2463914416 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c146010 0x7f8d9c0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:36.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da2271640 1 -- 172.21.15.60:0/2463914416 >> v1:172.21.15.100:6789/0 conn(0x7f8d9c146d70 legacy=0x7f8d9c0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:36.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da2271640 1 -- 172.21.15.60:0/2463914416 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c161a10 con 0x7f8d9c0b3400 2023-11-18T20:42:36.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.225+0000 7f8da1a70640 1 --2- 172.21.15.60:0/2463914416 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c146010 0x7f8d9c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.226+0000 7f8da2271640 1 --2- 172.21.15.60:0/2463914416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c0b3400 0x7f8d9c06c6b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009870 tx=0x7f8d9402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c266940cb010ea server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.226+0000 7f8da0a6e640 1 -- 172.21.15.60:0/2463914416 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94030840 con 0x7f8d9c0b3400 2023-11-18T20:42:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.226+0000 7f8da0a6e640 1 -- 172.21.15.60:0/2463914416 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d940309a0 con 0x7f8d9c0b3400 2023-11-18T20:42:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.226+0000 7f8da0a6e640 1 -- 172.21.15.60:0/2463914416 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94030d30 con 0x7f8d9c0b3400 2023-11-18T20:42:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.227+0000 7f8da3cd4640 1 -- 172.21.15.60:0/2463914416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c0b3400 msgr2=0x7f8d9c06c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.227+0000 7f8da3cd4640 1 --2- 172.21.15.60:0/2463914416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c0b3400 0x7f8d9c06c6b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009870 tx=0x7f8d9402b980 comp rx=0 tx=0).stop 2023-11-18T20:42:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.227+0000 7f8da3cd4640 1 -- 172.21.15.60:0/2463914416 shutdown_connections 2023-11-18T20:42:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.227+0000 7f8da3cd4640 1 --2- 172.21.15.60:0/2463914416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c0b3400 0x7f8d9c06c6b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.227+0000 7f8da3cd4640 1 --2- 172.21.15.60:0/2463914416 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c146010 0x7f8d9c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.227+0000 7f8da3cd4640 1 -- 172.21.15.60:0/2463914416 >> 172.21.15.60:0/2463914416 conn(0x7f8d9c06a4d0 msgr2=0x7f8d9c06a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.227+0000 7f8da3cd4640 1 -- 172.21.15.60:0/2463914416 shutdown_connections 2023-11-18T20:42:36.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.227+0000 7f8da3cd4640 1 -- 172.21.15.60:0/2463914416 wait complete. 2023-11-18T20:42:36.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.228+0000 7f8da3cd4640 1 Processor -- start 2023-11-18T20:42:36.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.228+0000 7f8da3cd4640 1 -- start start 2023-11-18T20:42:36.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da3cd4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c0b3400 0x7f8d9c0b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:36.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da3cd4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c146010 0x7f8d9c13fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da3cd4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c146d70 0x7f8d9c13e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da3cd4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1f7170 con 0x7f8d9c146d70 2023-11-18T20:42:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da3cd4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1f64f0 con 0x7f8d9c146010 2023-11-18T20:42:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da3cd4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1f67d0 con 0x7f8d9c0b3400 2023-11-18T20:42:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da2271640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c146d70 0x7f8d9c13e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da2271640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c146d70 0x7f8d9c13e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59960/0 (socket says 172.21.15.60:59960) 2023-11-18T20:42:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da2271640 1 -- 172.21.15.60:0/1398992163 learned_addr learned my addr 172.21.15.60:0/1398992163 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da126f640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c146010 0x7f8d9c13fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da1a70640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c0b3400 0x7f8d9c0b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da2271640 1 -- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c0b3400 msgr2=0x7f8d9c0b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da2271640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c0b3400 0x7f8d9c0b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da2271640 1 -- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c146010 msgr2=0x7f8d9c13fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da2271640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c146010 0x7f8d9c13fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da2271640 1 -- 172.21.15.60:0/1398992163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d940094d0 con 0x7f8d9c146d70 2023-11-18T20:42:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8da2271640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c146d70 0x7f8d9c13e0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009870 tx=0x7f8d94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8d927fc640 1 -- 172.21.15.60:0/1398992163 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94039c90 con 0x7f8d9c146d70 2023-11-18T20:42:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.229+0000 7f8d927fc640 1 -- 172.21.15.60:0/1398992163 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d94039e10 con 0x7f8d9c146d70 2023-11-18T20:42:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.230+0000 7f8da3cd4640 1 -- 172.21.15.60:0/1398992163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c146570 con 0x7f8d9c146d70 2023-11-18T20:42:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.230+0000 7f8d927fc640 1 -- 172.21.15.60:0/1398992163 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94043640 con 0x7f8d9c146d70 2023-11-18T20:42:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.230+0000 7f8da3cd4640 1 -- 172.21.15.60:0/1398992163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c1f6d90 con 0x7f8d9c146d70 2023-11-18T20:42:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.230+0000 7f8da3cd4640 1 -- 172.21.15.60:0/1398992163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9c1f72b0 con 0x7f8d9c146d70 2023-11-18T20:42:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.231+0000 7f8d927fc640 1 -- 172.21.15.60:0/1398992163 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d940437a0 con 0x7f8d9c146d70 2023-11-18T20:42:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.231+0000 7f8d927fc640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d8006ae40 0x7f8d8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.231+0000 7f8d927fc640 1 -- 172.21.15.60:0/1398992163 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f8d9403e070 con 0x7f8d9c146d70 2023-11-18T20:42:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.231+0000 7f8da1a70640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d8006ae40 0x7f8d8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.232+0000 7f8da1a70640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d8006ae40 0x7f8d8006d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d98003b80 tx=0x7f8d98009910 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:36.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.236+0000 7f8d927fc640 1 -- 172.21.15.60:0/1398992163 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d9c1f72b0 con 0x7f8d9c146d70 2023-11-18T20:42:36.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:36.412+0000 7f8da3cd4640 1 -- 172.21.15.60:0/1398992163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8d9c1f72b0 con 0x7f8d9c146d70 2023-11-18T20:42:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.087+0000 7f8d927fc640 1 -- 172.21.15.60:0/1398992163 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8d9c1f72b0 con 0x7f8d9c146d70 2023-11-18T20:42:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-11-18T20:42:37.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.091+0000 7f8da3cd4640 1 -- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d8006ae40 msgr2=0x7f8d8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:37.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.091+0000 7f8da3cd4640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d8006ae40 0x7f8d8006d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d98003b80 tx=0x7f8d98009910 comp rx=0 tx=0).stop 2023-11-18T20:42:37.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.091+0000 7f8da3cd4640 1 -- 172.21.15.60:0/1398992163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c146d70 msgr2=0x7f8d9c13e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:37.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.091+0000 7f8da3cd4640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c146d70 0x7f8d9c13e0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009870 tx=0x7f8d94004730 comp rx=0 tx=0).stop 2023-11-18T20:42:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.091+0000 7f8da3cd4640 1 -- 172.21.15.60:0/1398992163 shutdown_connections 2023-11-18T20:42:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.091+0000 7f8da3cd4640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d8006ae40 0x7f8d8006d300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.091+0000 7f8da3cd4640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9c146d70 0x7f8d9c13e0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.091+0000 7f8da3cd4640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9c146010 0x7f8d9c13fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.091+0000 7f8da3cd4640 1 --2- 172.21.15.60:0/1398992163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d9c0b3400 0x7f8d9c0b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.091+0000 7f8da3cd4640 1 -- 172.21.15.60:0/1398992163 >> 172.21.15.60:0/1398992163 conn(0x7f8d9c06a4d0 msgr2=0x7f8d9c0d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.092+0000 7f8da3cd4640 1 -- 172.21.15.60:0/1398992163 shutdown_connections 2023-11-18T20:42:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.092+0000 7f8da3cd4640 1 -- 172.21.15.60:0/1398992163 wait complete. 2023-11-18T20:42:37.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-18T20:42:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.232+0000 7f339b101640 1 Processor -- start 2023-11-18T20:42:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.232+0000 7f339b101640 1 -- start start 2023-11-18T20:42:37.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.232+0000 7f339b101640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f339415d2d0 0x7f33940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:37.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.232+0000 7f339b101640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33941614e0 con 0x7f3394160f70 2023-11-18T20:42:37.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.233+0000 7f339b101640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33941617c0 con 0x7f33940ce650 2023-11-18T20:42:37.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.233+0000 7f339b101640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3394161aa0 con 0x7f339415d2d0 2023-11-18T20:42:37.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.233+0000 7f3393fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3394160f70 0x7f33940cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55172/0 (socket says 172.21.15.60:55172) 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.233+0000 7f3393fff640 1 -- 172.21.15.60:0/761698610 learned_addr learned my addr 172.21.15.60:0/761698610 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.233+0000 7f3398e9d640 1 --2- 172.21.15.60:0/761698610 >> v2:172.21.15.100:3300/0 conn(0x7f339415d2d0 0x7f33940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.233+0000 7f33937fe640 1 -- 172.21.15.60:0/761698610 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224803575 0 0) 0x7f33941614e0 con 0x7f3394160f70 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.233+0000 7f33937fe640 1 -- 172.21.15.60:0/761698610 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3384003520 con 0x7f3394160f70 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.233+0000 7f3398e9d640 1 -- 172.21.15.60:0/761698610 >> v1:172.21.15.100:6789/0 conn(0x7f33940ce650 legacy=0x7f33940d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.233+0000 7f3398e9d640 1 -- 172.21.15.60:0/761698610 >> v1:172.21.15.60:6789/0 conn(0x7f3394160f70 legacy=0x7f33940cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.233+0000 7f3398e9d640 1 -- 172.21.15.60:0/761698610 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3394161d80 con 0x7f339415d2d0 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.234+0000 7f3398e9d640 1 --2- 172.21.15.60:0/761698610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f339415d2d0 0x7f33940b8b90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f33880097a0 tx=0x7f338802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b84d97d6fcf6ef server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.234+0000 7f33937fe640 1 -- 172.21.15.60:0/761698610 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388030610 con 0x7f339415d2d0 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.234+0000 7f33937fe640 1 -- 172.21.15.60:0/761698610 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3388030770 con 0x7f339415d2d0 2023-11-18T20:42:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.234+0000 7f33937fe640 1 -- 172.21.15.60:0/761698610 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388030b00 con 0x7f339415d2d0 2023-11-18T20:42:37.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.235+0000 7f339b101640 1 -- 172.21.15.60:0/761698610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f339415d2d0 msgr2=0x7f33940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:37.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.235+0000 7f339b101640 1 --2- 172.21.15.60:0/761698610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f339415d2d0 0x7f33940b8b90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f33880097a0 tx=0x7f338802b960 comp rx=0 tx=0).stop 2023-11-18T20:42:37.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.235+0000 7f339b101640 1 -- 172.21.15.60:0/761698610 shutdown_connections 2023-11-18T20:42:37.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.235+0000 7f339b101640 1 --2- 172.21.15.60:0/761698610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f339415d2d0 0x7f33940b8b90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:37.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.235+0000 7f339b101640 1 -- 172.21.15.60:0/761698610 >> 172.21.15.60:0/761698610 conn(0x7f3394072230 msgr2=0x7f33940cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:37.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.235+0000 7f339b101640 1 -- 172.21.15.60:0/761698610 shutdown_connections 2023-11-18T20:42:37.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.235+0000 7f339b101640 1 -- 172.21.15.60:0/761698610 wait complete. 2023-11-18T20:42:37.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.236+0000 7f339b101640 1 Processor -- start 2023-11-18T20:42:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.236+0000 7f339b101640 1 -- start start 2023-11-18T20:42:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f339b101640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33940ce650 0x7f33940b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f339b101640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339415d2d0 0x7f3394158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f339b101640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3394160f70 0x7f33941f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f339b101640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33940b1b10 con 0x7f339415d2d0 2023-11-18T20:42:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f339b101640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394159fc0 con 0x7f3394160f70 2023-11-18T20:42:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f339b101640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339415a2a0 con 0x7f33940ce650 2023-11-18T20:42:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f3393fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339415d2d0 0x7f3394158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f3393fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339415d2d0 0x7f3394158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59964/0 (socket says 172.21.15.60:59964) 2023-11-18T20:42:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f3393fff640 1 -- 172.21.15.60:0/3917248679 learned_addr learned my addr 172.21.15.60:0/3917248679 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f3398e9d640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33940ce650 0x7f33940b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f339969e640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3394160f70 0x7f33941f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f3393fff640 1 -- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33940ce650 msgr2=0x7f33940b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f3393fff640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33940ce650 0x7f33940b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f3393fff640 1 -- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3394160f70 msgr2=0x7f33941f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f3393fff640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3394160f70 0x7f33941f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f3393fff640 1 -- 172.21.15.60:0/3917248679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3388009400 con 0x7f339415d2d0 2023-11-18T20:42:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f3393fff640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339415d2d0 0x7f3394158530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f338000c410 tx=0x7f338000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.237+0000 7f33917fa640 1 -- 172.21.15.60:0/3917248679 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380005ce0 con 0x7f339415d2d0 2023-11-18T20:42:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.238+0000 7f33917fa640 1 -- 172.21.15.60:0/3917248679 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3380005e40 con 0x7f339415d2d0 2023-11-18T20:42:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.238+0000 7f339b101640 1 -- 172.21.15.60:0/3917248679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339415a580 con 0x7f339415d2d0 2023-11-18T20:42:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.238+0000 7f33917fa640 1 -- 172.21.15.60:0/3917248679 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800072f0 con 0x7f339415d2d0 2023-11-18T20:42:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.238+0000 7f339b101640 1 -- 172.21.15.60:0/3917248679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339415ab40 con 0x7f339415d2d0 2023-11-18T20:42:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.239+0000 7f339b101640 1 -- 172.21.15.60:0/3917248679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3394069400 con 0x7f339415d2d0 2023-11-18T20:42:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.239+0000 7f33917fa640 1 -- 172.21.15.60:0/3917248679 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3380007510 con 0x7f339415d2d0 2023-11-18T20:42:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.240+0000 7f33917fa640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f337806ae40 0x7f337806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.240+0000 7f33917fa640 1 -- 172.21.15.60:0/3917248679 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3380088b40 con 0x7f339415d2d0 2023-11-18T20:42:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.240+0000 7f3398e9d640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f337806ae40 0x7f337806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.240+0000 7f3398e9d640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f337806ae40 0x7f337806d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3388036000 tx=0x7f3388008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.244+0000 7f33917fa640 1 -- 172.21.15.60:0/3917248679 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3380053e50 con 0x7f339415d2d0 2023-11-18T20:42:37.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:37.458+0000 7f339b101640 1 -- 172.21.15.60:0/3917248679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f33940d0aa0 con 0x7f339415d2d0 2023-11-18T20:42:38.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.108+0000 7f33917fa640 1 -- 172.21.15.60:0/3917248679 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f33800170c0 con 0x7f339415d2d0 2023-11-18T20:42:38.110 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 2 and data pool 3 2023-11-18T20:42:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 -- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f337806ae40 msgr2=0x7f337806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f337806ae40 0x7f337806d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3388036000 tx=0x7f3388008030 comp rx=0 tx=0).stop 2023-11-18T20:42:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 -- 172.21.15.60:0/3917248679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339415d2d0 msgr2=0x7f3394158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339415d2d0 0x7f3394158530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f338000c410 tx=0x7f338000c8d0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 -- 172.21.15.60:0/3917248679 shutdown_connections 2023-11-18T20:42:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f337806ae40 0x7f337806d300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3394160f70 0x7f33941f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f339415d2d0 0x7f3394158530 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 --2- 172.21.15.60:0/3917248679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33940ce650 0x7f33940b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 -- 172.21.15.60:0/3917248679 >> 172.21.15.60:0/3917248679 conn(0x7f3394072230 msgr2=0x7f339415e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 -- 172.21.15.60:0/3917248679 shutdown_connections 2023-11-18T20:42:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.111+0000 7f339b101640 1 -- 172.21.15.60:0/3917248679 wait complete. 2023-11-18T20:42:38.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:38.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.239+0000 7f22eb894640 1 Processor -- start 2023-11-18T20:42:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.240+0000 7f22eb894640 1 -- start start 2023-11-18T20:42:38.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.240+0000 7f22eb894640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22e415d2d0 0x7f22e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.240+0000 7f22eb894640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22e4160f70 0x7f22e40cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.240+0000 7f22eb894640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f22e40ce480 0x7f22e406b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.240+0000 7f22eb894640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e41614e0 con 0x7f22e4160f70 2023-11-18T20:42:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.240+0000 7f22eb894640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e41617c0 con 0x7f22e40ce480 2023-11-18T20:42:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.240+0000 7f22eb894640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4161aa0 con 0x7f22e415d2d0 2023-11-18T20:42:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.240+0000 7f22e8e2f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22e4160f70 0x7f22e40cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.240+0000 7f22e8e2f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22e4160f70 0x7f22e40cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59968/0 (socket says 172.21.15.60:59968) 2023-11-18T20:42:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.241+0000 7f22e8e2f640 1 -- 172.21.15.60:0/3563481653 learned_addr learned my addr 172.21.15.60:0/3563481653 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.241+0000 7f22e9e31640 1 --2- 172.21.15.60:0/3563481653 >> v2:172.21.15.100:3300/0 conn(0x7f22e40ce480 0x7f22e406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.241+0000 7f22e9630640 1 --2- 172.21.15.60:0/3563481653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22e415d2d0 0x7f22e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.241+0000 7f22e8e2f640 1 -- 172.21.15.60:0/3563481653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22e415d2d0 msgr2=0x7f22e40b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.241+0000 7f22e8e2f640 1 --2- 172.21.15.60:0/3563481653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22e415d2d0 0x7f22e40b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.241+0000 7f22e8e2f640 1 -- 172.21.15.60:0/3563481653 >> v2:172.21.15.100:3300/0 conn(0x7f22e40ce480 msgr2=0x7f22e406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.241+0000 7f22e8e2f640 1 --2- 172.21.15.60:0/3563481653 >> v2:172.21.15.100:3300/0 conn(0x7f22e40ce480 0x7f22e406b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.241+0000 7f22e8e2f640 1 -- 172.21.15.60:0/3563481653 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4161d80 con 0x7f22e4160f70 2023-11-18T20:42:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.241+0000 7f22e9e31640 1 --2- 172.21.15.60:0/3563481653 >> v2:172.21.15.100:3300/0 conn(0x7f22e40ce480 0x7f22e406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:38.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22e8e2f640 1 --2- 172.21.15.60:0/3563481653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 0x7f22e40cdf40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f22e40cdb80 tx=0x7f22d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68860ddecfda9146 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22dbfff640 1 -- 172.21.15.60:0/3563481653 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4031870 con 0x7f22e4160f70 2023-11-18T20:42:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22dbfff640 1 -- 172.21.15.60:0/3563481653 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d40319d0 con 0x7f22e4160f70 2023-11-18T20:42:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22dbfff640 1 -- 172.21.15.60:0/3563481653 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4031d60 con 0x7f22e4160f70 2023-11-18T20:42:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22eb894640 1 -- 172.21.15.60:0/3563481653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 msgr2=0x7f22e40cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22eb894640 1 --2- 172.21.15.60:0/3563481653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 0x7f22e40cdf40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f22e40cdb80 tx=0x7f22d402b910 comp rx=0 tx=0).stop 2023-11-18T20:42:38.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22eb894640 1 -- 172.21.15.60:0/3563481653 shutdown_connections 2023-11-18T20:42:38.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22eb894640 1 --2- 172.21.15.60:0/3563481653 >> v2:172.21.15.100:3300/0 conn(0x7f22e40ce480 0x7f22e406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22eb894640 1 --2- 172.21.15.60:0/3563481653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 0x7f22e40cdf40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22eb894640 1 --2- 172.21.15.60:0/3563481653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22e415d2d0 0x7f22e40b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22eb894640 1 -- 172.21.15.60:0/3563481653 >> 172.21.15.60:0/3563481653 conn(0x7f22e4072230 msgr2=0x7f22e40cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:38.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22eb894640 1 -- 172.21.15.60:0/3563481653 shutdown_connections 2023-11-18T20:42:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.242+0000 7f22eb894640 1 -- 172.21.15.60:0/3563481653 wait complete. 2023-11-18T20:42:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22eb894640 1 Processor -- start 2023-11-18T20:42:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22eb894640 1 -- start start 2023-11-18T20:42:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22eb894640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e40ce480 0x7f22e4156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22eb894640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22e415d2d0 0x7f22e41f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22eb894640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 0x7f22e41f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22eb894640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4156ca0 con 0x7f22e4160f70 2023-11-18T20:42:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22eb894640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4158700 con 0x7f22e40ce480 2023-11-18T20:42:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22eb894640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e41589d0 con 0x7f22e415d2d0 2023-11-18T20:42:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22e9e31640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 0x7f22e41f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22e9e31640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 0x7f22e41f67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59970/0 (socket says 172.21.15.60:59970) 2023-11-18T20:42:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.244+0000 7f22e9e31640 1 -- 172.21.15.60:0/990998081 learned_addr learned my addr 172.21.15.60:0/990998081 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22e8e2f640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22e415d2d0 0x7f22e41f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22e9630640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e40ce480 0x7f22e4156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22e9e31640 1 -- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22e415d2d0 msgr2=0x7f22e41f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22e9e31640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22e415d2d0 0x7f22e41f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22e9e31640 1 -- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e40ce480 msgr2=0x7f22e4156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22e9e31640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e40ce480 0x7f22e4156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22e9e31640 1 -- 172.21.15.60:0/990998081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d40094d0 con 0x7f22e4160f70 2023-11-18T20:42:38.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22e9e31640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 0x7f22e41f67d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00e5e0 tx=0x7f22dc00eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:38.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22d9ffb640 1 -- 172.21.15.60:0/990998081 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc004230 con 0x7f22e4160f70 2023-11-18T20:42:38.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22d9ffb640 1 -- 172.21.15.60:0/990998081 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc004390 con 0x7f22e4160f70 2023-11-18T20:42:38.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22eb894640 1 -- 172.21.15.60:0/990998081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e4158cb0 con 0x7f22e4160f70 2023-11-18T20:42:38.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22d9ffb640 1 -- 172.21.15.60:0/990998081 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc003850 con 0x7f22e4160f70 2023-11-18T20:42:38.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.245+0000 7f22eb894640 1 -- 172.21.15.60:0/990998081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e415da00 con 0x7f22e4160f70 2023-11-18T20:42:38.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.246+0000 7f22eb894640 1 -- 172.21.15.60:0/990998081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e415dc20 con 0x7f22e4160f70 2023-11-18T20:42:38.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.247+0000 7f22d9ffb640 1 -- 172.21.15.60:0/990998081 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22dc0039b0 con 0x7f22e4160f70 2023-11-18T20:42:38.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.247+0000 7f22d9ffb640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22c806ae40 0x7f22c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.247+0000 7f22d9ffb640 1 -- 172.21.15.60:0/990998081 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22dc0890a0 con 0x7f22e4160f70 2023-11-18T20:42:38.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.247+0000 7f22e9630640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22c806ae40 0x7f22c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.248+0000 7f22e9630640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22c806ae40 0x7f22c806d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f22e40cc8d0 tx=0x7f22e00098d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:38.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.251+0000 7f22d9ffb640 1 -- 172.21.15.60:0/990998081 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22dc054290 con 0x7f22e4160f70 2023-11-18T20:42:38.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.428+0000 7f22eb894640 1 -- 172.21.15.60:0/990998081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22e4069400 con 0x7f22e4160f70 2023-11-18T20:42:38.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.429+0000 7f22d9ffb640 1 -- 172.21.15.60:0/990998081 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16506 (secure 0 0 0) 0x7f22dc0578b0 con 0x7f22e4160f70 2023-11-18T20:42:38.431 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:38.431 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":20,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:37.459248+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:38.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 -- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22c806ae40 msgr2=0x7f22c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22c806ae40 0x7f22c806d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f22e40cc8d0 tx=0x7f22e00098d0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 -- 172.21.15.60:0/990998081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 msgr2=0x7f22e41f67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 0x7f22e41f67d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00e5e0 tx=0x7f22dc00eaa0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 -- 172.21.15.60:0/990998081 shutdown_connections 2023-11-18T20:42:38.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22c806ae40 0x7f22c806d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22e4160f70 0x7f22e41f67d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22e415d2d0 0x7f22e41f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 --2- 172.21.15.60:0/990998081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22e40ce480 0x7f22e4156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 -- 172.21.15.60:0/990998081 >> 172.21.15.60:0/990998081 conn(0x7f22e4072230 msgr2=0x7f22e40b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:38.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 -- 172.21.15.60:0/990998081 shutdown_connections 2023-11-18T20:42:38.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.431+0000 7f22eb894640 1 -- 172.21.15.60:0/990998081 wait complete. 2023-11-18T20:42:38.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.548+0000 7fc9f2f7c640 1 Processor -- start 2023-11-18T20:42:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.548+0000 7fc9f2f7c640 1 -- start start 2023-11-18T20:42:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f2f7c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc9ec15d2d0 0x7fc9ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f2f7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec160f70 0x7fc9ec0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f2f7c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc9ec0ce480 0x7fc9ec06b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f2f7c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec1614e0 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f2f7c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec1617c0 con 0x7fc9ec0ce480 2023-11-18T20:42:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f2f7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec161aa0 con 0x7fc9ec160f70 2023-11-18T20:42:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f0d18640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc9ec15d2d0 0x7fc9ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f0d18640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc9ec15d2d0 0x7fc9ec0b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59976/0 (socket says 172.21.15.60:59976) 2023-11-18T20:42:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f0d18640 1 -- 172.21.15.60:0/381412462 learned_addr learned my addr 172.21.15.60:0/381412462 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f1519640 1 --2- 172.21.15.60:0/381412462 >> v2:172.21.15.100:3300/0 conn(0x7fc9ec0ce480 0x7fc9ec06b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9ebfff640 1 --2- 172.21.15.60:0/381412462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec160f70 0x7fc9ec0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f0d18640 1 -- 172.21.15.60:0/381412462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec160f70 msgr2=0x7fc9ec0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.549+0000 7fc9f0d18640 1 --2- 172.21.15.60:0/381412462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec160f70 0x7fc9ec0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.550+0000 7fc9f0d18640 1 -- 172.21.15.60:0/381412462 >> v2:172.21.15.100:3300/0 conn(0x7fc9ec0ce480 msgr2=0x7fc9ec06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:38.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.550+0000 7fc9f0d18640 1 --2- 172.21.15.60:0/381412462 >> v2:172.21.15.100:3300/0 conn(0x7fc9ec0ce480 0x7fc9ec06b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.550+0000 7fc9f0d18640 1 -- 172.21.15.60:0/381412462 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec161d80 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.550+0000 7fc9f1519640 1 --2- 172.21.15.60:0/381412462 >> v2:172.21.15.100:3300/0 conn(0x7fc9ec0ce480 0x7fc9ec06b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:38.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.550+0000 7fc9ebfff640 1 --2- 172.21.15.60:0/381412462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec160f70 0x7fc9ec0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:38.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.550+0000 7fc9f0d18640 1 --2- 172.21.15.60:0/381412462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 0x7fc9ec0b8b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0cdb80 tx=0x7fc9dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0bc6fc4e41a0f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:38.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.550+0000 7fc9eb7fe640 1 -- 172.21.15.60:0/381412462 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc031870 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.550+0000 7fc9eb7fe640 1 -- 172.21.15.60:0/381412462 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc0319d0 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.550+0000 7fc9eb7fe640 1 -- 172.21.15.60:0/381412462 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc031d60 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.551+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/381412462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 msgr2=0x7fc9ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.551+0000 7fc9f2f7c640 1 --2- 172.21.15.60:0/381412462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 0x7fc9ec0b8b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0cdb80 tx=0x7fc9dc02b910 comp rx=0 tx=0).stop 2023-11-18T20:42:38.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.551+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/381412462 shutdown_connections 2023-11-18T20:42:38.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.551+0000 7fc9f2f7c640 1 --2- 172.21.15.60:0/381412462 >> v2:172.21.15.100:3300/0 conn(0x7fc9ec0ce480 0x7fc9ec06b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.551+0000 7fc9f2f7c640 1 --2- 172.21.15.60:0/381412462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec160f70 0x7fc9ec0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.551+0000 7fc9f2f7c640 1 --2- 172.21.15.60:0/381412462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 0x7fc9ec0b8b90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.551+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/381412462 >> 172.21.15.60:0/381412462 conn(0x7fc9ec072230 msgr2=0x7fc9ec0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.551+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/381412462 shutdown_connections 2023-11-18T20:42:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.551+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/381412462 wait complete. 2023-11-18T20:42:38.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.552+0000 7fc9f2f7c640 1 Processor -- start 2023-11-18T20:42:38.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.552+0000 7fc9f2f7c640 1 -- start start 2023-11-18T20:42:38.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9f2f7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec0ce480 0x7fc9ec15b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9f2f7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 0x7fc9ec1f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9f2f7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ec160f70 0x7fc9ec1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9f2f7c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec1f3f20 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9f2f7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec1f3450 con 0x7fc9ec160f70 2023-11-18T20:42:38.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9f2f7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec1f3680 con 0x7fc9ec0ce480 2023-11-18T20:42:38.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9ebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 0x7fc9ec1f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9ebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 0x7fc9ec1f2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59990/0 (socket says 172.21.15.60:59990) 2023-11-18T20:42:38.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9ebfff640 1 -- 172.21.15.60:0/3136795569 learned_addr learned my addr 172.21.15.60:0/3136795569 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:38.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9f0d18640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec0ce480 0x7fc9ec15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.553+0000 7fc9f1519640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ec160f70 0x7fc9ec1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9ebfff640 1 -- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec0ce480 msgr2=0x7fc9ec15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9ebfff640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec0ce480 0x7fc9ec15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9ebfff640 1 -- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ec160f70 msgr2=0x7fc9ec1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9ebfff640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ec160f70 0x7fc9ec1586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9ebfff640 1 -- 172.21.15.60:0/3136795569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc0094d0 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9ebfff640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 0x7fc9ec1f2f10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c7f0 tx=0x7fc9e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:38.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9e97fa640 1 -- 172.21.15.60:0/3136795569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0007820 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9e97fa640 1 -- 172.21.15.60:0/3136795569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e0007980 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/3136795569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec1f3960 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9e97fa640 1 -- 172.21.15.60:0/3136795569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0007cf0 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.554+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/3136795569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec1f6660 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.555+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/3136795569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ec1f64c0 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.555+0000 7fc9e97fa640 1 -- 172.21.15.60:0/3136795569 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9e00113f0 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.556+0000 7fc9e97fa640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9d806ae40 0x7fc9d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.556+0000 7fc9e97fa640 1 -- 172.21.15.60:0/3136795569 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9e0089640 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.556+0000 7fc9f0d18640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9d806ae40 0x7fc9d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.557+0000 7fc9f0d18640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9d806ae40 0x7fc9d806d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc036000 tx=0x7fc9dc03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:38.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.560+0000 7fc9e97fa640 1 -- 172.21.15.60:0/3136795569 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9ec1f64c0 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.738+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/3136795569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9ec1f64c0 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.739+0000 7fc9e97fa640 1 -- 172.21.15.60:0/3136795569 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16506 (secure 0 0 0) 0x7fc9ec1f64c0 con 0x7fc9ec15d2d0 2023-11-18T20:42:38.740 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:38.741 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":20,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:37.459248+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9d806ae40 msgr2=0x7fc9d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9d806ae40 0x7fc9d806d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc036000 tx=0x7fc9dc03c040 comp rx=0 tx=0).stop 2023-11-18T20:42:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/3136795569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 msgr2=0x7fc9ec1f2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 0x7fc9ec1f2f10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c7f0 tx=0x7fc9e000ccb0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/3136795569 shutdown_connections 2023-11-18T20:42:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9d806ae40 0x7fc9d806d300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ec160f70 0x7fc9ec1586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ec15d2d0 0x7fc9ec1f2f10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 --2- 172.21.15.60:0/3136795569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ec0ce480 0x7fc9ec15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/3136795569 >> 172.21.15.60:0/3136795569 conn(0x7fc9ec072230 msgr2=0x7fc9ec15ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/3136795569 shutdown_connections 2023-11-18T20:42:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.741+0000 7fc9f2f7c640 1 -- 172.21.15.60:0/3136795569 wait complete. 2023-11-18T20:42:38.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-18T20:42:38.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:42:38.850 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:42:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.861+0000 7f41d7d1d640 1 Processor -- start 2023-11-18T20:42:38.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.861+0000 7f41d7d1d640 1 -- start start 2023-11-18T20:42:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.862+0000 7f41d7d1d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f41d0157b00 0x7f41d00714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.862+0000 7f41d7d1d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d0071a00 0x7f41d015a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.862+0000 7f41d7d1d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d0071e60 con 0x7f41d015ad60 2023-11-18T20:42:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.862+0000 7f41d7d1d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d006f510 con 0x7f41d0157b00 2023-11-18T20:42:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.862+0000 7f41d7d1d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d006e960 con 0x7f41d0071a00 2023-11-18T20:42:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.862+0000 7f41d52b8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d0071a00 0x7f41d015a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.862+0000 7f41d5ab9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f41d0157b00 0x7f41d00714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.862+0000 7f41d62ba640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f41d015ad60 0x7f41d015da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:60164/0 (socket says 172.21.15.60:60164) 2023-11-18T20:42:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.862+0000 7f41d62ba640 1 -- 172.21.15.60:0/1235053390 learned_addr learned my addr 172.21.15.60:0/1235053390 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.863+0000 7f41d52b8640 1 -- 172.21.15.60:0/1235053390 >> v2:172.21.15.100:3300/0 conn(0x7f41d0157b00 msgr2=0x7f41d00714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.863+0000 7f41d52b8640 1 --2- 172.21.15.60:0/1235053390 >> v2:172.21.15.100:3300/0 conn(0x7f41d0157b00 0x7f41d00714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.863+0000 7f41d4ab7640 1 -- 172.21.15.60:0/1235053390 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064071210 0 0) 0x7f41d0071e60 con 0x7f41d015ad60 2023-11-18T20:42:38.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.863+0000 7f41d52b8640 1 -- 172.21.15.60:0/1235053390 >> v1:172.21.15.100:6789/0 conn(0x7f41d015ad60 legacy=0x7f41d015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.863+0000 7f41d52b8640 1 -- 172.21.15.60:0/1235053390 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0157ee0 con 0x7f41d0071a00 2023-11-18T20:42:38.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.863+0000 7f41d52b8640 1 --2- 172.21.15.60:0/1235053390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d0071a00 0x7f41d015a820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0097a0 tx=0x7f41cc02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b4c5236f2234d25 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:38.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d4ab7640 1 -- 172.21.15.60:0/1235053390 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc02e070 con 0x7f41d0071a00 2023-11-18T20:42:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d4ab7640 1 -- 172.21.15.60:0/1235053390 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc0315a0 con 0x7f41d0071a00 2023-11-18T20:42:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d4ab7640 1 -- 172.21.15.60:0/1235053390 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc031930 con 0x7f41d0071a00 2023-11-18T20:42:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d7d1d640 1 -- 172.21.15.60:0/1235053390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d0071a00 msgr2=0x7f41d015a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d7d1d640 1 --2- 172.21.15.60:0/1235053390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d0071a00 0x7f41d015a820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0097a0 tx=0x7f41cc02b840 comp rx=0 tx=0).stop 2023-11-18T20:42:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d7d1d640 1 -- 172.21.15.60:0/1235053390 shutdown_connections 2023-11-18T20:42:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d7d1d640 1 --2- 172.21.15.60:0/1235053390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d0071a00 0x7f41d015a820 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d7d1d640 1 --2- 172.21.15.60:0/1235053390 >> v2:172.21.15.100:3300/0 conn(0x7f41d0157b00 0x7f41d00714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d7d1d640 1 -- 172.21.15.60:0/1235053390 >> 172.21.15.60:0/1235053390 conn(0x7f41d00b1de0 msgr2=0x7f41d00b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:38.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d7d1d640 1 -- 172.21.15.60:0/1235053390 shutdown_connections 2023-11-18T20:42:38.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.864+0000 7f41d7d1d640 1 -- 172.21.15.60:0/1235053390 wait complete. 2023-11-18T20:42:38.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.865+0000 7f41d7d1d640 1 Processor -- start 2023-11-18T20:42:38.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d7d1d640 1 -- start start 2023-11-18T20:42:38.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d7d1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41d0071a00 0x7f41d01489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d7d1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d0157b00 0x7f41d00afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d7d1d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d015ad60 0x7f41d00b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d7d1d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d01550f0 con 0x7f41d0071a00 2023-11-18T20:42:38.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d7d1d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d0155340 con 0x7f41d0157b00 2023-11-18T20:42:38.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d7d1d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d0155620 con 0x7f41d015ad60 2023-11-18T20:42:38.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d5ab9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41d0071a00 0x7f41d01489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d5ab9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41d0071a00 0x7f41d01489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59996/0 (socket says 172.21.15.60:59996) 2023-11-18T20:42:38.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d5ab9640 1 -- 172.21.15.60:0/3986049291 learned_addr learned my addr 172.21.15.60:0/3986049291 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:38.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d52b8640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d0157b00 0x7f41d00afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.866+0000 7f41d62ba640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d015ad60 0x7f41d00b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41d5ab9640 1 -- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d015ad60 msgr2=0x7f41d00b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41d5ab9640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d015ad60 0x7f41d00b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41d5ab9640 1 -- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d0157b00 msgr2=0x7f41d00afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41d5ab9640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d0157b00 0x7f41d00afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41d5ab9640 1 -- 172.21.15.60:0/3986049291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc009400 con 0x7f41d0071a00 2023-11-18T20:42:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41d5ab9640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41d0071a00 0x7f41d01489f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41c000e9b0 tx=0x7f41c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:38.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41c67fc640 1 -- 172.21.15.60:0/3986049291 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c0009670 con 0x7f41d0071a00 2023-11-18T20:42:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41c67fc640 1 -- 172.21.15.60:0/3986049291 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c000c9a0 con 0x7f41d0071a00 2023-11-18T20:42:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41d7d1d640 1 -- 172.21.15.60:0/3986049291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d0155900 con 0x7f41d0071a00 2023-11-18T20:42:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41c67fc640 1 -- 172.21.15.60:0/3986049291 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c000cd50 con 0x7f41d0071a00 2023-11-18T20:42:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.867+0000 7f41d7d1d640 1 -- 172.21.15.60:0/3986049291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d0148f30 con 0x7f41d0071a00 2023-11-18T20:42:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.868+0000 7f41d7d1d640 1 -- 172.21.15.60:0/3986049291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d0067420 con 0x7f41d0071a00 2023-11-18T20:42:38.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.870+0000 7f41c67fc640 1 -- 172.21.15.60:0/3986049291 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f41c0013470 con 0x7f41d0071a00 2023-11-18T20:42:38.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.870+0000 7f41c67fc640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41b406ae70 0x7f41b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:38.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.870+0000 7f41c67fc640 1 -- 172.21.15.60:0/3986049291 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41c00897a0 con 0x7f41d0071a00 2023-11-18T20:42:38.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.870+0000 7f41d52b8640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41b406ae70 0x7f41b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:38.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.871+0000 7f41d52b8640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41b406ae70 0x7f41b406d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41cc036000 tx=0x7f41cc03e040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:38.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:38.874+0000 7f41c67fc640 1 -- 172.21.15.60:0/3986049291 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41c0054910 con 0x7f41d0071a00 2023-11-18T20:42:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.092+0000 7f41d7d1d640 1 -- 172.21.15.60:0/3986049291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f41d006b1a0 con 0x7f41d0071a00 2023-11-18T20:42:39.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.109+0000 7f41c67fc640 1 -- 172.21.15.60:0/3986049291 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f41c0057f30 con 0x7f41d0071a00 2023-11-18T20:42:39.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 -- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41b406ae70 msgr2=0x7f41b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:39.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41b406ae70 0x7f41b406d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41cc036000 tx=0x7f41cc03e040 comp rx=0 tx=0).stop 2023-11-18T20:42:39.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 -- 172.21.15.60:0/3986049291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41d0071a00 msgr2=0x7f41d01489f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41d0071a00 0x7f41d01489f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41c000e9b0 tx=0x7f41c000ee70 comp rx=0 tx=0).stop 2023-11-18T20:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 -- 172.21.15.60:0/3986049291 shutdown_connections 2023-11-18T20:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41b406ae70 0x7f41b406d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:39.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41d015ad60 0x7f41d00b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41d0157b00 0x7f41d00afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 --2- 172.21.15.60:0/3986049291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41d0071a00 0x7f41d01489f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 -- 172.21.15.60:0/3986049291 >> 172.21.15.60:0/3986049291 conn(0x7f41d00b1de0 msgr2=0x7f41d00ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 -- 172.21.15.60:0/3986049291 shutdown_connections 2023-11-18T20:42:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.112+0000 7f41d7d1d640 1 -- 172.21.15.60:0/3986049291 wait complete. 2023-11-18T20:42:39.124 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-18T20:42:39.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-18T20:42:39.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.232+0000 7fc4c25b0640 1 Processor -- start 2023-11-18T20:42:39.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.232+0000 7fc4c25b0640 1 -- start start 2023-11-18T20:42:39.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.232+0000 7fc4c25b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc0ad640 0x7fc4bc06d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:39.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.232+0000 7fc4c25b0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc4bc06bbf0 0x7fc4bc06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:39.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4c25b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4bc0adae0 con 0x7fc4bc15d1d0 2023-11-18T20:42:39.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4c25b0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4bc0addc0 con 0x7fc4bc06bbf0 2023-11-18T20:42:39.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4c25b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0ae0a0 con 0x7fc4bc0ad640 2023-11-18T20:42:39.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4bb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc0ad640 0x7fc4bc06d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:39.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4c0b4d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc4bc06bbf0 0x7fc4bc06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:39.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4bb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc0ad640 0x7fc4bc06d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:33284/0 (socket says 172.21.15.60:33284) 2023-11-18T20:42:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4bb7fe640 1 -- 172.21.15.60:0/1730026911 learned_addr learned my addr 172.21.15.60:0/1730026911 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4bb7fe640 1 -- 172.21.15.60:0/1730026911 >> v2:172.21.15.100:3300/0 conn(0x7fc4bc06bbf0 msgr2=0x7fc4bc06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4bb7fe640 1 --2- 172.21.15.60:0/1730026911 >> v2:172.21.15.100:3300/0 conn(0x7fc4bc06bbf0 0x7fc4bc06d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4baffd640 1 -- 172.21.15.60:0/1730026911 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877450789 0 0) 0x7fc4bc0adae0 con 0x7fc4bc15d1d0 2023-11-18T20:42:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4bb7fe640 1 -- 172.21.15.60:0/1730026911 >> v1:172.21.15.100:6789/0 conn(0x7fc4bc15d1d0 legacy=0x7fc4bc06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.233+0000 7fc4bb7fe640 1 -- 172.21.15.60:0/1730026911 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc0ae380 con 0x7fc4bc0ad640 2023-11-18T20:42:39.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.234+0000 7fc4bb7fe640 1 --2- 172.21.15.60:0/1730026911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc0ad640 0x7fc4bc06d0b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00098a0 tx=0x7fc4b002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d9cdf93b5efd52 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:39.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.234+0000 7fc4baffd640 1 -- 172.21.15.60:0/1730026911 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0030840 con 0x7fc4bc0ad640 2023-11-18T20:42:39.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.234+0000 7fc4baffd640 1 -- 172.21.15.60:0/1730026911 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b00309a0 con 0x7fc4bc0ad640 2023-11-18T20:42:39.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.234+0000 7fc4baffd640 1 -- 172.21.15.60:0/1730026911 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0030d30 con 0x7fc4bc0ad640 2023-11-18T20:42:39.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.235+0000 7fc4c25b0640 1 -- 172.21.15.60:0/1730026911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc0ad640 msgr2=0x7fc4bc06d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:39.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.235+0000 7fc4c25b0640 1 --2- 172.21.15.60:0/1730026911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc0ad640 0x7fc4bc06d0b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00098a0 tx=0x7fc4b002b980 comp rx=0 tx=0).stop 2023-11-18T20:42:39.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.235+0000 7fc4c25b0640 1 -- 172.21.15.60:0/1730026911 shutdown_connections 2023-11-18T20:42:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.235+0000 7fc4c25b0640 1 --2- 172.21.15.60:0/1730026911 >> v2:172.21.15.100:3300/0 conn(0x7fc4bc06bbf0 0x7fc4bc06d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.235+0000 7fc4c25b0640 1 --2- 172.21.15.60:0/1730026911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc0ad640 0x7fc4bc06d0b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.235+0000 7fc4c25b0640 1 -- 172.21.15.60:0/1730026911 >> 172.21.15.60:0/1730026911 conn(0x7fc4bc070e70 msgr2=0x7fc4bc071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.235+0000 7fc4c25b0640 1 -- 172.21.15.60:0/1730026911 shutdown_connections 2023-11-18T20:42:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.235+0000 7fc4c25b0640 1 -- 172.21.15.60:0/1730026911 wait complete. 2023-11-18T20:42:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.236+0000 7fc4c25b0640 1 Processor -- start 2023-11-18T20:42:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.236+0000 7fc4c25b0640 1 -- start start 2023-11-18T20:42:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4c25b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc06bbf0 0x7fc4bc140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4c25b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4bc0ad640 0x7fc4bc15e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4c25b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4bc15d1d0 0x7fc4bc15eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4c25b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0cf2a0 con 0x7fc4bc0ad640 2023-11-18T20:42:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4c25b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0ce4e0 con 0x7fc4bc15d1d0 2023-11-18T20:42:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4c25b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0ce7c0 con 0x7fc4bc06bbf0 2023-11-18T20:42:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4bc0ad640 0x7fc4bc15e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4bc0ad640 0x7fc4bc15e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60012/0 (socket says 172.21.15.60:60012) 2023-11-18T20:42:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4bb7fe640 1 -- 172.21.15.60:0/65863743 learned_addr learned my addr 172.21.15.60:0/65863743 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4bbfff640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc06bbf0 0x7fc4bc140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4c0b4d640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4bc15d1d0 0x7fc4bc15eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4bb7fe640 1 -- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc06bbf0 msgr2=0x7fc4bc140120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4bb7fe640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc06bbf0 0x7fc4bc140120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4bb7fe640 1 -- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4bc15d1d0 msgr2=0x7fc4bc15eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:39.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4bb7fe640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4bc15d1d0 0x7fc4bc15eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:39.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.237+0000 7fc4bb7fe640 1 -- 172.21.15.60:0/65863743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b00094d0 con 0x7fc4bc0ad640 2023-11-18T20:42:39.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.238+0000 7fc4bb7fe640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4bc0ad640 0x7fc4bc15e5f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0000c00 tx=0x7fc4b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:39.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.238+0000 7fc4b8ff9640 1 -- 172.21.15.60:0/65863743 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b003e040 con 0x7fc4bc0ad640 2023-11-18T20:42:39.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.238+0000 7fc4b8ff9640 1 -- 172.21.15.60:0/65863743 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b0039c90 con 0x7fc4bc0ad640 2023-11-18T20:42:39.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.238+0000 7fc4c25b0640 1 -- 172.21.15.60:0/65863743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc0ceaa0 con 0x7fc4bc0ad640 2023-11-18T20:42:39.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.238+0000 7fc4b8ff9640 1 -- 172.21.15.60:0/65863743 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0004030 con 0x7fc4bc0ad640 2023-11-18T20:42:39.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.238+0000 7fc4c25b0640 1 -- 172.21.15.60:0/65863743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc0ce1d0 con 0x7fc4bc0ad640 2023-11-18T20:42:39.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.239+0000 7fc4c25b0640 1 -- 172.21.15.60:0/65863743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc0cefc0 con 0x7fc4bc0ad640 2023-11-18T20:42:39.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.239+0000 7fc4b8ff9640 1 -- 172.21.15.60:0/65863743 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc4b00041b0 con 0x7fc4bc0ad640 2023-11-18T20:42:39.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.240+0000 7fc4b8ff9640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4a806ae40 0x7fc4a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:39.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.240+0000 7fc4b8ff9640 1 -- 172.21.15.60:0/65863743 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4b00abc10 con 0x7fc4bc0ad640 2023-11-18T20:42:39.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.240+0000 7fc4bbfff640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4a806ae40 0x7fc4a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:39.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.240+0000 7fc4bbfff640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4a806ae40 0x7fc4a806d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac000f60 tx=0x7fc4ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:39.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.244+0000 7fc4b8ff9640 1 -- 172.21.15.60:0/65863743 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc4bc0cefc0 con 0x7fc4bc0ad640 2023-11-18T20:42:39.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:39.456+0000 7fc4c25b0640 1 -- 172.21.15.60:0/65863743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc4bc0cefc0 con 0x7fc4bc0ad640 2023-11-18T20:42:40.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.117+0000 7fc4b8ff9640 1 -- 172.21.15.60:0/65863743 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc4bc0cefc0 con 0x7fc4bc0ad640 2023-11-18T20:42:40.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 -- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4a806ae40 msgr2=0x7fc4a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4a806ae40 0x7fc4a806d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac000f60 tx=0x7fc4ac009040 comp rx=0 tx=0).stop 2023-11-18T20:42:40.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 -- 172.21.15.60:0/65863743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4bc0ad640 msgr2=0x7fc4bc15e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4bc0ad640 0x7fc4bc15e5f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0000c00 tx=0x7fc4b0004730 comp rx=0 tx=0).stop 2023-11-18T20:42:40.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 -- 172.21.15.60:0/65863743 shutdown_connections 2023-11-18T20:42:40.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4a806ae40 0x7fc4a806d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4bc15d1d0 0x7fc4bc15eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4bc0ad640 0x7fc4bc15e5f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 --2- 172.21.15.60:0/65863743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4bc06bbf0 0x7fc4bc140120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 -- 172.21.15.60:0/65863743 >> 172.21.15.60:0/65863743 conn(0x7fc4bc070e70 msgr2=0x7fc4bc06d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:40.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 -- 172.21.15.60:0/65863743 shutdown_connections 2023-11-18T20:42:40.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.120+0000 7fc4c25b0640 1 -- 172.21.15.60:0/65863743 wait complete. 2023-11-18T20:42:40.135 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-18T20:42:40.135 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:42:40.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.247+0000 7fae530d7640 1 Processor -- start 2023-11-18T20:42:40.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.247+0000 7fae530d7640 1 -- start start 2023-11-18T20:42:40.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae530d7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fae4c160f70 0x7fae4c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae530d7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c0ce480 0x7fae4c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae530d7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4c1614e0 con 0x7fae4c15d2d0 2023-11-18T20:42:40.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae530d7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4c1617c0 con 0x7fae4c160f70 2023-11-18T20:42:40.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae530d7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c161aa0 con 0x7fae4c0ce480 2023-11-18T20:42:40.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae50e73640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fae4c15d2d0 0x7fae4c0b8b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55188/0 (socket says 172.21.15.60:55188) 2023-11-18T20:42:40.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae50e73640 1 -- 172.21.15.60:0/2377689011 learned_addr learned my addr 172.21.15.60:0/2377689011 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:40.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae51674640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c0ce480 0x7fae4c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae4bfff640 1 --2- 172.21.15.60:0/2377689011 >> v2:172.21.15.100:3300/0 conn(0x7fae4c160f70 0x7fae4c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604489836 0 0) 0x7fae4c1614e0 con 0x7fae4c15d2d0 2023-11-18T20:42:40.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae3c003520 con 0x7fae4c15d2d0 2023-11-18T20:42:40.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1427167009 0 0) 0x7fae3c003520 con 0x7fae4c15d2d0 2023-11-18T20:42:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c0ce480 msgr2=0x7fae4c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae4b7fe640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c0ce480 0x7fae4c0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 >> v2:172.21.15.100:3300/0 conn(0x7fae4c160f70 msgr2=0x7fae4c0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae4b7fe640 1 --2- 172.21.15.60:0/2377689011 >> v2:172.21.15.100:3300/0 conn(0x7fae4c160f70 0x7fae4c0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c161d80 con 0x7fae4c15d2d0 2023-11-18T20:42:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.248+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fae40001670 con 0x7fae4c15d2d0 2023-11-18T20:42:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 >> v1:172.21.15.60:6789/0 conn(0x7fae4c15d2d0 legacy=0x7fae4c0b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae4b7fe640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae3c003970 0x7fae3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae4b7fe640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c0ce480 0x7fae3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae4b7fe640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3c004d00 0x7fae3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1617c0 con 0x7fae3c003970 2023-11-18T20:42:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c161aa0 con 0x7fae3c004d00 2023-11-18T20:42:40.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c00b980 con 0x7fae4c0ce480 2023-11-18T20:42:40.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae50e73640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae3c003970 0x7fae3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae4bfff640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c0ce480 0x7fae3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae51674640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3c004d00 0x7fae3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae50e73640 1 -- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c0ce480 msgr2=0x7fae3c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:40.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae50e73640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c0ce480 0x7fae3c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae50e73640 1 -- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3c004d00 msgr2=0x7fae3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae50e73640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3c004d00 0x7fae3c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.249+0000 7fae50e73640 1 -- 172.21.15.60:0/2377689011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c003520 con 0x7fae3c003970 2023-11-18T20:42:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae50e73640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae3c003970 0x7fae3c003d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fae4000c210 tx=0x7fae3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a4081f119383fe server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4002cb00 con 0x7fae3c003970 2023-11-18T20:42:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4002cc60 con 0x7fae3c003970 2023-11-18T20:42:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae4b7fe640 1 -- 172.21.15.60:0/2377689011 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae400058d0 con 0x7fae3c003970 2023-11-18T20:42:40.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae530d7640 1 -- 172.21.15.60:0/2377689011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae3c003970 msgr2=0x7fae3c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae530d7640 1 --2- 172.21.15.60:0/2377689011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae3c003970 0x7fae3c003d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fae4000c210 tx=0x7fae3c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae51674640 1 -- 172.21.15.60:0/2377689011 reap_dead start 2023-11-18T20:42:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae530d7640 1 -- 172.21.15.60:0/2377689011 shutdown_connections 2023-11-18T20:42:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae530d7640 1 -- 172.21.15.60:0/2377689011 >> 172.21.15.60:0/2377689011 conn(0x7fae4c072230 msgr2=0x7fae4c0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae530d7640 1 -- 172.21.15.60:0/2377689011 shutdown_connections 2023-11-18T20:42:40.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.250+0000 7fae530d7640 1 -- 172.21.15.60:0/2377689011 wait complete. 2023-11-18T20:42:40.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.252+0000 7fae530d7640 1 Processor -- start 2023-11-18T20:42:40.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.252+0000 7fae530d7640 1 -- start start 2023-11-18T20:42:40.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.252+0000 7fae530d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3c003970 0x7fae4c153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.252+0000 7fae530d7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c1f04a0 0x7fae4c1f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.252+0000 7fae530d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae4c1f3620 0x7fae4c1f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.252+0000 7fae530d7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1f41f0 con 0x7fae4c1f3620 2023-11-18T20:42:40.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.252+0000 7fae530d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c0b2f00 con 0x7fae3c003970 2023-11-18T20:42:40.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.252+0000 7fae530d7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1f4010 con 0x7fae4c1f04a0 2023-11-18T20:42:40.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae51674640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae4c1f3620 0x7fae4c1f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae51674640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae4c1f3620 0x7fae4c1f3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60020/0 (socket says 172.21.15.60:60020) 2023-11-18T20:42:40.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae51674640 1 -- 172.21.15.60:0/4181167091 learned_addr learned my addr 172.21.15.60:0/4181167091 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:40.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae50e73640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3c003970 0x7fae4c153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae4bfff640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c1f04a0 0x7fae4c1f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae51674640 1 -- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c1f04a0 msgr2=0x7fae4c1f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae51674640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c1f04a0 0x7fae4c1f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae51674640 1 -- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3c003970 msgr2=0x7fae4c153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae51674640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3c003970 0x7fae4c153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae51674640 1 -- 172.21.15.60:0/4181167091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c0b2b60 con 0x7fae4c1f3620 2023-11-18T20:42:40.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae51674640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae4c1f3620 0x7fae4c1f3ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0d31b0 tx=0x7fae4400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:40.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae497fa640 1 -- 172.21.15.60:0/4181167091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4400e040 con 0x7fae4c1f3620 2023-11-18T20:42:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae497fa640 1 -- 172.21.15.60:0/4181167091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4400f040 con 0x7fae4c1f3620 2023-11-18T20:42:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae497fa640 1 -- 172.21.15.60:0/4181167091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44015430 con 0x7fae4c1f3620 2023-11-18T20:42:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae530d7640 1 -- 172.21.15.60:0/4181167091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c1541f0 con 0x7fae4c1f3620 2023-11-18T20:42:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.253+0000 7fae530d7640 1 -- 172.21.15.60:0/4181167091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c1547b0 con 0x7fae4c1f3620 2023-11-18T20:42:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.254+0000 7fae530d7640 1 -- 172.21.15.60:0/4181167091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c0b3040 con 0x7fae4c1f3620 2023-11-18T20:42:40.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.255+0000 7fae497fa640 1 -- 172.21.15.60:0/4181167091 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae44015590 con 0x7fae4c1f3620 2023-11-18T20:42:40.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.255+0000 7fae497fa640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fae3006ae70 0x7fae3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.255+0000 7fae497fa640 1 -- 172.21.15.60:0/4181167091 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae44089bc0 con 0x7fae4c1f3620 2023-11-18T20:42:40.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.255+0000 7fae50e73640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fae3006ae70 0x7fae3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.256+0000 7fae50e73640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fae3006ae70 0x7fae3006d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fae40034000 tx=0x7fae40009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:40.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.260+0000 7fae497fa640 1 -- 172.21.15.60:0/4181167091 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae4c0b3040 con 0x7fae4c1f3620 2023-11-18T20:42:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.474+0000 7fae530d7640 1 -- 172.21.15.60:0/4181167091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae4c0b3040 con 0x7fae4c1f3620 2023-11-18T20:42:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.474+0000 7fae497fa640 1 -- 172.21.15.60:0/4181167091 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fae4c0b3040 con 0x7fae4c1f3620 2023-11-18T20:42:40.476 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:40.476 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1530496056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1530496056},{"type":"v1","addr":"172.21.15.60:6833","nonce":1530496056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3842389676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3842389676},{"type":"v1","addr":"172.21.15.60:6835","nonce":3842389676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4248907695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4248907695},{"type":"v1","addr":"172.21.15.100:6836","nonce":4248907695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:42:37.459556+0000","modified":"2023-11-18T20:42:39.458016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1851816776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1851816776},{"type":"v1","addr":"172.21.15.100:6837","nonce":1851816776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-18T20:42:40.476 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2023-11-18T20:42:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 -- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fae3006ae70 msgr2=0x7fae3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fae3006ae70 0x7fae3006d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fae40034000 tx=0x7fae40009030 comp rx=0 tx=0).stop 2023-11-18T20:42:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 -- 172.21.15.60:0/4181167091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae4c1f3620 msgr2=0x7fae4c1f3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae4c1f3620 0x7fae4c1f3ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0d31b0 tx=0x7fae4400f1b0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 -- 172.21.15.60:0/4181167091 shutdown_connections 2023-11-18T20:42:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fae3006ae70 0x7fae3006d330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae4c1f3620 0x7fae4c1f3ad0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae4c1f04a0 0x7fae4c1f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 --2- 172.21.15.60:0/4181167091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae3c003970 0x7fae4c153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 -- 172.21.15.60:0/4181167091 >> 172.21.15.60:0/4181167091 conn(0x7fae4c072230 msgr2=0x7fae4c0d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 -- 172.21.15.60:0/4181167091 shutdown_connections 2023-11-18T20:42:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.476+0000 7fae530d7640 1 -- 172.21.15.60:0/4181167091 wait complete. 2023-11-18T20:42:40.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:40.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.593+0000 7fc2993f0640 1 Processor -- start 2023-11-18T20:42:40.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.593+0000 7fc2993f0640 1 -- start start 2023-11-18T20:42:40.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc2993f0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc29415c2f0 0x7fc29406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc2993f0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 0x7fc29413f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc2993f0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29415ba40 con 0x7fc29406f040 2023-11-18T20:42:40.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc2993f0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29415c6d0 con 0x7fc29415c2f0 2023-11-18T20:42:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc2993f0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29413e530 con 0x7fc294140f10 2023-11-18T20:42:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc292ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc29415c2f0 0x7fc29406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc292ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc29415c2f0 0x7fc29406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40962/0 (socket says 172.21.15.60:40962) 2023-11-18T20:42:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc292ffd640 1 -- 172.21.15.60:0/3560581551 learned_addr learned my addr 172.21.15.60:0/3560581551 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:40.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc2937fe640 1 --2- 172.21.15.60:0/3560581551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 0x7fc29413f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc292ffd640 1 -- 172.21.15.60:0/3560581551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 msgr2=0x7fc29413f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc292ffd640 1 --2- 172.21.15.60:0/3560581551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 0x7fc29413f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc292ffd640 1 -- 172.21.15.60:0/3560581551 >> v1:172.21.15.100:6789/0 conn(0x7fc29406f040 legacy=0x7fc294140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc292ffd640 1 -- 172.21.15.60:0/3560581551 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29406f4a0 con 0x7fc29415c2f0 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.594+0000 7fc2937fe640 1 --2- 172.21.15.60:0/3560581551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 0x7fc29413f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.595+0000 7fc292ffd640 1 --2- 172.21.15.60:0/3560581551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29415c2f0 0x7fc29406eb00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2840098a0 tx=0x7fc28402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9307d0950000ec6e server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.595+0000 7fc2927fc640 1 -- 172.21.15.60:0/3560581551 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284030840 con 0x7fc29415c2f0 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.595+0000 7fc2927fc640 1 -- 172.21.15.60:0/3560581551 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2840309a0 con 0x7fc29415c2f0 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.595+0000 7fc2927fc640 1 -- 172.21.15.60:0/3560581551 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284030d30 con 0x7fc29415c2f0 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.596+0000 7fc2993f0640 1 -- 172.21.15.60:0/3560581551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29415c2f0 msgr2=0x7fc29406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.596+0000 7fc2993f0640 1 --2- 172.21.15.60:0/3560581551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29415c2f0 0x7fc29406eb00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2840098a0 tx=0x7fc28402b980 comp rx=0 tx=0).stop 2023-11-18T20:42:40.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.596+0000 7fc2993f0640 1 -- 172.21.15.60:0/3560581551 shutdown_connections 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.596+0000 7fc2993f0640 1 --2- 172.21.15.60:0/3560581551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 0x7fc29413f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.596+0000 7fc2993f0640 1 --2- 172.21.15.60:0/3560581551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29415c2f0 0x7fc29406eb00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.596+0000 7fc2993f0640 1 -- 172.21.15.60:0/3560581551 >> 172.21.15.60:0/3560581551 conn(0x7fc29406a6b0 msgr2=0x7fc29406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.596+0000 7fc2993f0640 1 -- 172.21.15.60:0/3560581551 shutdown_connections 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.596+0000 7fc2993f0640 1 -- 172.21.15.60:0/3560581551 wait complete. 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.597+0000 7fc2993f0640 1 Processor -- start 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.597+0000 7fc2993f0640 1 -- start start 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc2993f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29406f040 0x7fc29413d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc2993f0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 0x7fc294152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc2993f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc29415c2f0 0x7fc2940b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc2993f0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2941f6310 con 0x7fc29406f040 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc2993f0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc294152c50 con 0x7fc29415c2f0 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc2993f0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2941f6560 con 0x7fc294140f10 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc292ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29406f040 0x7fc29413d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc292ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29406f040 0x7fc29413d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40964/0 (socket says 172.21.15.60:40964) 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc292ffd640 1 -- 172.21.15.60:0/48586491 learned_addr learned my addr 172.21.15.60:0/48586491 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc2937fe640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc29415c2f0 0x7fc2940b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc28bfff640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 0x7fc294152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc292ffd640 1 -- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 msgr2=0x7fc294152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc292ffd640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 0x7fc294152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc292ffd640 1 -- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc29415c2f0 msgr2=0x7fc2940b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc292ffd640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc29415c2f0 0x7fc2940b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc292ffd640 1 -- 172.21.15.60:0/48586491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2840094d0 con 0x7fc29406f040 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc28bfff640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 0x7fc294152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc292ffd640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29406f040 0x7fc29413d790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2840098a0 tx=0x7fc284004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc28b7fe640 1 -- 172.21.15.60:0/48586491 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284039a90 con 0x7fc29406f040 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc28b7fe640 1 -- 172.21.15.60:0/48586491 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc284039bf0 con 0x7fc29406f040 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc2993f0640 1 -- 172.21.15.60:0/48586491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2941f6840 con 0x7fc29406f040 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc28b7fe640 1 -- 172.21.15.60:0/48586491 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284043410 con 0x7fc29406f040 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.598+0000 7fc2993f0640 1 -- 172.21.15.60:0/48586491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2941f6e00 con 0x7fc29406f040 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.599+0000 7fc2993f0640 1 -- 172.21.15.60:0/48586491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2941f6ff0 con 0x7fc29406f040 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.600+0000 7fc28b7fe640 1 -- 172.21.15.60:0/48586491 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc28404c3b0 con 0x7fc29406f040 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.600+0000 7fc28b7fe640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc27406ae40 0x7fc27406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.600+0000 7fc28b7fe640 1 -- 172.21.15.60:0/48586491 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2840aba10 con 0x7fc29406f040 2023-11-18T20:42:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.600+0000 7fc28bfff640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc27406ae40 0x7fc27406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.601+0000 7fc28bfff640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc27406ae40 0x7fc27406d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc2940b2940 tx=0x7fc27c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:40.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.605+0000 7fc28b7fe640 1 -- 172.21.15.60:0/48586491 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc284076c70 con 0x7fc29406f040 2023-11-18T20:42:40.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.781+0000 7fc2993f0640 1 -- 172.21.15.60:0/48586491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc294069400 con 0x7fc29406f040 2023-11-18T20:42:40.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.782+0000 7fc28b7fe640 1 -- 172.21.15.60:0/48586491 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fc28407a290 con 0x7fc29406f040 2023-11-18T20:42:40.784 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:40.785 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:40.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 -- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc27406ae40 msgr2=0x7fc27406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc27406ae40 0x7fc27406d300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc2940b2940 tx=0x7fc27c009040 comp rx=0 tx=0).stop 2023-11-18T20:42:40.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 -- 172.21.15.60:0/48586491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29406f040 msgr2=0x7fc29413d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29406f040 0x7fc29413d790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2840098a0 tx=0x7fc284004730 comp rx=0 tx=0).stop 2023-11-18T20:42:40.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 -- 172.21.15.60:0/48586491 shutdown_connections 2023-11-18T20:42:40.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc27406ae40 0x7fc27406d300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc29415c2f0 0x7fc2940b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc294140f10 0x7fc294152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 --2- 172.21.15.60:0/48586491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc29406f040 0x7fc29413d790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 -- 172.21.15.60:0/48586491 >> 172.21.15.60:0/48586491 conn(0x7fc29406a6b0 msgr2=0x7fc29406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:40.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 -- 172.21.15.60:0/48586491 shutdown_connections 2023-11-18T20:42:40.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.784+0000 7fc2993f0640 1 -- 172.21.15.60:0/48586491 wait complete. 2023-11-18T20:42:40.796 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-18T20:42:40.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:42:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.902+0000 7f95bdc44640 1 Processor -- start 2023-11-18T20:42:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.902+0000 7f95bdc44640 1 -- start start 2023-11-18T20:42:40.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95bdc44640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95b80ad640 0x7f95b806d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95bdc44640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95b806bbf0 0x7f95b806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95bdc44640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b80adae0 con 0x7f95b815d1d0 2023-11-18T20:42:40.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95bdc44640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b80addc0 con 0x7f95b80ad640 2023-11-18T20:42:40.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95bdc44640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b80ae0a0 con 0x7f95b806bbf0 2023-11-18T20:42:40.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95b6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95b80ad640 0x7f95b806d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95b6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95b80ad640 0x7f95b806d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40978/0 (socket says 172.21.15.60:40978) 2023-11-18T20:42:40.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95b6ffd640 1 -- 172.21.15.60:0/4073944992 learned_addr learned my addr 172.21.15.60:0/4073944992 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:40.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95b7fff640 1 --2- 172.21.15.60:0/4073944992 >> v2:172.21.15.100:3300/0 conn(0x7f95b806bbf0 0x7f95b806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95b6ffd640 1 -- 172.21.15.60:0/4073944992 >> v2:172.21.15.100:3300/0 conn(0x7f95b806bbf0 msgr2=0x7f95b806d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95b6ffd640 1 --2- 172.21.15.60:0/4073944992 >> v2:172.21.15.100:3300/0 conn(0x7f95b806bbf0 0x7f95b806d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95b6ffd640 1 -- 172.21.15.60:0/4073944992 >> v1:172.21.15.100:6789/0 conn(0x7f95b815d1d0 legacy=0x7f95b806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95b6ffd640 1 -- 172.21.15.60:0/4073944992 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b80ae380 con 0x7f95b80ad640 2023-11-18T20:42:40.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.903+0000 7f95b7fff640 1 --2- 172.21.15.60:0/4073944992 >> v2:172.21.15.100:3300/0 conn(0x7f95b806bbf0 0x7f95b806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:40.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.904+0000 7f95b6ffd640 1 --2- 172.21.15.60:0/4073944992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80ad640 0x7f95b806d0b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f95b80b3d20 tx=0x7f95ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2968f4c8ae0600 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:40.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.904+0000 7f95b67fc640 1 -- 172.21.15.60:0/4073944992 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac031870 con 0x7f95b80ad640 2023-11-18T20:42:40.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.904+0000 7f95b67fc640 1 -- 172.21.15.60:0/4073944992 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac0319d0 con 0x7f95b80ad640 2023-11-18T20:42:40.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.904+0000 7f95b67fc640 1 -- 172.21.15.60:0/4073944992 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac031d60 con 0x7f95b80ad640 2023-11-18T20:42:40.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.905+0000 7f95bdc44640 1 -- 172.21.15.60:0/4073944992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80ad640 msgr2=0x7f95b806d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.905+0000 7f95bdc44640 1 --2- 172.21.15.60:0/4073944992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80ad640 0x7f95b806d0b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f95b80b3d20 tx=0x7f95ac02b910 comp rx=0 tx=0).stop 2023-11-18T20:42:40.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.905+0000 7f95bdc44640 1 -- 172.21.15.60:0/4073944992 shutdown_connections 2023-11-18T20:42:40.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.905+0000 7f95bdc44640 1 --2- 172.21.15.60:0/4073944992 >> v2:172.21.15.100:3300/0 conn(0x7f95b806bbf0 0x7f95b806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.905+0000 7f95bdc44640 1 --2- 172.21.15.60:0/4073944992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b80ad640 0x7f95b806d0b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.905+0000 7f95bdc44640 1 -- 172.21.15.60:0/4073944992 >> 172.21.15.60:0/4073944992 conn(0x7f95b8070e70 msgr2=0x7f95b8071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:40.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.905+0000 7f95bdc44640 1 -- 172.21.15.60:0/4073944992 shutdown_connections 2023-11-18T20:42:40.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.905+0000 7f95bdc44640 1 -- 172.21.15.60:0/4073944992 wait complete. 2023-11-18T20:42:40.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.906+0000 7f95bdc44640 1 Processor -- start 2023-11-18T20:42:40.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.906+0000 7f95bdc44640 1 -- start start 2023-11-18T20:42:40.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95bdc44640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b806bbf0 0x7f95b8151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95bdc44640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95b80ad640 0x7f95b8155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95bdc44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b815d1d0 0x7f95b815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95bdc44640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b81f3ef0 con 0x7f95b815d1d0 2023-11-18T20:42:40.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95bdc44640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b8151880 con 0x7f95b806bbf0 2023-11-18T20:42:40.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95bdc44640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b81f3370 con 0x7f95b80ad640 2023-11-18T20:42:40.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b815d1d0 0x7f95b815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b815d1d0 0x7f95b815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40986/0 (socket says 172.21.15.60:40986) 2023-11-18T20:42:40.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b7fff640 1 -- 172.21.15.60:0/3791324990 learned_addr learned my addr 172.21.15.60:0/3791324990 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:40.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b77fe640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b806bbf0 0x7f95b8151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b6ffd640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95b80ad640 0x7f95b8155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b7fff640 1 -- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95b80ad640 msgr2=0x7f95b8155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b7fff640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95b80ad640 0x7f95b8155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b7fff640 1 -- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b806bbf0 msgr2=0x7f95b8151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:40.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b7fff640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b806bbf0 0x7f95b8151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:40.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b7fff640 1 -- 172.21.15.60:0/3791324990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0094d0 con 0x7f95b815d1d0 2023-11-18T20:42:40.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b77fe640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b806bbf0 0x7f95b8151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:40.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b6ffd640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95b80ad640 0x7f95b8155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:40.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.907+0000 7f95b7fff640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b815d1d0 0x7f95b815b810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f95b000cef0 tx=0x7f95b000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:40.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.908+0000 7f959ffff640 1 -- 172.21.15.60:0/3791324990 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b000d840 con 0x7f95b815d1d0 2023-11-18T20:42:40.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.908+0000 7f959ffff640 1 -- 172.21.15.60:0/3791324990 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b000d9c0 con 0x7f95b815d1d0 2023-11-18T20:42:40.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.908+0000 7f959ffff640 1 -- 172.21.15.60:0/3791324990 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b000dd70 con 0x7f95b815d1d0 2023-11-18T20:42:40.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.908+0000 7f95bdc44640 1 -- 172.21.15.60:0/3791324990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b81f3650 con 0x7f95b815d1d0 2023-11-18T20:42:40.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.908+0000 7f95bdc44640 1 -- 172.21.15.60:0/3791324990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b81f3c10 con 0x7f95b815d1d0 2023-11-18T20:42:40.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.909+0000 7f95bdc44640 1 -- 172.21.15.60:0/3791324990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b8069400 con 0x7f95b815d1d0 2023-11-18T20:42:40.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.909+0000 7f959ffff640 1 -- 172.21.15.60:0/3791324990 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f95b001e3b0 con 0x7f95b815d1d0 2023-11-18T20:42:40.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.910+0000 7f959ffff640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95a406ae40 0x7f95a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:40.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.910+0000 7f959ffff640 1 -- 172.21.15.60:0/3791324990 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95b008a040 con 0x7f95b815d1d0 2023-11-18T20:42:40.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.910+0000 7f95b77fe640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95a406ae40 0x7f95a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:40.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.910+0000 7f95b77fe640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95a406ae40 0x7f95a406d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f95a800a830 tx=0x7f95a8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:40.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:40.914+0000 7f959ffff640 1 -- 172.21.15.60:0/3791324990 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95b00bce90 con 0x7f95b815d1d0 2023-11-18T20:42:41.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.089+0000 7f95bdc44640 1 -- 172.21.15.60:0/3791324990 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95b8066650 con 0x7f95a406ae40 2023-11-18T20:42:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.096+0000 7f959ffff640 1 -- 172.21.15.60:0/3791324990 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249857 (secure 0 0 0) 0x7f95b8066650 con 0x7f95a406ae40 2023-11-18T20:42:41.101 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:41.103 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:42:41.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 -- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95a406ae40 msgr2=0x7f95a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95a406ae40 0x7f95a406d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f95a800a830 tx=0x7f95a8008040 comp rx=0 tx=0).stop 2023-11-18T20:42:41.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 -- 172.21.15.60:0/3791324990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b815d1d0 msgr2=0x7f95b815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b815d1d0 0x7f95b815b810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f95b000cef0 tx=0x7f95b000a710 comp rx=0 tx=0).stop 2023-11-18T20:42:41.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 -- 172.21.15.60:0/3791324990 shutdown_connections 2023-11-18T20:42:41.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95a406ae40 0x7f95a406d300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95b815d1d0 0x7f95b815b810 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95b80ad640 0x7f95b8155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 --2- 172.21.15.60:0/3791324990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95b806bbf0 0x7f95b8151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 -- 172.21.15.60:0/3791324990 >> 172.21.15.60:0/3791324990 conn(0x7f95b8070e70 msgr2=0x7f95b80cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:41.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 -- 172.21.15.60:0/3791324990 shutdown_connections 2023-11-18T20:42:41.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.099+0000 7f95bdc44640 1 -- 172.21.15.60:0/3791324990 wait complete. 2023-11-18T20:42:41.111 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-18T20:42:39.435967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236028,"kb_used_data":2468,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738692,"statfs":{"total":773094113280,"available":772852420608,"internally_reserved":0,"allocated":2527232,"data_stored":1583429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000856"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-11-18T20:42:29.148453+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:42:29.148453+0000","last_peered":"2023-11-18T20:42:29.148453+0000","last_clean":"2023-11-18T20:42:29.148453+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:42:29.148453+0000","last_undegraded":"2023-11-18T20:42:29.148453+0000","last_fullsized":"2023-11-18T20:42:29.148453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:37.085490+0000","last_change":"2023-11-18T20:42:37.085490+0000","last_active":"2023-11-18T20:42:37.085490+0000","last_peered":"2023-11-18T20:42:37.085490+0000","last_clean":"2023-11-18T20:42:37.085490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:37.085490+0000","last_undegraded":"2023-11-18T20:42:37.085490+0000","last_fullsized":"2023-11-18T20:42:37.085490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-18T20:42:36.080881+0000","last_change":"2023-11-18T20:42:36.080881+0000","last_active":"2023-11-18T20:42:36.080881+0000","last_peered":"2023-11-18T20:42:36.080881+0000","last_clean":"2023-11-18T20:42:36.080881+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-18T20:42:36.080881+0000","last_undegraded":"2023-11-18T20:42:36.080881+0000","last_fullsized":"2023-11-18T20:42:36.080881+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":761856,"data_stored":640782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31348,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31352,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":757760,"data_stored":640163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:42:41.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:42:41.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.256+0000 7f9c70a94640 1 Processor -- start 2023-11-18T20:42:41.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.256+0000 7f9c70a94640 1 -- start start 2023-11-18T20:42:41.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c70a94640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c150560 0x7f9c6c0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c70a94640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9c6c0b3390 0x7f9c6c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c70a94640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c6c150940 0x7f9c6c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c70a94640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c6c0b49f0 con 0x7f9c6c150940 2023-11-18T20:42:41.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c70a94640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c6c0b3de0 con 0x7f9c6c0b3390 2023-11-18T20:42:41.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c70a94640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c0b40c0 con 0x7f9c6c150560 2023-11-18T20:42:41.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c6ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c6c150940 0x7f9c6c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c6ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c6c150940 0x7f9c6c15d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40994/0 (socket says 172.21.15.60:40994) 2023-11-18T20:42:41.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c6ad9d640 1 -- 172.21.15.60:0/32762751 learned_addr learned my addr 172.21.15.60:0/32762751 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:41.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c6a59c640 1 --2- 172.21.15.60:0/32762751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c150560 0x7f9c6c0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.257+0000 7f9c69d9b640 1 --2- 172.21.15.60:0/32762751 >> v2:172.21.15.100:3300/0 conn(0x7f9c6c0b3390 0x7f9c6c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c6ad9d640 1 -- 172.21.15.60:0/32762751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c150560 msgr2=0x7f9c6c0b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:41.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c6ad9d640 1 --2- 172.21.15.60:0/32762751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c150560 0x7f9c6c0b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c6ad9d640 1 -- 172.21.15.60:0/32762751 >> v2:172.21.15.100:3300/0 conn(0x7f9c6c0b3390 msgr2=0x7f9c6c15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:41.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c6ad9d640 1 --2- 172.21.15.60:0/32762751 >> v2:172.21.15.100:3300/0 conn(0x7f9c6c0b3390 0x7f9c6c15d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c6ad9d640 1 -- 172.21.15.60:0/32762751 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c0b43a0 con 0x7f9c6c150940 2023-11-18T20:42:41.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c69d9b640 1 --2- 172.21.15.60:0/32762751 >> v2:172.21.15.100:3300/0 conn(0x7f9c6c0b3390 0x7f9c6c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:41.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c6a59c640 1 --2- 172.21.15.60:0/32762751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c150560 0x7f9c6c0b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:41.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c6ad9d640 1 --2- 172.21.15.60:0/32762751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150940 0x7f9c6c15d740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c06e680 tx=0x7f9c5c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c784ae429189c3de server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:41.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c6959a640 1 -- 172.21.15.60:0/32762751 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c02ca70 con 0x7f9c6c150940 2023-11-18T20:42:41.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c6959a640 1 -- 172.21.15.60:0/32762751 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c02cbd0 con 0x7f9c6c150940 2023-11-18T20:42:41.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.258+0000 7f9c6959a640 1 -- 172.21.15.60:0/32762751 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0343f0 con 0x7f9c6c150940 2023-11-18T20:42:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.259+0000 7f9c70a94640 1 -- 172.21.15.60:0/32762751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150940 msgr2=0x7f9c6c15d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.259+0000 7f9c70a94640 1 --2- 172.21.15.60:0/32762751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150940 0x7f9c6c15d740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c06e680 tx=0x7f9c5c02bed0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.259+0000 7f9c70a94640 1 -- 172.21.15.60:0/32762751 shutdown_connections 2023-11-18T20:42:41.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.259+0000 7f9c70a94640 1 --2- 172.21.15.60:0/32762751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150940 0x7f9c6c15d740 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.259+0000 7f9c70a94640 1 --2- 172.21.15.60:0/32762751 >> v2:172.21.15.100:3300/0 conn(0x7f9c6c0b3390 0x7f9c6c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.259+0000 7f9c70a94640 1 --2- 172.21.15.60:0/32762751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c150560 0x7f9c6c0b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.259+0000 7f9c70a94640 1 -- 172.21.15.60:0/32762751 >> 172.21.15.60:0/32762751 conn(0x7f9c6c06a4d0 msgr2=0x7f9c6c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.259+0000 7f9c70a94640 1 -- 172.21.15.60:0/32762751 shutdown_connections 2023-11-18T20:42:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.259+0000 7f9c70a94640 1 -- 172.21.15.60:0/32762751 wait complete. 2023-11-18T20:42:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.260+0000 7f9c70a94640 1 Processor -- start 2023-11-18T20:42:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.260+0000 7f9c70a94640 1 -- start start 2023-11-18T20:42:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c70a94640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c0b3390 0x7f9c6c13b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c70a94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150560 0x7f9c6c15d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c70a94640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c157810 0x7f9c6c1495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c70a94640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c158970 con 0x7f9c6c150560 2023-11-18T20:42:41.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c70a94640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c157d60 con 0x7f9c6c0b3390 2023-11-18T20:42:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c70a94640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c158040 con 0x7f9c6c157810 2023-11-18T20:42:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c69d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150560 0x7f9c6c15d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c69d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150560 0x7f9c6c15d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41006/0 (socket says 172.21.15.60:41006) 2023-11-18T20:42:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c69d9b640 1 -- 172.21.15.60:0/579273684 learned_addr learned my addr 172.21.15.60:0/579273684 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c6ad9d640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c157810 0x7f9c6c1495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.261+0000 7f9c6a59c640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c0b3390 0x7f9c6c13b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c69d9b640 1 -- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c157810 msgr2=0x7f9c6c1495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c69d9b640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c157810 0x7f9c6c1495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c69d9b640 1 -- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c0b3390 msgr2=0x7f9c6c13b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c69d9b640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c0b3390 0x7f9c6c13b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c69d9b640 1 -- 172.21.15.60:0/579273684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c009b30 con 0x7f9c6c150560 2023-11-18T20:42:41.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c6a59c640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c0b3390 0x7f9c6c13b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c69d9b640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150560 0x7f9c6c15d0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c54010710 tx=0x7f9c54010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c62ffd640 1 -- 172.21.15.60:0/579273684 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54004230 con 0x7f9c6c150560 2023-11-18T20:42:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c62ffd640 1 -- 172.21.15.60:0/579273684 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c54004390 con 0x7f9c6c150560 2023-11-18T20:42:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c70a94640 1 -- 172.21.15.60:0/579273684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6c158320 con 0x7f9c6c150560 2023-11-18T20:42:41.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c62ffd640 1 -- 172.21.15.60:0/579273684 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54009b10 con 0x7f9c6c150560 2023-11-18T20:42:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.262+0000 7f9c70a94640 1 -- 172.21.15.60:0/579273684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c13bd10 con 0x7f9c6c150560 2023-11-18T20:42:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.263+0000 7f9c70a94640 1 -- 172.21.15.60:0/579273684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6c158b20 con 0x7f9c6c150560 2023-11-18T20:42:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.263+0000 7f9c62ffd640 1 -- 172.21.15.60:0/579273684 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c54009c70 con 0x7f9c6c150560 2023-11-18T20:42:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.264+0000 7f9c62ffd640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c4c06ae40 0x7f9c4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.264+0000 7f9c62ffd640 1 -- 172.21.15.60:0/579273684 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c5408b8f0 con 0x7f9c6c150560 2023-11-18T20:42:41.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.264+0000 7f9c6a59c640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c4c06ae40 0x7f9c4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.265+0000 7f9c6a59c640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c4c06ae40 0x7f9c4c06d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c64006fd0 tx=0x7f9c64008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.268+0000 7f9c62ffd640 1 -- 172.21.15.60:0/579273684 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c6c158b20 con 0x7f9c6c150560 2023-11-18T20:42:41.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.440+0000 7f9c70a94640 1 -- 172.21.15.60:0/579273684 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c6c0b4e80 con 0x7f9c4c06ae40 2023-11-18T20:42:41.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.448+0000 7f9c62ffd640 1 -- 172.21.15.60:0/579273684 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264685 (secure 0 0 0) 0x7f9c6400a810 con 0x7f9c4c06ae40 2023-11-18T20:42:41.451 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:41.453 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:42:41.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 -- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c4c06ae40 msgr2=0x7f9c4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c4c06ae40 0x7f9c4c06d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c64006fd0 tx=0x7f9c64008040 comp rx=0 tx=0).stop 2023-11-18T20:42:41.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 -- 172.21.15.60:0/579273684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150560 msgr2=0x7f9c6c15d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150560 0x7f9c6c15d0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c54010710 tx=0x7f9c54010bd0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 -- 172.21.15.60:0/579273684 shutdown_connections 2023-11-18T20:42:41.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c4c06ae40 0x7f9c4c06d300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c6c157810 0x7f9c6c1495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c6c150560 0x7f9c6c15d0e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 --2- 172.21.15.60:0/579273684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c0b3390 0x7f9c6c13b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 -- 172.21.15.60:0/579273684 >> 172.21.15.60:0/579273684 conn(0x7f9c6c06a4d0 msgr2=0x7f9c6c0b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:41.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 -- 172.21.15.60:0/579273684 shutdown_connections 2023-11-18T20:42:41.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.451+0000 7f9c70a94640 1 -- 172.21.15.60:0/579273684 wait complete. 2023-11-18T20:42:41.463 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-18T20:42:41.436342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235636,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739084,"statfs":{"total":773094113280,"available":772852822016,"internally_reserved":0,"allocated":2887680,"data_stored":1739185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.349976"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084419+0000","last_change":"2023-11-18T20:42:38.103913+0000","last_active":"2023-11-18T20:42:39.084419+0000","last_peered":"2023-11-18T20:42:39.084419+0000","last_clean":"2023-11-18T20:42:39.084419+0000","last_became_active":"2023-11-18T20:42:38.103117+0000","last_became_peered":"2023-11-18T20:42:38.103117+0000","last_unstale":"2023-11-18T20:42:39.084419+0000","last_undegraded":"2023-11-18T20:42:39.084419+0000","last_fullsized":"2023-11-18T20:42:39.084419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:48:49.998802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085771+0000","last_change":"2023-11-18T20:42:38.089225+0000","last_active":"2023-11-18T20:42:39.085771+0000","last_peered":"2023-11-18T20:42:39.085771+0000","last_clean":"2023-11-18T20:42:39.085771+0000","last_became_active":"2023-11-18T20:42:38.088874+0000","last_became_peered":"2023-11-18T20:42:38.088874+0000","last_unstale":"2023-11-18T20:42:39.085771+0000","last_undegraded":"2023-11-18T20:42:39.085771+0000","last_fullsized":"2023-11-18T20:42:39.085771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:12:16.083479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085159+0000","last_change":"2023-11-18T20:42:38.090166+0000","last_active":"2023-11-18T20:42:39.085159+0000","last_peered":"2023-11-18T20:42:39.085159+0000","last_clean":"2023-11-18T20:42:39.085159+0000","last_became_active":"2023-11-18T20:42:38.089879+0000","last_became_peered":"2023-11-18T20:42:38.089879+0000","last_unstale":"2023-11-18T20:42:39.085159+0000","last_undegraded":"2023-11-18T20:42:39.085159+0000","last_fullsized":"2023-11-18T20:42:39.085159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:02.525692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086389+0000","last_change":"2023-11-18T20:42:38.105690+0000","last_active":"2023-11-18T20:42:39.086389+0000","last_peered":"2023-11-18T20:42:39.086389+0000","last_clean":"2023-11-18T20:42:39.086389+0000","last_became_active":"2023-11-18T20:42:38.105419+0000","last_became_peered":"2023-11-18T20:42:38.105419+0000","last_unstale":"2023-11-18T20:42:39.086389+0000","last_undegraded":"2023-11-18T20:42:39.086389+0000","last_fullsized":"2023-11-18T20:42:39.086389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:35:56.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084162+0000","last_change":"2023-11-18T20:42:38.106388+0000","last_active":"2023-11-18T20:42:39.084162+0000","last_peered":"2023-11-18T20:42:39.084162+0000","last_clean":"2023-11-18T20:42:39.084162+0000","last_became_active":"2023-11-18T20:42:38.105875+0000","last_became_peered":"2023-11-18T20:42:38.105875+0000","last_unstale":"2023-11-18T20:42:39.084162+0000","last_undegraded":"2023-11-18T20:42:39.084162+0000","last_fullsized":"2023-11-18T20:42:39.084162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:35:58.132615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086539+0000","last_change":"2023-11-18T20:42:38.104968+0000","last_active":"2023-11-18T20:42:39.086539+0000","last_peered":"2023-11-18T20:42:39.086539+0000","last_clean":"2023-11-18T20:42:39.086539+0000","last_became_active":"2023-11-18T20:42:38.104671+0000","last_became_peered":"2023-11-18T20:42:38.104671+0000","last_unstale":"2023-11-18T20:42:39.086539+0000","last_undegraded":"2023-11-18T20:42:39.086539+0000","last_fullsized":"2023-11-18T20:42:39.086539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:51:04.970201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083359+0000","last_change":"2023-11-18T20:42:38.101566+0000","last_active":"2023-11-18T20:42:39.083359+0000","last_peered":"2023-11-18T20:42:39.083359+0000","last_clean":"2023-11-18T20:42:39.083359+0000","last_became_active":"2023-11-18T20:42:38.101316+0000","last_became_peered":"2023-11-18T20:42:38.101316+0000","last_unstale":"2023-11-18T20:42:39.083359+0000","last_undegraded":"2023-11-18T20:42:39.083359+0000","last_fullsized":"2023-11-18T20:42:39.083359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:28.378102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084443+0000","last_change":"2023-11-18T20:42:38.102722+0000","last_active":"2023-11-18T20:42:39.084443+0000","last_peered":"2023-11-18T20:42:39.084443+0000","last_clean":"2023-11-18T20:42:39.084443+0000","last_became_active":"2023-11-18T20:42:38.102348+0000","last_became_peered":"2023-11-18T20:42:38.102348+0000","last_unstale":"2023-11-18T20:42:39.084443+0000","last_undegraded":"2023-11-18T20:42:39.084443+0000","last_fullsized":"2023-11-18T20:42:39.084443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:02.525692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084379+0000","last_change":"2023-11-18T20:42:38.107273+0000","last_active":"2023-11-18T20:42:39.084379+0000","last_peered":"2023-11-18T20:42:39.084379+0000","last_clean":"2023-11-18T20:42:39.084379+0000","last_became_active":"2023-11-18T20:42:38.107006+0000","last_became_peered":"2023-11-18T20:42:38.107006+0000","last_unstale":"2023-11-18T20:42:39.084379+0000","last_undegraded":"2023-11-18T20:42:39.084379+0000","last_fullsized":"2023-11-18T20:42:39.084379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:44:25.920568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086226+0000","last_change":"2023-11-18T20:42:38.105599+0000","last_active":"2023-11-18T20:42:39.086226+0000","last_peered":"2023-11-18T20:42:39.086226+0000","last_clean":"2023-11-18T20:42:39.086226+0000","last_became_active":"2023-11-18T20:42:38.105338+0000","last_became_peered":"2023-11-18T20:42:38.105338+0000","last_unstale":"2023-11-18T20:42:39.086226+0000","last_undegraded":"2023-11-18T20:42:39.086226+0000","last_fullsized":"2023-11-18T20:42:39.086226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086057+0000","last_change":"2023-11-18T20:42:38.100971+0000","last_active":"2023-11-18T20:42:39.086057+0000","last_peered":"2023-11-18T20:42:39.086057+0000","last_clean":"2023-11-18T20:42:39.086057+0000","last_became_active":"2023-11-18T20:42:38.100686+0000","last_became_peered":"2023-11-18T20:42:38.100686+0000","last_unstale":"2023-11-18T20:42:39.086057+0000","last_undegraded":"2023-11-18T20:42:39.086057+0000","last_fullsized":"2023-11-18T20:42:39.086057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083618+0000","last_change":"2023-11-18T20:42:38.098826+0000","last_active":"2023-11-18T20:42:39.083618+0000","last_peered":"2023-11-18T20:42:39.083618+0000","last_clean":"2023-11-18T20:42:39.083618+0000","last_became_active":"2023-11-18T20:42:38.098277+0000","last_became_peered":"2023-11-18T20:42:38.098277+0000","last_unstale":"2023-11-18T20:42:39.083618+0000","last_undegraded":"2023-11-18T20:42:39.083618+0000","last_fullsized":"2023-11-18T20:42:39.083618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:13.814862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085307+0000","last_change":"2023-11-18T20:42:38.098784+0000","last_active":"2023-11-18T20:42:39.085307+0000","last_peered":"2023-11-18T20:42:39.085307+0000","last_clean":"2023-11-18T20:42:39.085307+0000","last_became_active":"2023-11-18T20:42:38.098486+0000","last_became_peered":"2023-11-18T20:42:38.098486+0000","last_unstale":"2023-11-18T20:42:39.085307+0000","last_undegraded":"2023-11-18T20:42:39.085307+0000","last_fullsized":"2023-11-18T20:42:39.085307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084851+0000","last_change":"2023-11-18T20:42:38.092883+0000","last_active":"2023-11-18T20:42:39.084851+0000","last_peered":"2023-11-18T20:42:39.084851+0000","last_clean":"2023-11-18T20:42:39.084851+0000","last_became_active":"2023-11-18T20:42:38.092521+0000","last_became_peered":"2023-11-18T20:42:38.092521+0000","last_unstale":"2023-11-18T20:42:39.084851+0000","last_undegraded":"2023-11-18T20:42:39.084851+0000","last_fullsized":"2023-11-18T20:42:39.084851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:18.854895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084758+0000","last_change":"2023-11-18T20:42:38.090613+0000","last_active":"2023-11-18T20:42:39.084758+0000","last_peered":"2023-11-18T20:42:39.084758+0000","last_clean":"2023-11-18T20:42:39.084758+0000","last_became_active":"2023-11-18T20:42:38.090293+0000","last_became_peered":"2023-11-18T20:42:38.090293+0000","last_unstale":"2023-11-18T20:42:39.084758+0000","last_undegraded":"2023-11-18T20:42:39.084758+0000","last_fullsized":"2023-11-18T20:42:39.084758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086000+0000","last_change":"2023-11-18T20:42:38.098541+0000","last_active":"2023-11-18T20:42:39.086000+0000","last_peered":"2023-11-18T20:42:39.086000+0000","last_clean":"2023-11-18T20:42:39.086000+0000","last_became_active":"2023-11-18T20:42:38.098239+0000","last_became_peered":"2023-11-18T20:42:38.098239+0000","last_unstale":"2023-11-18T20:42:39.086000+0000","last_undegraded":"2023-11-18T20:42:39.086000+0000","last_fullsized":"2023-11-18T20:42:39.086000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086377+0000","last_change":"2023-11-18T20:42:38.107966+0000","last_active":"2023-11-18T20:42:39.086377+0000","last_peered":"2023-11-18T20:42:39.086377+0000","last_clean":"2023-11-18T20:42:39.086377+0000","last_became_active":"2023-11-18T20:42:38.107169+0000","last_became_peered":"2023-11-18T20:42:38.107169+0000","last_unstale":"2023-11-18T20:42:39.086377+0000","last_undegraded":"2023-11-18T20:42:39.086377+0000","last_fullsized":"2023-11-18T20:42:39.086377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:12:16.083479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082349+0000","last_change":"2023-11-18T20:42:38.095583+0000","last_active":"2023-11-18T20:42:39.082349+0000","last_peered":"2023-11-18T20:42:39.082349+0000","last_clean":"2023-11-18T20:42:39.082349+0000","last_became_active":"2023-11-18T20:42:38.095274+0000","last_became_peered":"2023-11-18T20:42:38.095274+0000","last_unstale":"2023-11-18T20:42:39.082349+0000","last_undegraded":"2023-11-18T20:42:39.082349+0000","last_fullsized":"2023-11-18T20:42:39.082349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:28:13.121246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083819+0000","last_change":"2023-11-18T20:42:38.107739+0000","last_active":"2023-11-18T20:42:39.083819+0000","last_peered":"2023-11-18T20:42:39.083819+0000","last_clean":"2023-11-18T20:42:39.083819+0000","last_became_active":"2023-11-18T20:42:38.106917+0000","last_became_peered":"2023-11-18T20:42:38.106917+0000","last_unstale":"2023-11-18T20:42:39.083819+0000","last_undegraded":"2023-11-18T20:42:39.083819+0000","last_fullsized":"2023-11-18T20:42:39.083819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:48:04.295517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086374+0000","last_change":"2023-11-18T20:42:38.103858+0000","last_active":"2023-11-18T20:42:39.086374+0000","last_peered":"2023-11-18T20:42:39.086374+0000","last_clean":"2023-11-18T20:42:39.086374+0000","last_became_active":"2023-11-18T20:42:38.103576+0000","last_became_peered":"2023-11-18T20:42:38.103576+0000","last_unstale":"2023-11-18T20:42:39.086374+0000","last_undegraded":"2023-11-18T20:42:39.086374+0000","last_fullsized":"2023-11-18T20:42:39.086374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085692+0000","last_change":"2023-11-18T20:42:38.100901+0000","last_active":"2023-11-18T20:42:39.085692+0000","last_peered":"2023-11-18T20:42:39.085692+0000","last_clean":"2023-11-18T20:42:39.085692+0000","last_became_active":"2023-11-18T20:42:38.100356+0000","last_became_peered":"2023-11-18T20:42:38.100356+0000","last_unstale":"2023-11-18T20:42:39.085692+0000","last_undegraded":"2023-11-18T20:42:39.085692+0000","last_fullsized":"2023-11-18T20:42:39.085692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:26.508116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086544+0000","last_change":"2023-11-18T20:42:38.102174+0000","last_active":"2023-11-18T20:42:39.086544+0000","last_peered":"2023-11-18T20:42:39.086544+0000","last_clean":"2023-11-18T20:42:39.086544+0000","last_became_active":"2023-11-18T20:42:38.100136+0000","last_became_peered":"2023-11-18T20:42:38.100136+0000","last_unstale":"2023-11-18T20:42:39.086544+0000","last_undegraded":"2023-11-18T20:42:39.086544+0000","last_fullsized":"2023-11-18T20:42:39.086544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:41:14.693957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084520+0000","last_change":"2023-11-18T20:42:38.099808+0000","last_active":"2023-11-18T20:42:39.084520+0000","last_peered":"2023-11-18T20:42:39.084520+0000","last_clean":"2023-11-18T20:42:39.084520+0000","last_became_active":"2023-11-18T20:42:38.099512+0000","last_became_peered":"2023-11-18T20:42:38.099512+0000","last_unstale":"2023-11-18T20:42:39.084520+0000","last_undegraded":"2023-11-18T20:42:39.084520+0000","last_fullsized":"2023-11-18T20:42:39.084520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:19.165207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086125+0000","last_change":"2023-11-18T20:42:38.100064+0000","last_active":"2023-11-18T20:42:39.086125+0000","last_peered":"2023-11-18T20:42:39.086125+0000","last_clean":"2023-11-18T20:42:39.086125+0000","last_became_active":"2023-11-18T20:42:38.099778+0000","last_became_peered":"2023-11-18T20:42:38.099778+0000","last_unstale":"2023-11-18T20:42:39.086125+0000","last_undegraded":"2023-11-18T20:42:39.086125+0000","last_fullsized":"2023-11-18T20:42:39.086125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084743+0000","last_change":"2023-11-18T20:42:38.101526+0000","last_active":"2023-11-18T20:42:39.084743+0000","last_peered":"2023-11-18T20:42:39.084743+0000","last_clean":"2023-11-18T20:42:39.084743+0000","last_became_active":"2023-11-18T20:42:38.099171+0000","last_became_peered":"2023-11-18T20:42:38.099171+0000","last_unstale":"2023-11-18T20:42:39.084743+0000","last_undegraded":"2023-11-18T20:42:39.084743+0000","last_fullsized":"2023-11-18T20:42:39.084743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:26.508116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086361+0000","last_change":"2023-11-18T20:42:38.100225+0000","last_active":"2023-11-18T20:42:39.086361+0000","last_peered":"2023-11-18T20:42:39.086361+0000","last_clean":"2023-11-18T20:42:39.086361+0000","last_became_active":"2023-11-18T20:42:38.099966+0000","last_became_peered":"2023-11-18T20:42:38.099966+0000","last_unstale":"2023-11-18T20:42:39.086361+0000","last_undegraded":"2023-11-18T20:42:39.086361+0000","last_fullsized":"2023-11-18T20:42:39.086361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085054+0000","last_change":"2023-11-18T20:42:38.094313+0000","last_active":"2023-11-18T20:42:39.085054+0000","last_peered":"2023-11-18T20:42:39.085054+0000","last_clean":"2023-11-18T20:42:39.085054+0000","last_became_active":"2023-11-18T20:42:38.094005+0000","last_became_peered":"2023-11-18T20:42:38.094005+0000","last_unstale":"2023-11-18T20:42:39.085054+0000","last_undegraded":"2023-11-18T20:42:39.085054+0000","last_fullsized":"2023-11-18T20:42:39.085054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085575+0000","last_change":"2023-11-18T20:42:38.100511+0000","last_active":"2023-11-18T20:42:39.085575+0000","last_peered":"2023-11-18T20:42:39.085575+0000","last_clean":"2023-11-18T20:42:39.085575+0000","last_became_active":"2023-11-18T20:42:38.100119+0000","last_became_peered":"2023-11-18T20:42:38.100119+0000","last_unstale":"2023-11-18T20:42:39.085575+0000","last_undegraded":"2023-11-18T20:42:39.085575+0000","last_fullsized":"2023-11-18T20:42:39.085575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:12:16.083479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085818+0000","last_change":"2023-11-18T20:42:38.098009+0000","last_active":"2023-11-18T20:42:39.085818+0000","last_peered":"2023-11-18T20:42:39.085818+0000","last_clean":"2023-11-18T20:42:39.085818+0000","last_became_active":"2023-11-18T20:42:38.097728+0000","last_became_peered":"2023-11-18T20:42:38.097728+0000","last_unstale":"2023-11-18T20:42:39.085818+0000","last_undegraded":"2023-11-18T20:42:39.085818+0000","last_fullsized":"2023-11-18T20:42:39.085818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086654+0000","last_change":"2023-11-18T20:42:38.099195+0000","last_active":"2023-11-18T20:42:39.086654+0000","last_peered":"2023-11-18T20:42:39.086654+0000","last_clean":"2023-11-18T20:42:39.086654+0000","last_became_active":"2023-11-18T20:42:38.098918+0000","last_became_peered":"2023-11-18T20:42:38.098918+0000","last_unstale":"2023-11-18T20:42:39.086654+0000","last_undegraded":"2023-11-18T20:42:39.086654+0000","last_fullsized":"2023-11-18T20:42:39.086654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:48:04.295517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082460+0000","last_change":"2023-11-18T20:42:38.095631+0000","last_active":"2023-11-18T20:42:39.082460+0000","last_peered":"2023-11-18T20:42:39.082460+0000","last_clean":"2023-11-18T20:42:39.082460+0000","last_became_active":"2023-11-18T20:42:38.095319+0000","last_became_peered":"2023-11-18T20:42:38.095319+0000","last_unstale":"2023-11-18T20:42:39.082460+0000","last_undegraded":"2023-11-18T20:42:39.082460+0000","last_fullsized":"2023-11-18T20:42:39.082460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:13.814862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085792+0000","last_change":"2023-11-18T20:42:38.103076+0000","last_active":"2023-11-18T20:42:39.085792+0000","last_peered":"2023-11-18T20:42:39.085792+0000","last_clean":"2023-11-18T20:42:39.085792+0000","last_became_active":"2023-11-18T20:42:38.102829+0000","last_became_peered":"2023-11-18T20:42:38.102829+0000","last_unstale":"2023-11-18T20:42:39.085792+0000","last_undegraded":"2023-11-18T20:42:39.085792+0000","last_fullsized":"2023-11-18T20:42:39.085792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:14.531419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083227+0000","last_change":"2023-11-18T20:42:38.089642+0000","last_active":"2023-11-18T20:42:39.083227+0000","last_peered":"2023-11-18T20:42:39.083227+0000","last_clean":"2023-11-18T20:42:39.083227+0000","last_became_active":"2023-11-18T20:42:38.089367+0000","last_became_peered":"2023-11-18T20:42:38.089367+0000","last_unstale":"2023-11-18T20:42:39.083227+0000","last_undegraded":"2023-11-18T20:42:39.083227+0000","last_fullsized":"2023-11-18T20:42:39.083227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:28:13.121246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085545+0000","last_change":"2023-11-18T20:42:37.111681+0000","last_active":"2023-11-18T20:42:39.085545+0000","last_peered":"2023-11-18T20:42:39.085545+0000","last_clean":"2023-11-18T20:42:39.085545+0000","last_became_active":"2023-11-18T20:42:37.111280+0000","last_became_peered":"2023-11-18T20:42:37.111280+0000","last_unstale":"2023-11-18T20:42:39.085545+0000","last_undegraded":"2023-11-18T20:42:39.085545+0000","last_fullsized":"2023-11-18T20:42:39.085545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:31.712397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085932+0000","last_change":"2023-11-18T20:42:38.097798+0000","last_active":"2023-11-18T20:42:39.085932+0000","last_peered":"2023-11-18T20:42:39.085932+0000","last_clean":"2023-11-18T20:42:39.085932+0000","last_became_active":"2023-11-18T20:42:38.096529+0000","last_became_peered":"2023-11-18T20:42:38.096529+0000","last_unstale":"2023-11-18T20:42:39.085932+0000","last_undegraded":"2023-11-18T20:42:39.085932+0000","last_fullsized":"2023-11-18T20:42:39.085932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086235+0000","last_change":"2023-11-18T20:42:37.106910+0000","last_active":"2023-11-18T20:42:39.086235+0000","last_peered":"2023-11-18T20:42:39.086235+0000","last_clean":"2023-11-18T20:42:39.086235+0000","last_became_active":"2023-11-18T20:42:37.106625+0000","last_became_peered":"2023-11-18T20:42:37.106625+0000","last_unstale":"2023-11-18T20:42:39.086235+0000","last_undegraded":"2023-11-18T20:42:39.086235+0000","last_fullsized":"2023-11-18T20:42:39.086235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:43.316226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084601+0000","last_change":"2023-11-18T20:42:37.109273+0000","last_active":"2023-11-18T20:42:39.084601+0000","last_peered":"2023-11-18T20:42:39.084601+0000","last_clean":"2023-11-18T20:42:39.084601+0000","last_became_active":"2023-11-18T20:42:37.108948+0000","last_became_peered":"2023-11-18T20:42:37.108948+0000","last_unstale":"2023-11-18T20:42:39.084601+0000","last_undegraded":"2023-11-18T20:42:39.084601+0000","last_fullsized":"2023-11-18T20:42:39.084601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:12.810253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084564+0000","last_change":"2023-11-18T20:42:38.101286+0000","last_active":"2023-11-18T20:42:39.084564+0000","last_peered":"2023-11-18T20:42:39.084564+0000","last_clean":"2023-11-18T20:42:39.084564+0000","last_became_active":"2023-11-18T20:42:38.100974+0000","last_became_peered":"2023-11-18T20:42:38.100974+0000","last_unstale":"2023-11-18T20:42:39.084564+0000","last_undegraded":"2023-11-18T20:42:39.084564+0000","last_fullsized":"2023-11-18T20:42:39.084564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:14.531419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086311+0000","last_change":"2023-11-18T20:42:37.105690+0000","last_active":"2023-11-18T20:42:39.086311+0000","last_peered":"2023-11-18T20:42:39.086311+0000","last_clean":"2023-11-18T20:42:39.086311+0000","last_became_active":"2023-11-18T20:42:37.105410+0000","last_became_peered":"2023-11-18T20:42:37.105410+0000","last_unstale":"2023-11-18T20:42:39.086311+0000","last_undegraded":"2023-11-18T20:42:39.086311+0000","last_fullsized":"2023-11-18T20:42:39.086311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:14:29.503121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084618+0000","last_change":"2023-11-18T20:42:38.096986+0000","last_active":"2023-11-18T20:42:39.084618+0000","last_peered":"2023-11-18T20:42:39.084618+0000","last_clean":"2023-11-18T20:42:39.084618+0000","last_became_active":"2023-11-18T20:42:38.095964+0000","last_became_peered":"2023-11-18T20:42:38.095964+0000","last_unstale":"2023-11-18T20:42:39.084618+0000","last_undegraded":"2023-11-18T20:42:39.084618+0000","last_fullsized":"2023-11-18T20:42:39.084618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086366+0000","last_change":"2023-11-18T20:42:37.111419+0000","last_active":"2023-11-18T20:42:39.086366+0000","last_peered":"2023-11-18T20:42:39.086366+0000","last_clean":"2023-11-18T20:42:39.086366+0000","last_became_active":"2023-11-18T20:42:37.110877+0000","last_became_peered":"2023-11-18T20:42:37.110877+0000","last_unstale":"2023-11-18T20:42:39.086366+0000","last_undegraded":"2023-11-18T20:42:39.086366+0000","last_fullsized":"2023-11-18T20:42:39.086366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086126+0000","last_change":"2023-11-18T20:42:38.108594+0000","last_active":"2023-11-18T20:42:39.086126+0000","last_peered":"2023-11-18T20:42:39.086126+0000","last_clean":"2023-11-18T20:42:39.086126+0000","last_became_active":"2023-11-18T20:42:38.108325+0000","last_became_peered":"2023-11-18T20:42:38.108325+0000","last_unstale":"2023-11-18T20:42:39.086126+0000","last_undegraded":"2023-11-18T20:42:39.086126+0000","last_fullsized":"2023-11-18T20:42:39.086126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086218+0000","last_change":"2023-11-18T20:42:37.106865+0000","last_active":"2023-11-18T20:42:39.086218+0000","last_peered":"2023-11-18T20:42:39.086218+0000","last_clean":"2023-11-18T20:42:39.086218+0000","last_became_active":"2023-11-18T20:42:37.106561+0000","last_became_peered":"2023-11-18T20:42:37.106561+0000","last_unstale":"2023-11-18T20:42:39.086218+0000","last_undegraded":"2023-11-18T20:42:39.086218+0000","last_fullsized":"2023-11-18T20:42:39.086218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:25.503507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085723+0000","last_change":"2023-11-18T20:42:38.101545+0000","last_active":"2023-11-18T20:42:39.085723+0000","last_peered":"2023-11-18T20:42:39.085723+0000","last_clean":"2023-11-18T20:42:39.085723+0000","last_became_active":"2023-11-18T20:42:38.101247+0000","last_became_peered":"2023-11-18T20:42:38.101247+0000","last_unstale":"2023-11-18T20:42:39.085723+0000","last_undegraded":"2023-11-18T20:42:39.085723+0000","last_fullsized":"2023-11-18T20:42:39.085723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085146+0000","last_change":"2023-11-18T20:42:37.109345+0000","last_active":"2023-11-18T20:42:39.085146+0000","last_peered":"2023-11-18T20:42:39.085146+0000","last_clean":"2023-11-18T20:42:39.085146+0000","last_became_active":"2023-11-18T20:42:37.109081+0000","last_became_peered":"2023-11-18T20:42:37.109081+0000","last_unstale":"2023-11-18T20:42:39.085146+0000","last_undegraded":"2023-11-18T20:42:39.085146+0000","last_fullsized":"2023-11-18T20:42:39.085146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083745+0000","last_change":"2023-11-18T20:42:38.100125+0000","last_active":"2023-11-18T20:42:39.083745+0000","last_peered":"2023-11-18T20:42:39.083745+0000","last_clean":"2023-11-18T20:42:39.083745+0000","last_became_active":"2023-11-18T20:42:38.099646+0000","last_became_peered":"2023-11-18T20:42:38.099646+0000","last_unstale":"2023-11-18T20:42:39.083745+0000","last_undegraded":"2023-11-18T20:42:39.083745+0000","last_fullsized":"2023-11-18T20:42:39.083745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:14:30.507730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086459+0000","last_change":"2023-11-18T20:42:37.106986+0000","last_active":"2023-11-18T20:42:39.086459+0000","last_peered":"2023-11-18T20:42:39.086459+0000","last_clean":"2023-11-18T20:42:39.086459+0000","last_became_active":"2023-11-18T20:42:37.106707+0000","last_became_peered":"2023-11-18T20:42:37.106707+0000","last_unstale":"2023-11-18T20:42:39.086459+0000","last_undegraded":"2023-11-18T20:42:39.086459+0000","last_fullsized":"2023-11-18T20:42:39.086459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:31.712397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083280+0000","last_change":"2023-11-18T20:42:38.107769+0000","last_active":"2023-11-18T20:42:39.083280+0000","last_peered":"2023-11-18T20:42:39.083280+0000","last_clean":"2023-11-18T20:42:39.083280+0000","last_became_active":"2023-11-18T20:42:38.104253+0000","last_became_peered":"2023-11-18T20:42:38.104253+0000","last_unstale":"2023-11-18T20:42:39.083280+0000","last_undegraded":"2023-11-18T20:42:39.083280+0000","last_fullsized":"2023-11-18T20:42:39.083280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:45:28.196671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085912+0000","last_change":"2023-11-18T20:42:37.098519+0000","last_active":"2023-11-18T20:42:39.085912+0000","last_peered":"2023-11-18T20:42:39.085912+0000","last_clean":"2023-11-18T20:42:39.085912+0000","last_became_active":"2023-11-18T20:42:37.097972+0000","last_became_peered":"2023-11-18T20:42:37.097972+0000","last_unstale":"2023-11-18T20:42:39.085912+0000","last_undegraded":"2023-11-18T20:42:39.085912+0000","last_fullsized":"2023-11-18T20:42:39.085912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:12.810253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084892+0000","last_change":"2023-11-18T20:42:38.100733+0000","last_active":"2023-11-18T20:42:39.084892+0000","last_peered":"2023-11-18T20:42:39.084892+0000","last_clean":"2023-11-18T20:42:39.084892+0000","last_became_active":"2023-11-18T20:42:38.100390+0000","last_became_peered":"2023-11-18T20:42:38.100390+0000","last_unstale":"2023-11-18T20:42:39.084892+0000","last_undegraded":"2023-11-18T20:42:39.084892+0000","last_fullsized":"2023-11-18T20:42:39.084892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:32.717006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086635+0000","last_change":"2023-11-18T20:42:37.110943+0000","last_active":"2023-11-18T20:42:39.086635+0000","last_peered":"2023-11-18T20:42:39.086635+0000","last_clean":"2023-11-18T20:42:39.086635+0000","last_became_active":"2023-11-18T20:42:37.110657+0000","last_became_peered":"2023-11-18T20:42:37.110657+0000","last_unstale":"2023-11-18T20:42:39.086635+0000","last_undegraded":"2023-11-18T20:42:39.086635+0000","last_fullsized":"2023-11-18T20:42:39.086635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084234+0000","last_change":"2023-11-18T20:42:38.102821+0000","last_active":"2023-11-18T20:42:39.084234+0000","last_peered":"2023-11-18T20:42:39.084234+0000","last_clean":"2023-11-18T20:42:39.084234+0000","last_became_active":"2023-11-18T20:42:38.102491+0000","last_became_peered":"2023-11-18T20:42:38.102491+0000","last_unstale":"2023-11-18T20:42:39.084234+0000","last_undegraded":"2023-11-18T20:42:39.084234+0000","last_fullsized":"2023-11-18T20:42:39.084234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:17:48.228298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085793+0000","last_change":"2023-11-18T20:42:37.107187+0000","last_active":"2023-11-18T20:42:39.085793+0000","last_peered":"2023-11-18T20:42:39.085793+0000","last_clean":"2023-11-18T20:42:39.085793+0000","last_became_active":"2023-11-18T20:42:37.106904+0000","last_became_peered":"2023-11-18T20:42:37.106904+0000","last_unstale":"2023-11-18T20:42:39.085793+0000","last_undegraded":"2023-11-18T20:42:39.085793+0000","last_fullsized":"2023-11-18T20:42:39.085793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:17.850286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085803+0000","last_change":"2023-11-18T20:42:38.102486+0000","last_active":"2023-11-18T20:42:39.085803+0000","last_peered":"2023-11-18T20:42:39.085803+0000","last_clean":"2023-11-18T20:42:39.085803+0000","last_became_active":"2023-11-18T20:42:38.102239+0000","last_became_peered":"2023-11-18T20:42:38.102239+0000","last_unstale":"2023-11-18T20:42:39.085803+0000","last_undegraded":"2023-11-18T20:42:39.085803+0000","last_fullsized":"2023-11-18T20:42:39.085803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:26.508116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082180+0000","last_change":"2023-11-18T20:42:37.105948+0000","last_active":"2023-11-18T20:42:39.082180+0000","last_peered":"2023-11-18T20:42:39.082180+0000","last_clean":"2023-11-18T20:42:39.082180+0000","last_became_active":"2023-11-18T20:42:37.105280+0000","last_became_peered":"2023-11-18T20:42:37.105280+0000","last_unstale":"2023-11-18T20:42:39.082180+0000","last_undegraded":"2023-11-18T20:42:39.082180+0000","last_fullsized":"2023-11-18T20:42:39.082180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:55:27.947068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083496+0000","last_change":"2023-11-18T20:42:38.107573+0000","last_active":"2023-11-18T20:42:39.083496+0000","last_peered":"2023-11-18T20:42:39.083496+0000","last_clean":"2023-11-18T20:42:39.083496+0000","last_became_active":"2023-11-18T20:42:38.103361+0000","last_became_peered":"2023-11-18T20:42:38.103361+0000","last_unstale":"2023-11-18T20:42:39.083496+0000","last_undegraded":"2023-11-18T20:42:39.083496+0000","last_fullsized":"2023-11-18T20:42:39.083496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:04:55.965852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083454+0000","last_change":"2023-11-18T20:42:37.097968+0000","last_active":"2023-11-18T20:42:39.083454+0000","last_peered":"2023-11-18T20:42:39.083454+0000","last_clean":"2023-11-18T20:42:39.083454+0000","last_became_active":"2023-11-18T20:42:37.097659+0000","last_became_peered":"2023-11-18T20:42:37.097659+0000","last_unstale":"2023-11-18T20:42:39.083454+0000","last_undegraded":"2023-11-18T20:42:39.083454+0000","last_fullsized":"2023-11-18T20:42:39.083454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:08:51.955879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083672+0000","last_change":"2023-11-18T20:42:38.107457+0000","last_active":"2023-11-18T20:42:39.083672+0000","last_peered":"2023-11-18T20:42:39.083672+0000","last_clean":"2023-11-18T20:42:39.083672+0000","last_became_active":"2023-11-18T20:42:38.103260+0000","last_became_peered":"2023-11-18T20:42:38.103260+0000","last_unstale":"2023-11-18T20:42:39.083672+0000","last_undegraded":"2023-11-18T20:42:39.083672+0000","last_fullsized":"2023-11-18T20:42:39.083672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:41:14.693957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082379+0000","last_change":"2023-11-18T20:42:37.109692+0000","last_active":"2023-11-18T20:42:39.082379+0000","last_peered":"2023-11-18T20:42:39.082379+0000","last_clean":"2023-11-18T20:42:39.082379+0000","last_became_active":"2023-11-18T20:42:37.109328+0000","last_became_peered":"2023-11-18T20:42:37.109328+0000","last_unstale":"2023-11-18T20:42:39.082379+0000","last_undegraded":"2023-11-18T20:42:39.082379+0000","last_fullsized":"2023-11-18T20:42:39.082379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:08:51.955879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084464+0000","last_change":"2023-11-18T20:42:38.107442+0000","last_active":"2023-11-18T20:42:39.084464+0000","last_peered":"2023-11-18T20:42:39.084464+0000","last_clean":"2023-11-18T20:42:39.084464+0000","last_became_active":"2023-11-18T20:42:38.106939+0000","last_became_peered":"2023-11-18T20:42:38.106939+0000","last_unstale":"2023-11-18T20:42:39.084464+0000","last_undegraded":"2023-11-18T20:42:39.084464+0000","last_fullsized":"2023-11-18T20:42:39.084464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:52:14.406378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085180+0000","last_change":"2023-11-18T20:42:37.107734+0000","last_active":"2023-11-18T20:42:39.085180+0000","last_peered":"2023-11-18T20:42:39.085180+0000","last_clean":"2023-11-18T20:42:39.085180+0000","last_became_active":"2023-11-18T20:42:37.107466+0000","last_became_peered":"2023-11-18T20:42:37.107466+0000","last_unstale":"2023-11-18T20:42:39.085180+0000","last_undegraded":"2023-11-18T20:42:39.085180+0000","last_fullsized":"2023-11-18T20:42:39.085180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:13.526810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086537+0000","last_change":"2023-11-18T20:42:38.105663+0000","last_active":"2023-11-18T20:42:39.086537+0000","last_peered":"2023-11-18T20:42:39.086537+0000","last_clean":"2023-11-18T20:42:39.086537+0000","last_became_active":"2023-11-18T20:42:38.105269+0000","last_became_peered":"2023-11-18T20:42:38.105269+0000","last_unstale":"2023-11-18T20:42:39.086537+0000","last_undegraded":"2023-11-18T20:42:39.086537+0000","last_fullsized":"2023-11-18T20:42:39.086537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:04:55.965852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082362+0000","last_change":"2023-11-18T20:42:37.109021+0000","last_active":"2023-11-18T20:42:39.082362+0000","last_peered":"2023-11-18T20:42:39.082362+0000","last_clean":"2023-11-18T20:42:39.082362+0000","last_became_active":"2023-11-18T20:42:37.108256+0000","last_became_peered":"2023-11-18T20:42:37.108256+0000","last_unstale":"2023-11-18T20:42:39.082362+0000","last_undegraded":"2023-11-18T20:42:39.082362+0000","last_fullsized":"2023-11-18T20:42:39.082362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:17:02.258981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083639+0000","last_change":"2023-11-18T20:42:38.107956+0000","last_active":"2023-11-18T20:42:39.083639+0000","last_peered":"2023-11-18T20:42:39.083639+0000","last_clean":"2023-11-18T20:42:39.083639+0000","last_became_active":"2023-11-18T20:42:38.107488+0000","last_became_peered":"2023-11-18T20:42:38.107488+0000","last_unstale":"2023-11-18T20:42:39.083639+0000","last_undegraded":"2023-11-18T20:42:39.083639+0000","last_fullsized":"2023-11-18T20:42:39.083639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:51:04.970201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-18T20:42:38.097227+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:42:38.097227+0000","last_peered":"2023-11-18T20:42:38.097227+0000","last_clean":"2023-11-18T20:42:38.097227+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:42:38.097227+0000","last_undegraded":"2023-11-18T20:42:38.097227+0000","last_fullsized":"2023-11-18T20:42:38.097227+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086001+0000","last_change":"2023-11-18T20:42:37.110512+0000","last_active":"2023-11-18T20:42:39.086001+0000","last_peered":"2023-11-18T20:42:39.086001+0000","last_clean":"2023-11-18T20:42:39.086001+0000","last_became_active":"2023-11-18T20:42:37.110213+0000","last_became_peered":"2023-11-18T20:42:37.110213+0000","last_unstale":"2023-11-18T20:42:39.086001+0000","last_undegraded":"2023-11-18T20:42:39.086001+0000","last_fullsized":"2023-11-18T20:42:39.086001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:43.316226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084751+0000","last_change":"2023-11-18T20:42:38.105196+0000","last_active":"2023-11-18T20:42:39.084751+0000","last_peered":"2023-11-18T20:42:39.084751+0000","last_clean":"2023-11-18T20:42:39.084751+0000","last_became_active":"2023-11-18T20:42:38.104886+0000","last_became_peered":"2023-11-18T20:42:38.104886+0000","last_unstale":"2023-11-18T20:42:39.084751+0000","last_undegraded":"2023-11-18T20:42:39.084751+0000","last_fullsized":"2023-11-18T20:42:39.084751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084031+0000","last_change":"2023-11-18T20:42:37.108345+0000","last_active":"2023-11-18T20:42:39.084031+0000","last_peered":"2023-11-18T20:42:39.084031+0000","last_clean":"2023-11-18T20:42:39.084031+0000","last_became_active":"2023-11-18T20:42:37.107995+0000","last_became_peered":"2023-11-18T20:42:37.107995+0000","last_unstale":"2023-11-18T20:42:39.084031+0000","last_undegraded":"2023-11-18T20:42:39.084031+0000","last_fullsized":"2023-11-18T20:42:39.084031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:15.427234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084749+0000","last_change":"2023-11-18T20:42:38.088232+0000","last_active":"2023-11-18T20:42:39.084749+0000","last_peered":"2023-11-18T20:42:39.084749+0000","last_clean":"2023-11-18T20:42:39.084749+0000","last_became_active":"2023-11-18T20:42:38.087631+0000","last_became_peered":"2023-11-18T20:42:38.087631+0000","last_unstale":"2023-11-18T20:42:39.084749+0000","last_undegraded":"2023-11-18T20:42:39.084749+0000","last_fullsized":"2023-11-18T20:42:39.084749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:44.320835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082457+0000","last_change":"2023-11-18T20:42:37.107910+0000","last_active":"2023-11-18T20:42:39.082457+0000","last_peered":"2023-11-18T20:42:39.082457+0000","last_clean":"2023-11-18T20:42:39.082457+0000","last_became_active":"2023-11-18T20:42:37.106792+0000","last_became_peered":"2023-11-18T20:42:37.106792+0000","last_unstale":"2023-11-18T20:42:39.082457+0000","last_undegraded":"2023-11-18T20:42:39.082457+0000","last_fullsized":"2023-11-18T20:42:39.082457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:03:11.287757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084900+0000","last_change":"2023-11-18T20:42:38.090117+0000","last_active":"2023-11-18T20:42:39.084900+0000","last_peered":"2023-11-18T20:42:39.084900+0000","last_clean":"2023-11-18T20:42:39.084900+0000","last_became_active":"2023-11-18T20:42:38.089821+0000","last_became_peered":"2023-11-18T20:42:38.089821+0000","last_unstale":"2023-11-18T20:42:39.084900+0000","last_undegraded":"2023-11-18T20:42:39.084900+0000","last_fullsized":"2023-11-18T20:42:39.084900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086690+0000","last_change":"2023-11-18T20:42:37.111162+0000","last_active":"2023-11-18T20:42:39.086690+0000","last_peered":"2023-11-18T20:42:39.086690+0000","last_clean":"2023-11-18T20:42:39.086690+0000","last_became_active":"2023-11-18T20:42:37.110692+0000","last_became_peered":"2023-11-18T20:42:37.110692+0000","last_unstale":"2023-11-18T20:42:39.086690+0000","last_undegraded":"2023-11-18T20:42:39.086690+0000","last_fullsized":"2023-11-18T20:42:39.086690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084091+0000","last_change":"2023-11-18T20:42:38.091366+0000","last_active":"2023-11-18T20:42:39.084091+0000","last_peered":"2023-11-18T20:42:39.084091+0000","last_clean":"2023-11-18T20:42:39.084091+0000","last_became_active":"2023-11-18T20:42:38.089726+0000","last_became_peered":"2023-11-18T20:42:38.089726+0000","last_unstale":"2023-11-18T20:42:39.084091+0000","last_undegraded":"2023-11-18T20:42:39.084091+0000","last_fullsized":"2023-11-18T20:42:39.084091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:44.320835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085074+0000","last_change":"2023-11-18T20:42:37.103840+0000","last_active":"2023-11-18T20:42:39.085074+0000","last_peered":"2023-11-18T20:42:39.085074+0000","last_clean":"2023-11-18T20:42:39.085074+0000","last_became_active":"2023-11-18T20:42:37.103516+0000","last_became_peered":"2023-11-18T20:42:37.103516+0000","last_unstale":"2023-11-18T20:42:39.085074+0000","last_undegraded":"2023-11-18T20:42:39.085074+0000","last_fullsized":"2023-11-18T20:42:39.085074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086520+0000","last_change":"2023-11-18T20:42:38.100140+0000","last_active":"2023-11-18T20:42:39.086520+0000","last_peered":"2023-11-18T20:42:39.086520+0000","last_clean":"2023-11-18T20:42:39.086520+0000","last_became_active":"2023-11-18T20:42:38.099880+0000","last_became_peered":"2023-11-18T20:42:38.099880+0000","last_unstale":"2023-11-18T20:42:39.086520+0000","last_undegraded":"2023-11-18T20:42:39.086520+0000","last_fullsized":"2023-11-18T20:42:39.086520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:28.378102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083840+0000","last_change":"2023-11-18T20:42:37.103907+0000","last_active":"2023-11-18T20:42:39.083840+0000","last_peered":"2023-11-18T20:42:39.083840+0000","last_clean":"2023-11-18T20:42:39.083840+0000","last_became_active":"2023-11-18T20:42:37.103642+0000","last_became_peered":"2023-11-18T20:42:37.103642+0000","last_unstale":"2023-11-18T20:42:39.083840+0000","last_undegraded":"2023-11-18T20:42:39.083840+0000","last_fullsized":"2023-11-18T20:42:39.083840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:25.503507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084990+0000","last_change":"2023-11-18T20:42:38.090609+0000","last_active":"2023-11-18T20:42:39.084990+0000","last_peered":"2023-11-18T20:42:39.084990+0000","last_clean":"2023-11-18T20:42:39.084990+0000","last_became_active":"2023-11-18T20:42:38.090275+0000","last_became_peered":"2023-11-18T20:42:38.090275+0000","last_unstale":"2023-11-18T20:42:39.084990+0000","last_undegraded":"2023-11-18T20:42:39.084990+0000","last_fullsized":"2023-11-18T20:42:39.084990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085844+0000","last_change":"2023-11-18T20:42:37.110209+0000","last_active":"2023-11-18T20:42:39.085844+0000","last_peered":"2023-11-18T20:42:39.085844+0000","last_clean":"2023-11-18T20:42:39.085844+0000","last_became_active":"2023-11-18T20:42:37.109906+0000","last_became_peered":"2023-11-18T20:42:37.109906+0000","last_unstale":"2023-11-18T20:42:39.085844+0000","last_undegraded":"2023-11-18T20:42:39.085844+0000","last_fullsized":"2023-11-18T20:42:39.085844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:17.850286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084866+0000","last_change":"2023-11-18T20:42:38.104224+0000","last_active":"2023-11-18T20:42:39.084866+0000","last_peered":"2023-11-18T20:42:39.084866+0000","last_clean":"2023-11-18T20:42:39.084866+0000","last_became_active":"2023-11-18T20:42:38.103950+0000","last_became_peered":"2023-11-18T20:42:38.103950+0000","last_unstale":"2023-11-18T20:42:39.084866+0000","last_undegraded":"2023-11-18T20:42:39.084866+0000","last_fullsized":"2023-11-18T20:42:39.084866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086459+0000","last_change":"2023-11-18T20:42:37.102910+0000","last_active":"2023-11-18T20:42:39.086459+0000","last_peered":"2023-11-18T20:42:39.086459+0000","last_clean":"2023-11-18T20:42:39.086459+0000","last_became_active":"2023-11-18T20:42:37.102612+0000","last_became_peered":"2023-11-18T20:42:37.102612+0000","last_unstale":"2023-11-18T20:42:39.086459+0000","last_undegraded":"2023-11-18T20:42:39.086459+0000","last_fullsized":"2023-11-18T20:42:39.086459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085442+0000","last_change":"2023-11-18T20:42:38.094021+0000","last_active":"2023-11-18T20:42:39.085442+0000","last_peered":"2023-11-18T20:42:39.085442+0000","last_clean":"2023-11-18T20:42:39.085442+0000","last_became_active":"2023-11-18T20:42:38.093125+0000","last_became_peered":"2023-11-18T20:42:38.093125+0000","last_unstale":"2023-11-18T20:42:39.085442+0000","last_undegraded":"2023-11-18T20:42:39.085442+0000","last_fullsized":"2023-11-18T20:42:39.085442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:19.165207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083934+0000","last_change":"2023-11-18T20:42:37.102964+0000","last_active":"2023-11-18T20:42:39.083934+0000","last_peered":"2023-11-18T20:42:39.083934+0000","last_clean":"2023-11-18T20:42:39.083934+0000","last_became_active":"2023-11-18T20:42:37.102696+0000","last_became_peered":"2023-11-18T20:42:37.102696+0000","last_unstale":"2023-11-18T20:42:39.083934+0000","last_undegraded":"2023-11-18T20:42:39.083934+0000","last_fullsized":"2023-11-18T20:42:39.083934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:51.367829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084007+0000","last_change":"2023-11-18T20:42:38.089921+0000","last_active":"2023-11-18T20:42:39.084007+0000","last_peered":"2023-11-18T20:42:39.084007+0000","last_clean":"2023-11-18T20:42:39.084007+0000","last_became_active":"2023-11-18T20:42:38.089451+0000","last_became_peered":"2023-11-18T20:42:38.089451+0000","last_unstale":"2023-11-18T20:42:39.084007+0000","last_undegraded":"2023-11-18T20:42:39.084007+0000","last_fullsized":"2023-11-18T20:42:39.084007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:03:26.636590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084840+0000","last_change":"2023-11-18T20:42:37.110785+0000","last_active":"2023-11-18T20:42:39.084840+0000","last_peered":"2023-11-18T20:42:39.084840+0000","last_clean":"2023-11-18T20:42:39.084840+0000","last_became_active":"2023-11-18T20:42:37.110523+0000","last_became_peered":"2023-11-18T20:42:37.110523+0000","last_unstale":"2023-11-18T20:42:39.084840+0000","last_undegraded":"2023-11-18T20:42:39.084840+0000","last_fullsized":"2023-11-18T20:42:39.084840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:43.316226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085653+0000","last_change":"2023-11-18T20:42:38.097248+0000","last_active":"2023-11-18T20:42:39.085653+0000","last_peered":"2023-11-18T20:42:39.085653+0000","last_clean":"2023-11-18T20:42:39.085653+0000","last_became_active":"2023-11-18T20:42:38.096931+0000","last_became_peered":"2023-11-18T20:42:38.096931+0000","last_unstale":"2023-11-18T20:42:39.085653+0000","last_undegraded":"2023-11-18T20:42:39.085653+0000","last_fullsized":"2023-11-18T20:42:39.085653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:35:56.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085556+0000","last_change":"2023-11-18T20:42:37.106332+0000","last_active":"2023-11-18T20:42:39.085556+0000","last_peered":"2023-11-18T20:42:39.085556+0000","last_clean":"2023-11-18T20:42:39.085556+0000","last_became_active":"2023-11-18T20:42:37.106048+0000","last_became_peered":"2023-11-18T20:42:37.106048+0000","last_unstale":"2023-11-18T20:42:39.085556+0000","last_undegraded":"2023-11-18T20:42:39.085556+0000","last_fullsized":"2023-11-18T20:42:39.085556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:31.712397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085454+0000","last_change":"2023-11-18T20:42:38.100884+0000","last_active":"2023-11-18T20:42:39.085454+0000","last_peered":"2023-11-18T20:42:39.085454+0000","last_clean":"2023-11-18T20:42:39.085454+0000","last_became_active":"2023-11-18T20:42:38.100591+0000","last_became_peered":"2023-11-18T20:42:38.100591+0000","last_unstale":"2023-11-18T20:42:39.085454+0000","last_undegraded":"2023-11-18T20:42:39.085454+0000","last_fullsized":"2023-11-18T20:42:39.085454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084262+0000","last_change":"2023-11-18T20:42:37.099362+0000","last_active":"2023-11-18T20:42:39.084262+0000","last_peered":"2023-11-18T20:42:39.084262+0000","last_clean":"2023-11-18T20:42:39.084262+0000","last_became_active":"2023-11-18T20:42:37.099107+0000","last_became_peered":"2023-11-18T20:42:37.099107+0000","last_unstale":"2023-11-18T20:42:39.084262+0000","last_undegraded":"2023-11-18T20:42:39.084262+0000","last_fullsized":"2023-11-18T20:42:39.084262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086177+0000","last_change":"2023-11-18T20:42:38.101488+0000","last_active":"2023-11-18T20:42:39.086177+0000","last_peered":"2023-11-18T20:42:39.086177+0000","last_clean":"2023-11-18T20:42:39.086177+0000","last_became_active":"2023-11-18T20:42:38.101205+0000","last_became_peered":"2023-11-18T20:42:38.101205+0000","last_unstale":"2023-11-18T20:42:39.086177+0000","last_undegraded":"2023-11-18T20:42:39.086177+0000","last_fullsized":"2023-11-18T20:42:39.086177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086311+0000","last_change":"2023-11-18T20:42:38.106746+0000","last_active":"2023-11-18T20:42:39.086311+0000","last_peered":"2023-11-18T20:42:39.086311+0000","last_clean":"2023-11-18T20:42:39.086311+0000","last_became_active":"2023-11-18T20:42:38.101774+0000","last_became_peered":"2023-11-18T20:42:38.101774+0000","last_unstale":"2023-11-18T20:42:39.086311+0000","last_undegraded":"2023-11-18T20:42:39.086311+0000","last_fullsized":"2023-11-18T20:42:39.086311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083126+0000","last_change":"2023-11-18T20:42:37.110955+0000","last_active":"2023-11-18T20:42:39.083126+0000","last_peered":"2023-11-18T20:42:39.083126+0000","last_clean":"2023-11-18T20:42:39.083126+0000","last_became_active":"2023-11-18T20:42:37.110681+0000","last_became_peered":"2023-11-18T20:42:37.110681+0000","last_unstale":"2023-11-18T20:42:39.083126+0000","last_undegraded":"2023-11-18T20:42:39.083126+0000","last_fullsized":"2023-11-18T20:42:39.083126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:28:12.116637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086444+0000","last_change":"2023-11-18T20:42:38.105146+0000","last_active":"2023-11-18T20:42:39.086444+0000","last_peered":"2023-11-18T20:42:39.086444+0000","last_clean":"2023-11-18T20:42:39.086444+0000","last_became_active":"2023-11-18T20:42:38.101793+0000","last_became_peered":"2023-11-18T20:42:38.101793+0000","last_unstale":"2023-11-18T20:42:39.086444+0000","last_undegraded":"2023-11-18T20:42:39.086444+0000","last_fullsized":"2023-11-18T20:42:39.086444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:45:28.196671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083739+0000","last_change":"2023-11-18T20:42:37.102836+0000","last_active":"2023-11-18T20:42:39.083739+0000","last_peered":"2023-11-18T20:42:39.083739+0000","last_clean":"2023-11-18T20:42:39.083739+0000","last_became_active":"2023-11-18T20:42:37.102550+0000","last_became_peered":"2023-11-18T20:42:37.102550+0000","last_unstale":"2023-11-18T20:42:39.083739+0000","last_undegraded":"2023-11-18T20:42:39.083739+0000","last_fullsized":"2023-11-18T20:42:39.083739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085598+0000","last_change":"2023-11-18T20:42:38.102190+0000","last_active":"2023-11-18T20:42:39.085598+0000","last_peered":"2023-11-18T20:42:39.085598+0000","last_clean":"2023-11-18T20:42:39.085598+0000","last_became_active":"2023-11-18T20:42:38.101883+0000","last_became_peered":"2023-11-18T20:42:38.101883+0000","last_unstale":"2023-11-18T20:42:39.085598+0000","last_undegraded":"2023-11-18T20:42:39.085598+0000","last_fullsized":"2023-11-18T20:42:39.085598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:14.531419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085117+0000","last_change":"2023-11-18T20:42:37.108374+0000","last_active":"2023-11-18T20:42:39.085117+0000","last_peered":"2023-11-18T20:42:39.085117+0000","last_clean":"2023-11-18T20:42:39.085117+0000","last_became_active":"2023-11-18T20:42:37.108079+0000","last_became_peered":"2023-11-18T20:42:37.108079+0000","last_unstale":"2023-11-18T20:42:39.085117+0000","last_undegraded":"2023-11-18T20:42:39.085117+0000","last_fullsized":"2023-11-18T20:42:39.085117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:03:25.631982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085115+0000","last_change":"2023-11-18T20:42:38.095628+0000","last_active":"2023-11-18T20:42:39.085115+0000","last_peered":"2023-11-18T20:42:39.085115+0000","last_clean":"2023-11-18T20:42:39.085115+0000","last_became_active":"2023-11-18T20:42:38.095315+0000","last_became_peered":"2023-11-18T20:42:38.095315+0000","last_unstale":"2023-11-18T20:42:39.085115+0000","last_undegraded":"2023-11-18T20:42:39.085115+0000","last_fullsized":"2023-11-18T20:42:39.085115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085811+0000","last_change":"2023-11-18T20:42:37.101566+0000","last_active":"2023-11-18T20:42:39.085811+0000","last_peered":"2023-11-18T20:42:39.085811+0000","last_clean":"2023-11-18T20:42:39.085811+0000","last_became_active":"2023-11-18T20:42:37.101286+0000","last_became_peered":"2023-11-18T20:42:37.101286+0000","last_unstale":"2023-11-18T20:42:39.085811+0000","last_undegraded":"2023-11-18T20:42:39.085811+0000","last_fullsized":"2023-11-18T20:42:39.085811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:14:29.503121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":660231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":69407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":69281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.859}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.464}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":69441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":659668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:42:41.469 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-18T20:42:41.469 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-18T20:42:41.470 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-18T20:42:41.470 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-18T20:42:41.470 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-18T20:42:41.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.608+0000 7fc343f89640 1 Processor -- start 2023-11-18T20:42:41.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.608+0000 7fc343f89640 1 -- start start 2023-11-18T20:42:41.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.609+0000 7fc343f89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c0d3900 0x7fc33c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.609+0000 7fc343f89640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c0b2b10 con 0x7fc33c0d2910 2023-11-18T20:42:41.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.609+0000 7fc343f89640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c0d1590 con 0x7fc33c15d330 2023-11-18T20:42:41.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.609+0000 7fc343f89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c0d1870 con 0x7fc33c0d3900 2023-11-18T20:42:41.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.609+0000 7fc341524640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc33c0d2910 0x7fc33c0d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39074/0 (socket says 172.21.15.60:39074) 2023-11-18T20:42:41.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.609+0000 7fc341524640 1 -- 172.21.15.60:0/652244253 learned_addr learned my addr 172.21.15.60:0/652244253 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:41.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.609+0000 7fc342526640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c0d3900 0x7fc33c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174087180 0 0) 0x7fc33c0b2b10 con 0x7fc33c0d2910 2023-11-18T20:42:41.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc328003520 con 0x7fc33c0d2910 2023-11-18T20:42:41.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425928024 0 0) 0x7fc33c0d1590 con 0x7fc33c15d330 2023-11-18T20:42:41.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc33c0b2b10 con 0x7fc33c15d330 2023-11-18T20:42:41.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2542147439 0 0) 0x7fc328003520 con 0x7fc33c0d2910 2023-11-18T20:42:41.622 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c0d3900 msgr2=0x7fc33c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:41.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c0d3900 0x7fc33c0b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 >> v1:172.21.15.100:6789/0 conn(0x7fc33c15d330 legacy=0x7fc33c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c0d1b50 con 0x7fc33c0d2910 2023-11-18T20:42:41.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fc32c001670 con 0x7fc33c0d2910 2023-11-18T20:42:41.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc342526640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c0d3900 0x7fc33c0b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:41.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 >> v1:172.21.15.60:6789/0 conn(0x7fc33c0d2910 legacy=0x7fc33c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c0d3900 0x7fc328003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc328004070 0x7fc3280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc328004d00 0x7fc328005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c0d1870 con 0x7fc328004d00 2023-11-18T20:42:41.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32800bac0 con 0x7fc328004070 2023-11-18T20:42:41.624 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32800bc80 con 0x7fc33c0d3900 2023-11-18T20:42:41.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc342526640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc328004d00 0x7fc328005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc341d25640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c0d3900 0x7fc328003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc341524640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc328004070 0x7fc3280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc342526640 1 -- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c0d3900 msgr2=0x7fc328003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:41.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc342526640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c0d3900 0x7fc328003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc342526640 1 -- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc328004070 msgr2=0x7fc3280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:41.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc342526640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc328004070 0x7fc3280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.610+0000 7fc342526640 1 -- 172.21.15.60:0/652244253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338009400 con 0x7fc328004d00 2023-11-18T20:42:41.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.611+0000 7fc342526640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc328004d00 0x7fc328005180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc338009c70 tx=0x7fc3280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edeb8d3fcf625aaf server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:41.626 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.611+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33802c910 con 0x7fc328004d00 2023-11-18T20:42:41.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.611+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33802ca70 con 0x7fc328004d00 2023-11-18T20:42:41.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.611+0000 7fc340d23640 1 -- 172.21.15.60:0/652244253 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33802ce20 con 0x7fc328004d00 2023-11-18T20:42:41.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.612+0000 7fc343f89640 1 -- 172.21.15.60:0/652244253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc328004d00 msgr2=0x7fc328005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.612+0000 7fc343f89640 1 --2- 172.21.15.60:0/652244253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc328004d00 0x7fc328005180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc338009c70 tx=0x7fc3280057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.612+0000 7fc342526640 1 -- 172.21.15.60:0/652244253 reap_dead start 2023-11-18T20:42:41.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.612+0000 7fc343f89640 1 -- 172.21.15.60:0/652244253 shutdown_connections 2023-11-18T20:42:41.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.612+0000 7fc343f89640 1 -- 172.21.15.60:0/652244253 >> 172.21.15.60:0/652244253 conn(0x7fc33c072020 msgr2=0x7fc33c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:41.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.612+0000 7fc343f89640 1 -- 172.21.15.60:0/652244253 shutdown_connections 2023-11-18T20:42:41.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.612+0000 7fc343f89640 1 -- 172.21.15.60:0/652244253 wait complete. 2023-11-18T20:42:41.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.613+0000 7fc343f89640 1 Processor -- start 2023-11-18T20:42:41.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.613+0000 7fc343f89640 1 -- start start 2023-11-18T20:42:41.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc343f89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc328004d00 0x7fc33c155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc343f89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33c0c95c0 0x7fc33c145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc343f89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c1459c0 0x7fc33c1433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc343f89640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c154e40 con 0x7fc33c0c95c0 2023-11-18T20:42:41.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc343f89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c145e40 con 0x7fc328004d00 2023-11-18T20:42:41.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc343f89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c1557a0 con 0x7fc33c1459c0 2023-11-18T20:42:41.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc341524640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33c0c95c0 0x7fc33c145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc341524640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33c0c95c0 0x7fc33c145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41016/0 (socket says 172.21.15.60:41016) 2023-11-18T20:42:41.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc341524640 1 -- 172.21.15.60:0/2442710835 learned_addr learned my addr 172.21.15.60:0/2442710835 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:41.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc341d25640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc328004d00 0x7fc33c155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc342526640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c1459c0 0x7fc33c1433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc341524640 1 -- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c1459c0 msgr2=0x7fc33c1433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc341524640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c1459c0 0x7fc33c1433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc341524640 1 -- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc328004d00 msgr2=0x7fc33c155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc341524640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc328004d00 0x7fc33c155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc341524640 1 -- 172.21.15.60:0/2442710835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c1541c0 con 0x7fc33c0c95c0 2023-11-18T20:42:41.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc341524640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33c0c95c0 0x7fc33c145480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc32c007310 tx=0x7fc32c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:41.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.614+0000 7fc3327fc640 1 -- 172.21.15.60:0/2442710835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00f8e0 con 0x7fc33c0c95c0 2023-11-18T20:42:41.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.615+0000 7fc3327fc640 1 -- 172.21.15.60:0/2442710835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c00fa60 con 0x7fc33c0c95c0 2023-11-18T20:42:41.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.615+0000 7fc3327fc640 1 -- 172.21.15.60:0/2442710835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00fe10 con 0x7fc33c0c95c0 2023-11-18T20:42:41.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.615+0000 7fc343f89640 1 -- 172.21.15.60:0/2442710835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33c1544a0 con 0x7fc33c0c95c0 2023-11-18T20:42:41.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.615+0000 7fc343f89640 1 -- 172.21.15.60:0/2442710835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33c154a60 con 0x7fc33c0c95c0 2023-11-18T20:42:41.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.616+0000 7fc343f89640 1 -- 172.21.15.60:0/2442710835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33c154f80 con 0x7fc33c0c95c0 2023-11-18T20:42:41.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.616+0000 7fc3327fc640 1 -- 172.21.15.60:0/2442710835 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc32c01d460 con 0x7fc33c0c95c0 2023-11-18T20:42:41.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.616+0000 7fc3327fc640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc32006ae70 0x7fc32006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.617+0000 7fc3327fc640 1 -- 172.21.15.60:0/2442710835 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc32c089840 con 0x7fc33c0c95c0 2023-11-18T20:42:41.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.617+0000 7fc341d25640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc32006ae70 0x7fc32006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.617+0000 7fc341d25640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc32006ae70 0x7fc32006d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc334003f30 tx=0x7fc334009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:41.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.621+0000 7fc3327fc640 1 -- 172.21.15.60:0/2442710835 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc33c154f80 con 0x7fc33c0c95c0 2023-11-18T20:42:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.843+0000 7fc343f89640 1 -- 172.21.15.60:0/2442710835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc33c154f80 con 0x7fc33c0c95c0 2023-11-18T20:42:41.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.846+0000 7fc3327fc640 1 -- 172.21.15.60:0/2442710835 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171035 (secure 0 0 0) 0x7fc33c154f80 con 0x7fc33c0c95c0 2023-11-18T20:42:41.850 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:41.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 -- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc32006ae70 msgr2=0x7fc32006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc32006ae70 0x7fc32006d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc334003f30 tx=0x7fc334009040 comp rx=0 tx=0).stop 2023-11-18T20:42:41.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 -- 172.21.15.60:0/2442710835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33c0c95c0 msgr2=0x7fc33c145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33c0c95c0 0x7fc33c145480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc32c007310 tx=0x7fc32c0077d0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 -- 172.21.15.60:0/2442710835 shutdown_connections 2023-11-18T20:42:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc32006ae70 0x7fc32006d330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc33c1459c0 0x7fc33c1433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33c0c95c0 0x7fc33c145480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 --2- 172.21.15.60:0/2442710835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc328004d00 0x7fc33c155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 -- 172.21.15.60:0/2442710835 >> 172.21.15.60:0/2442710835 conn(0x7fc33c072020 msgr2=0x7fc33c15f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 -- 172.21.15.60:0/2442710835 shutdown_connections 2023-11-18T20:42:41.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.849+0000 7fc343f89640 1 -- 172.21.15.60:0/2442710835 wait complete. 2023-11-18T20:42:41.861 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":910625938},{"type":"v1","addr":"172.21.15.100:6833","nonce":910625938}]},"active_addr":"172.21.15.100:6833/910625938","active_change":"2023-11-18T20:42:17.421136+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1310129391}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":4285981536}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1092135094}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":224785920}]}]} 2023-11-18T20:42:41.864 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-18T20:42:41.864 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-18T20:42:41.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:41.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.969+0000 7fd3dc1b0640 1 Processor -- start 2023-11-18T20:42:41.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.969+0000 7fd3dc1b0640 1 -- start start 2023-11-18T20:42:41.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3dc1b0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd3d406ea50 0x7fd3d406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3dc1b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3d40d0ed0 0x7fd3d40d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3dc1b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d4161d20 con 0x7fd3d4160fb0 2023-11-18T20:42:41.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3dc1b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d406f370 con 0x7fd3d40d0ed0 2023-11-18T20:42:41.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3dc1b0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d415e760 con 0x7fd3d406ea50 2023-11-18T20:42:41.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3da74d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3d40d0ed0 0x7fd3d40d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3da74d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3d40d0ed0 0x7fd3d40d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41018/0 (socket says 172.21.15.60:41018) 2023-11-18T20:42:41.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3da74d640 1 -- 172.21.15.60:0/1087076360 learned_addr learned my addr 172.21.15.60:0/1087076360 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3d9f4c640 1 --2- 172.21.15.60:0/1087076360 >> v2:172.21.15.100:3300/0 conn(0x7fd3d406ea50 0x7fd3d406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3da74d640 1 -- 172.21.15.60:0/1087076360 >> v2:172.21.15.100:3300/0 conn(0x7fd3d406ea50 msgr2=0x7fd3d406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3da74d640 1 --2- 172.21.15.60:0/1087076360 >> v2:172.21.15.100:3300/0 conn(0x7fd3d406ea50 0x7fd3d406ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.970+0000 7fd3da74d640 1 -- 172.21.15.60:0/1087076360 >> v1:172.21.15.100:6789/0 conn(0x7fd3d4160fb0 legacy=0x7fd3d4161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.971+0000 7fd3da74d640 1 -- 172.21.15.60:0/1087076360 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d415d9a0 con 0x7fd3d40d0ed0 2023-11-18T20:42:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.971+0000 7fd3da74d640 1 --2- 172.21.15.60:0/1087076360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40d0ed0 0x7fd3d40d1320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd3d406f680 tx=0x7fd3cc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3401418bace80d77 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.971+0000 7fd3d8f4a640 1 -- 172.21.15.60:0/1087076360 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc0317e0 con 0x7fd3d40d0ed0 2023-11-18T20:42:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.971+0000 7fd3d8f4a640 1 -- 172.21.15.60:0/1087076360 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc031940 con 0x7fd3d40d0ed0 2023-11-18T20:42:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.971+0000 7fd3d8f4a640 1 -- 172.21.15.60:0/1087076360 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc031cd0 con 0x7fd3d40d0ed0 2023-11-18T20:42:41.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.972+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/1087076360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40d0ed0 msgr2=0x7fd3d40d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.972+0000 7fd3dc1b0640 1 --2- 172.21.15.60:0/1087076360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40d0ed0 0x7fd3d40d1320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd3d406f680 tx=0x7fd3cc02b880 comp rx=0 tx=0).stop 2023-11-18T20:42:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.972+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/1087076360 shutdown_connections 2023-11-18T20:42:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.972+0000 7fd3dc1b0640 1 --2- 172.21.15.60:0/1087076360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d40d0ed0 0x7fd3d40d1320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.972+0000 7fd3dc1b0640 1 --2- 172.21.15.60:0/1087076360 >> v2:172.21.15.100:3300/0 conn(0x7fd3d406ea50 0x7fd3d406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.972+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/1087076360 >> 172.21.15.60:0/1087076360 conn(0x7fd3d406a6b0 msgr2=0x7fd3d406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.972+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/1087076360 shutdown_connections 2023-11-18T20:42:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.972+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/1087076360 wait complete. 2023-11-18T20:42:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.973+0000 7fd3dc1b0640 1 Processor -- start 2023-11-18T20:42:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.973+0000 7fd3dc1b0640 1 -- start start 2023-11-18T20:42:41.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3dc1b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3d406ea50 0x7fd3d413e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3dc1b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3d40d0ed0 0x7fd3d413e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3dc1b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d4160fb0 0x7fd3d41567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3dc1b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d41589e0 con 0x7fd3d4160fb0 2023-11-18T20:42:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3dc1b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d413e780 con 0x7fd3d406ea50 2023-11-18T20:42:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3dc1b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4158040 con 0x7fd3d40d0ed0 2023-11-18T20:42:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3da74d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d4160fb0 0x7fd3d41567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3da74d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d4160fb0 0x7fd3d41567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41030/0 (socket says 172.21.15.60:41030) 2023-11-18T20:42:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3da74d640 1 -- 172.21.15.60:0/3634295476 learned_addr learned my addr 172.21.15.60:0/3634295476 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3d974b640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3d40d0ed0 0x7fd3d413e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3d9f4c640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3d406ea50 0x7fd3d413e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.974+0000 7fd3da74d640 1 -- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3d40d0ed0 msgr2=0x7fd3d413e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3da74d640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3d40d0ed0 0x7fd3d413e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3da74d640 1 -- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3d406ea50 msgr2=0x7fd3d413e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3da74d640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3d406ea50 0x7fd3d413e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3da74d640 1 -- 172.21.15.60:0/3634295476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0094d0 con 0x7fd3d4160fb0 2023-11-18T20:42:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3d9f4c640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3d406ea50 0x7fd3d413e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3da74d640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d4160fb0 0x7fd3d41567f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0090b0 tx=0x7fd3cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3d27fc640 1 -- 172.21.15.60:0/3634295476 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc032880 con 0x7fd3d4160fb0 2023-11-18T20:42:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3d27fc640 1 -- 172.21.15.60:0/3634295476 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc032a00 con 0x7fd3d4160fb0 2023-11-18T20:42:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3d27fc640 1 -- 172.21.15.60:0/3634295476 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc032db0 con 0x7fd3d4160fb0 2023-11-18T20:42:41.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/3634295476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d4158320 con 0x7fd3d4160fb0 2023-11-18T20:42:41.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.975+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/3634295476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d4156d30 con 0x7fd3d4160fb0 2023-11-18T20:42:41.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.976+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/3634295476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d4158b20 con 0x7fd3d4160fb0 2023-11-18T20:42:41.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.977+0000 7fd3d27fc640 1 -- 172.21.15.60:0/3634295476 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd3cc04e450 con 0x7fd3d4160fb0 2023-11-18T20:42:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.977+0000 7fd3d27fc640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd3c006ae70 0x7fd3c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.977+0000 7fd3d27fc640 1 -- 172.21.15.60:0/3634295476 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3cc0ab610 con 0x7fd3d4160fb0 2023-11-18T20:42:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.977+0000 7fd3d9f4c640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd3c006ae70 0x7fd3c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.978+0000 7fd3d9f4c640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd3c006ae70 0x7fd3c006d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009fd0 tx=0x7fd3c400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:41.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:41.981+0000 7fd3d27fc640 1 -- 172.21.15.60:0/3634295476 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3cc076750 con 0x7fd3d4160fb0 2023-11-18T20:42:42.092 INFO:tasks.ceph.mds.b.smithi100.stderr:2023-11-18T20:42:42.091+0000 7f7b09c15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-18T20:42:42.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.159+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/3634295476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3d4069400 con 0x7fd3d4160fb0 2023-11-18T20:42:42.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.160+0000 7fd3d27fc640 1 -- 172.21.15.60:0/3634295476 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fd3cc079d70 con 0x7fd3d4160fb0 2023-11-18T20:42:42.161 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:42.162 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:42.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd3c006ae70 msgr2=0x7fd3c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd3c006ae70 0x7fd3c006d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009fd0 tx=0x7fd3c400b040 comp rx=0 tx=0).stop 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/3634295476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d4160fb0 msgr2=0x7fd3d41567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d4160fb0 0x7fd3d41567f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0090b0 tx=0x7fd3cc0046c0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/3634295476 shutdown_connections 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd3c006ae70 0x7fd3c006d330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3d4160fb0 0x7fd3d41567f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3d40d0ed0 0x7fd3d413e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 --2- 172.21.15.60:0/3634295476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3d406ea50 0x7fd3d413e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/3634295476 >> 172.21.15.60:0/3634295476 conn(0x7fd3d406a6b0 msgr2=0x7fd3d406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/3634295476 shutdown_connections 2023-11-18T20:42:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.162+0000 7fd3dc1b0640 1 -- 172.21.15.60:0/3634295476 wait complete. 2023-11-18T20:42:42.174 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-18T20:42:42.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:42.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.281+0000 7f2b7810f640 1 Processor -- start 2023-11-18T20:42:42.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.281+0000 7f2b7810f640 1 -- start start 2023-11-18T20:42:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b7810f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b7015d2d0 0x7f2b700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b7810f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2b700ce650 0x7f2b700d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b7810f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b701614e0 con 0x7f2b70160f70 2023-11-18T20:42:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b7810f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b701617c0 con 0x7f2b700ce650 2023-11-18T20:42:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b7810f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70161aa0 con 0x7f2b7015d2d0 2023-11-18T20:42:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b766ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2b700ce650 0x7f2b700d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b766ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2b700ce650 0x7f2b700d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41038/0 (socket says 172.21.15.60:41038) 2023-11-18T20:42:42.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b766ac640 1 -- 172.21.15.60:0/1870757845 learned_addr learned my addr 172.21.15.60:0/1870757845 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b75eab640 1 --2- 172.21.15.60:0/1870757845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b7015d2d0 0x7f2b700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b766ac640 1 -- 172.21.15.60:0/1870757845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b7015d2d0 msgr2=0x7f2b700b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b766ac640 1 --2- 172.21.15.60:0/1870757845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b7015d2d0 0x7f2b700b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b766ac640 1 -- 172.21.15.60:0/1870757845 >> v1:172.21.15.100:6789/0 conn(0x7f2b70160f70 legacy=0x7f2b700cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.282+0000 7f2b766ac640 1 -- 172.21.15.60:0/1870757845 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70161d80 con 0x7f2b700ce650 2023-11-18T20:42:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.283+0000 7f2b766ac640 1 --2- 172.21.15.60:0/1870757845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b700ce650 0x7f2b700d1af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009e50 tx=0x7f2b6002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13573f2ebc8da614 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.283+0000 7f2b74ea9640 1 -- 172.21.15.60:0/1870757845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6002ca70 con 0x7f2b700ce650 2023-11-18T20:42:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.283+0000 7f2b74ea9640 1 -- 172.21.15.60:0/1870757845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6002cbd0 con 0x7f2b700ce650 2023-11-18T20:42:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.283+0000 7f2b74ea9640 1 -- 172.21.15.60:0/1870757845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b600343f0 con 0x7f2b700ce650 2023-11-18T20:42:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.284+0000 7f2b7810f640 1 -- 172.21.15.60:0/1870757845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b700ce650 msgr2=0x7f2b700d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.284+0000 7f2b7810f640 1 --2- 172.21.15.60:0/1870757845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b700ce650 0x7f2b700d1af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009e50 tx=0x7f2b6002bed0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.284+0000 7f2b7810f640 1 -- 172.21.15.60:0/1870757845 shutdown_connections 2023-11-18T20:42:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.284+0000 7f2b7810f640 1 --2- 172.21.15.60:0/1870757845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b700ce650 0x7f2b700d1af0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.284+0000 7f2b7810f640 1 --2- 172.21.15.60:0/1870757845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b7015d2d0 0x7f2b700b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.284+0000 7f2b7810f640 1 -- 172.21.15.60:0/1870757845 >> 172.21.15.60:0/1870757845 conn(0x7f2b70072230 msgr2=0x7f2b700cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.284+0000 7f2b7810f640 1 -- 172.21.15.60:0/1870757845 shutdown_connections 2023-11-18T20:42:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.284+0000 7f2b7810f640 1 -- 172.21.15.60:0/1870757845 wait complete. 2023-11-18T20:42:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.285+0000 7f2b7810f640 1 Processor -- start 2023-11-18T20:42:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.285+0000 7f2b7810f640 1 -- start start 2023-11-18T20:42:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b7810f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b700ce650 0x7f2b700b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b7810f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b7015d2d0 0x7f2b700c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b7810f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b70160f70 0x7f2b7015fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b7810f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b701f73d0 con 0x7f2b7015d2d0 2023-11-18T20:42:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b7810f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b701f6800 con 0x7f2b700ce650 2023-11-18T20:42:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b7810f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b701f6aa0 con 0x7f2b70160f70 2023-11-18T20:42:42.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b756aa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b7015d2d0 0x7f2b700c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b756aa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b7015d2d0 0x7f2b700c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41050/0 (socket says 172.21.15.60:41050) 2023-11-18T20:42:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b756aa640 1 -- 172.21.15.60:0/1474202648 learned_addr learned my addr 172.21.15.60:0/1474202648 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b75eab640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b700ce650 0x7f2b700b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b766ac640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b70160f70 0x7f2b7015fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b756aa640 1 -- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b70160f70 msgr2=0x7f2b7015fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b756aa640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b70160f70 0x7f2b7015fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b756aa640 1 -- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b700ce650 msgr2=0x7f2b700b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b756aa640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b700ce650 0x7f2b700b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b756aa640 1 -- 172.21.15.60:0/1474202648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b60009b30 con 0x7f2b7015d2d0 2023-11-18T20:42:42.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b75eab640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b700ce650 0x7f2b700b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b766ac640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b70160f70 0x7f2b7015fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.286+0000 7f2b756aa640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b7015d2d0 0x7f2b700c8bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400b3f0 tx=0x7f2b6400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.287+0000 7f2b6e7fc640 1 -- 172.21.15.60:0/1474202648 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64011830 con 0x7f2b7015d2d0 2023-11-18T20:42:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.287+0000 7f2b6e7fc640 1 -- 172.21.15.60:0/1474202648 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b64011990 con 0x7f2b7015d2d0 2023-11-18T20:42:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.287+0000 7f2b6e7fc640 1 -- 172.21.15.60:0/1474202648 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64011d40 con 0x7f2b7015d2d0 2023-11-18T20:42:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.287+0000 7f2b7810f640 1 -- 172.21.15.60:0/1474202648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b701f6d80 con 0x7f2b7015d2d0 2023-11-18T20:42:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.287+0000 7f2b7810f640 1 -- 172.21.15.60:0/1474202648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b700c9a40 con 0x7f2b7015d2d0 2023-11-18T20:42:42.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.288+0000 7f2b7810f640 1 -- 172.21.15.60:0/1474202648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b70069400 con 0x7f2b7015d2d0 2023-11-18T20:42:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.288+0000 7f2b6e7fc640 1 -- 172.21.15.60:0/1474202648 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b6401a400 con 0x7f2b7015d2d0 2023-11-18T20:42:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.288+0000 7f2b6e7fc640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b5c06ae40 0x7f2b5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.289+0000 7f2b6e7fc640 1 -- 172.21.15.60:0/1474202648 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b64089380 con 0x7f2b7015d2d0 2023-11-18T20:42:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.289+0000 7f2b75eab640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b5c06ae40 0x7f2b5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.289+0000 7f2b75eab640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b5c06ae40 0x7f2b5c06d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800a9d0 tx=0x7f2b68008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.293+0000 7f2b6e7fc640 1 -- 172.21.15.60:0/1474202648 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b640544c0 con 0x7f2b7015d2d0 2023-11-18T20:42:42.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.468+0000 7f2b7810f640 1 -- 172.21.15.60:0/1474202648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b700d0aa0 con 0x7f2b7015d2d0 2023-11-18T20:42:42.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.469+0000 7f2b6e7fc640 1 -- 172.21.15.60:0/1474202648 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f2b64057ae0 con 0x7f2b7015d2d0 2023-11-18T20:42:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:42.471 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:42.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 -- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b5c06ae40 msgr2=0x7f2b5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b5c06ae40 0x7f2b5c06d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800a9d0 tx=0x7f2b68008040 comp rx=0 tx=0).stop 2023-11-18T20:42:42.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 -- 172.21.15.60:0/1474202648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b7015d2d0 msgr2=0x7f2b700c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b7015d2d0 0x7f2b700c8bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400b3f0 tx=0x7f2b6400b8b0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 -- 172.21.15.60:0/1474202648 shutdown_connections 2023-11-18T20:42:42.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b5c06ae40 0x7f2b5c06d300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b70160f70 0x7f2b7015fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b7015d2d0 0x7f2b700c8bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 --2- 172.21.15.60:0/1474202648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b700ce650 0x7f2b700b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 -- 172.21.15.60:0/1474202648 >> 172.21.15.60:0/1474202648 conn(0x7f2b70072230 msgr2=0x7f2b70155750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 -- 172.21.15.60:0/1474202648 shutdown_connections 2023-11-18T20:42:42.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.471+0000 7f2b7810f640 1 -- 172.21.15.60:0/1474202648 wait complete. 2023-11-18T20:42:42.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-18T20:42:42.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-18T20:42:42.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-18T20:42:42.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-18T20:42:42.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-18T20:42:42.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-18T20:42:42.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-18T20:42:42.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-18T20:42:42.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.664+0000 7f9f44b8c640 1 Processor -- start 2023-11-18T20:42:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.664+0000 7f9f44b8c640 1 -- start start 2023-11-18T20:42:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f44b8c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f3804fbe0 0x7f9f38051740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f44b8c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9f38008140 0x7f9f3804ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f44b8c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38050500 0x7f9f38053870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f44b8c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f380484f0 con 0x7f9f3804fbe0 2023-11-18T20:42:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f44b8c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f38048790 con 0x7f9f38008140 2023-11-18T20:42:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f44b8c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38048a50 con 0x7f9f38050500 2023-11-18T20:42:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f3f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f3804fbe0 0x7f9f38051740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f3f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f3804fbe0 0x7f9f38051740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41060/0 (socket says 172.21.15.60:41060) 2023-11-18T20:42:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f3f7fe640 1 -- 172.21.15.60:0/1961080897 learned_addr learned my addr 172.21.15.60:0/1961080897 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f3ffff640 1 --2- 172.21.15.60:0/1961080897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38050500 0x7f9f38053870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f3f7fe640 1 -- 172.21.15.60:0/1961080897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38050500 msgr2=0x7f9f38053870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f3f7fe640 1 --2- 172.21.15.60:0/1961080897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38050500 0x7f9f38053870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f3f7fe640 1 -- 172.21.15.60:0/1961080897 >> v2:172.21.15.100:3300/0 conn(0x7f9f38008140 msgr2=0x7f9f3804ffc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-18T20:42:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f3f7fe640 1 --2- 172.21.15.60:0/1961080897 >> v2:172.21.15.100:3300/0 conn(0x7f9f38008140 0x7f9f3804ffc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.666+0000 7f9f3f7fe640 1 -- 172.21.15.60:0/1961080897 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f38053070 con 0x7f9f3804fbe0 2023-11-18T20:42:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.667+0000 7f57fba4b640 1 Processor -- start 2023-11-18T20:42:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.668+0000 7f5b23e15640 1 Processor -- start 2023-11-18T20:42:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.668+0000 7f9f3f7fe640 1 --2- 172.21.15.60:0/1961080897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3804fbe0 0x7f9f38051740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f38008dd0 tx=0x7f9f2c02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae6990006f606e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.668+0000 7f94be2e7640 1 Processor -- start 2023-11-18T20:42:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.668+0000 7f94be2e7640 1 -- start start 2023-11-18T20:42:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.668+0000 7f57fba4b640 1 -- start start 2023-11-18T20:42:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.668+0000 7f5b23e15640 1 -- start start 2023-11-18T20:42:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.669+0000 7f94be2e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94b00df220 0x7f94b0038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f94b77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f94b0053c70 0x7f94b004fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39094/0 (socket says 172.21.15.60:39094) 2023-11-18T20:42:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.668+0000 7f9f3e7fc640 1 -- 172.21.15.60:0/1961080897 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c030800 con 0x7f9f3804fbe0 2023-11-18T20:42:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.669+0000 7f9f3e7fc640 1 -- 172.21.15.60:0/1961080897 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c030960 con 0x7f9f3804fbe0 2023-11-18T20:42:42.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f94b77fe640 1 -- 172.21.15.60:0/1256816878 learned_addr learned my addr 172.21.15.60:0/1256816878 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:42.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f94b7fff640 1 --2- 172.21.15.60:0/1256816878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94b00df220 0x7f94b0038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.669+0000 7f5b23e15640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5b1400aaa0 0x7f5b140df1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.669+0000 7f5b23e15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b1404fa90 0x7f5b1404fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.669+0000 7f5b23e15640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b14050430 con 0x7f5b140382c0 2023-11-18T20:42:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.669+0000 7f5b23e15640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b14007770 con 0x7f5b1400aaa0 2023-11-18T20:42:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.669+0000 7f5b23e15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140068a0 con 0x7f5b1404fa90 2023-11-18T20:42:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f5b21bb1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5b1400aaa0 0x7f5b140df1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f5b21bb1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5b1400aaa0 0x7f5b140df1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49600/0 (socket says 172.21.15.60:49600) 2023-11-18T20:42:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f5b21bb1640 1 -- 172.21.15.60:0/3215344927 learned_addr learned my addr 172.21.15.60:0/3215344927 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f5b213b0640 1 --2- 172.21.15.60:0/3215344927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b1404fa90 0x7f5b1404fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f9f44b8c640 1 -- 172.21.15.60:0/1961080897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3804fbe0 msgr2=0x7f9f38051740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f9f44b8c640 1 --2- 172.21.15.60:0/1961080897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3804fbe0 0x7f9f38051740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f38008dd0 tx=0x7f9f2c02b940 comp rx=0 tx=0).stop 2023-11-18T20:42:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f9f44b8c640 1 -- 172.21.15.60:0/1961080897 shutdown_connections 2023-11-18T20:42:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f9f44b8c640 1 --2- 172.21.15.60:0/1961080897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38050500 0x7f9f38053870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f9f44b8c640 1 --2- 172.21.15.60:0/1961080897 >> v2:172.21.15.100:3300/0 conn(0x7f9f38008140 0x7f9f3804ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f9f44b8c640 1 --2- 172.21.15.60:0/1961080897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f3804fbe0 0x7f9f38051740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f9f44b8c640 1 -- 172.21.15.60:0/1961080897 >> 172.21.15.60:0/1961080897 conn(0x7f9f38055320 msgr2=0x7f9f38055730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.669+0000 7f57fba4b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57f4158fa0 0x7f57f40b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.669+0000 7f57fba4b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f40cd300 con 0x7f57f4159420 2023-11-18T20:42:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57fba4b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f40cd5e0 con 0x7f57f40b1880 2023-11-18T20:42:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57fba4b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f415ffb0 con 0x7f57f4158fa0 2023-11-18T20:42:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57f97e7640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f57f40b1880 0x7f57f415a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:52962/0 (socket says 172.21.15.60:52962) 2023-11-18T20:42:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57f97e7640 1 -- 172.21.15.60:0/2250557727 learned_addr learned my addr 172.21.15.60:0/2250557727 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f94be2e7640 1 -- 172.21.15.60:0/1256816878 --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b00544c0 con 0x7f94b0053c70 2023-11-18T20:42:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57f8fe6640 1 --2- 172.21.15.60:0/2250557727 >> v2:172.21.15.60:3300/0 conn(0x7f57f4158fa0 0x7f57f40b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995042904 0 0) 0x7f57f40cd5e0 con 0x7f57f40b1880 2023-11-18T20:42:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f5b21bb1640 1 -- 172.21.15.60:0/3215344927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b1404fa90 msgr2=0x7f5b1404fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f5b21bb1640 1 --2- 172.21.15.60:0/3215344927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b1404fa90 0x7f5b1404fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f5b21bb1640 1 -- 172.21.15.60:0/3215344927 >> v1:172.21.15.100:6789/0 conn(0x7f5b140382c0 legacy=0x7f5b14038750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-18T20:42:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f5b21bb1640 1 -- 172.21.15.60:0/3215344927 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140e5d20 con 0x7f5b1400aaa0 2023-11-18T20:42:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e0003520 con 0x7f57f40b1880 2023-11-18T20:42:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f94be2e7640 1 -- 172.21.15.60:0/1256816878 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b00541e0 con 0x7f94b00501a0 2023-11-18T20:42:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f94be2e7640 1 -- 172.21.15.60:0/1256816878 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b0005f30 con 0x7f94b00df220 2023-11-18T20:42:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263480766 0 0) 0x7f57e0003520 con 0x7f57f40b1880 2023-11-18T20:42:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 >> v2:172.21.15.60:3300/0 conn(0x7f57f4158fa0 msgr2=0x7f57f40b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57ebfff640 1 --2- 172.21.15.60:0/2250557727 >> v2:172.21.15.60:3300/0 conn(0x7f57f4158fa0 0x7f57f40b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.670+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 >> v1:172.21.15.60:6789/0 conn(0x7f57f4159420 legacy=0x7f57f40b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f9f44b8c640 1 -- 172.21.15.60:0/1961080897 shutdown_connections 2023-11-18T20:42:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f0f1a190640 1 Processor -- start 2023-11-18T20:42:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f94b6ffd640 1 -- 172.21.15.60:0/1256816878 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240782488 0 0) 0x7f94b00544c0 con 0x7f94b0053c70 2023-11-18T20:42:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f94b6ffd640 1 -- 172.21.15.60:0/1256816878 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a8003520 con 0x7f94b0053c70 2023-11-18T20:42:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f94b7fff640 1 -- 172.21.15.60:0/1256816878 >> v1:172.21.15.100:6789/0 conn(0x7f94b00501a0 legacy=0x7f94b0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f94b7fff640 1 -- 172.21.15.60:0/1256816878 >> v1:172.21.15.60:6789/0 conn(0x7f94b0053c70 legacy=0x7f94b004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f94b7fff640 1 -- 172.21.15.60:0/1256816878 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b0006210 con 0x7f94b00df220 2023-11-18T20:42:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f5b21bb1640 1 --2- 172.21.15.60:0/3215344927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b1400aaa0 0x7f5b140df1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009870 tx=0x7f5b0c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=543b3db3a9b40d62 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f0f1a190640 1 -- start start 2023-11-18T20:42:42.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f5b20baf640 1 -- 172.21.15.60:0/3215344927 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c030840 con 0x7f5b1400aaa0 2023-11-18T20:42:42.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40cd8c0 con 0x7f57f40b1880 2023-11-18T20:42:42.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f57f0001670 con 0x7f57f40b1880 2023-11-18T20:42:42.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 >> v1:172.21.15.100:6789/0 conn(0x7f57f40b1880 legacy=0x7f57f415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57ebfff640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e00039c0 0x7f57e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.682 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57f97e7640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e00039c0 0x7f57e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57ebfff640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57e00047c0 0x7f57e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57ebfff640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57e00051d0 0x7f57e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57f8fe6640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57e00047c0 0x7f57e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57f9fe8640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57e00051d0 0x7f57e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f415ffb0 con 0x7f57e00039c0 2023-11-18T20:42:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e000c110 con 0x7f57e00051d0 2023-11-18T20:42:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e000c2d0 con 0x7f57e00047c0 2023-11-18T20:42:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f57f97e7640 1 -- 172.21.15.60:0/2250557727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57e00047c0 msgr2=0x7f57e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f57f97e7640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57e00047c0 0x7f57e0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f57f97e7640 1 -- 172.21.15.60:0/2250557727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57e00051d0 msgr2=0x7f57e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f57f97e7640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57e00051d0 0x7f57e000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f57f97e7640 1 -- 172.21.15.60:0/2250557727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0003520 con 0x7f57e00039c0 2023-11-18T20:42:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f5b20baf640 1 -- 172.21.15.60:0/3215344927 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0309a0 con 0x7f5b1400aaa0 2023-11-18T20:42:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f5b20baf640 1 -- 172.21.15.60:0/3215344927 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c030d30 con 0x7f5b1400aaa0 2023-11-18T20:42:42.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f0f1a190640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0f140b34a0 0x7f0f141661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f0f1a190640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f140cd020 con 0x7f0f140b1880 2023-11-18T20:42:42.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f0f1a190640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f140cd300 con 0x7f0f14159180 2023-11-18T20:42:42.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f94b7fff640 1 --2- 172.21.15.60:0/1256816878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94b00df220 0x7f94b0038080 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f94a4010bc0 tx=0x7f94a402ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdcf68e320a3de37 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f58eb367640 1 Processor -- start 2023-11-18T20:42:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f58eb367640 1 -- start start 2023-11-18T20:42:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.672+0000 7f0f1a190640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f140cd5e0 con 0x7f0f140b34a0 2023-11-18T20:42:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.671+0000 7f9f44b8c640 1 -- 172.21.15.60:0/1961080897 wait complete. 2023-11-18T20:42:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f0f13fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0f140b34a0 0x7f0f141661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f94b6ffd640 1 -- 172.21.15.60:0/1256816878 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4030d40 con 0x7f94b00df220 2023-11-18T20:42:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f94b6ffd640 1 -- 172.21.15.60:0/1256816878 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a4038440 con 0x7f94b00df220 2023-11-18T20:42:42.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f57f8fe6640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57e00047c0 0x7f57e0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:42.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f57f9fe8640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57e00051d0 0x7f57e000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:42.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f57f9fe8640 1 -- 172.21.15.60:0/2250557727 reap_dead start 2023-11-18T20:42:42.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f94b6ffd640 1 -- 172.21.15.60:0/1256816878 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4030ea0 con 0x7f94b00df220 2023-11-18T20:42:42.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f0f13fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0f140b34a0 0x7f0f141661d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49612/0 (socket says 172.21.15.60:49612) 2023-11-18T20:42:42.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f0f13fff640 1 -- 172.21.15.60:0/487080160 learned_addr learned my addr 172.21.15.60:0/487080160 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f5b23e15640 1 -- 172.21.15.60:0/3215344927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b1400aaa0 msgr2=0x7f5b140df1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f5b23e15640 1 --2- 172.21.15.60:0/3215344927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b1400aaa0 0x7f5b140df1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009870 tx=0x7f5b0c02b980 comp rx=0 tx=0).stop 2023-11-18T20:42:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f5b23e15640 1 -- 172.21.15.60:0/3215344927 shutdown_connections 2023-11-18T20:42:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f5b23e15640 1 --2- 172.21.15.60:0/3215344927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b1404fa90 0x7f5b1404fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f5b23e15640 1 --2- 172.21.15.60:0/3215344927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b1400aaa0 0x7f5b140df1b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f5b23e15640 1 -- 172.21.15.60:0/3215344927 >> 172.21.15.60:0/3215344927 conn(0x7f5b14009fb0 msgr2=0x7f5b1400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f5b23e15640 1 -- 172.21.15.60:0/3215344927 shutdown_connections 2023-11-18T20:42:42.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f5b23e15640 1 -- 172.21.15.60:0/3215344927 wait complete. 2023-11-18T20:42:42.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f57f97e7640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e00039c0 0x7f57e0003de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f57f00104f0 tx=0x7f57e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3d46d4e2f5da44 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7fac0b59e640 1 Processor -- start 2023-11-18T20:42:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f003ddb0 con 0x7f57e00039c0 2023-11-18T20:42:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f0004870 con 0x7f57e00039c0 2023-11-18T20:42:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f94be2e7640 1 -- 172.21.15.60:0/1256816878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94b00df220 msgr2=0x7f94b0038080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f9f44b8c640 1 Processor -- start 2023-11-18T20:42:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58eb367640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58e406df00 0x7f58e4147520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58eb367640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f58e406e2e0 0x7f58e4161a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58eb367640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 0x7f58e4071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58eb367640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e41448b0 con 0x7f58e406df00 2023-11-18T20:42:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58eb367640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e4162a00 con 0x7f58e406e2e0 2023-11-18T20:42:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58eb367640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40cb750 con 0x7f58e4161fa0 2023-11-18T20:42:42.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f94be2e7640 1 --2- 172.21.15.60:0/1256816878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94b00df220 0x7f94b0038080 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f94a4010bc0 tx=0x7f94a402ff90 comp rx=0 tx=0).stop 2023-11-18T20:42:42.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58e9b64640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f58e406e2e0 0x7f58e4161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f0726613640 1 Processor -- start 2023-11-18T20:42:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f0f13fff640 1 -- 172.21.15.60:0/487080160 >> v1:172.21.15.100:6789/0 conn(0x7f0f14159180 legacy=0x7f0f140b40b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-18T20:42:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f0f13fff640 1 -- 172.21.15.60:0/487080160 >> v1:172.21.15.60:6789/0 conn(0x7f0f140b1880 legacy=0x7f0f1415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.673+0000 7f0f13fff640 1 -- 172.21.15.60:0/487080160 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f140cd8c0 con 0x7f0f140b34a0 2023-11-18T20:42:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f57ebfff640 1 -- 172.21.15.60:0/2250557727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0004ce0 con 0x7f57e00039c0 2023-11-18T20:42:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58eab66640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 0x7f58e4071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f9f44b8c640 1 -- start start 2023-11-18T20:42:42.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7fac0b59e640 1 -- start start 2023-11-18T20:42:42.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f0f13fff640 1 --2- 172.21.15.60:0/487080160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f140b34a0 0x7f0f141661d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009870 tx=0x7f0f0802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b348dafc9b214794 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58eab66640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 0x7f58e4071260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56112/0 (socket says 172.21.15.60:56112) 2023-11-18T20:42:42.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58e9b64640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f58e406e2e0 0x7f58e4161a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49620/0 (socket says 172.21.15.60:49620) 2023-11-18T20:42:42.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58eab66640 1 -- 172.21.15.60:0/5647244 learned_addr learned my addr 172.21.15.60:0/5647244 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.674+0000 7f58ea365640 1 --2- 172.21.15.60:0/5647244 >> v2:172.21.15.60:3300/0 conn(0x7f58e406df00 0x7f58e4147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f58eab66640 1 -- 172.21.15.60:0/5647244 >> v2:172.21.15.100:3300/0 conn(0x7f58e406e2e0 msgr2=0x7f58e4161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f58eab66640 1 --2- 172.21.15.60:0/5647244 >> v2:172.21.15.100:3300/0 conn(0x7f58e406e2e0 0x7f58e4161a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f58eab66640 1 -- 172.21.15.60:0/5647244 >> v2:172.21.15.60:3300/0 conn(0x7f58e406df00 msgr2=0x7f58e4147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f58eab66640 1 --2- 172.21.15.60:0/5647244 >> v2:172.21.15.60:3300/0 conn(0x7f58e406df00 0x7f58e4147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f58eab66640 1 -- 172.21.15.60:0/5647244 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e4147a60 con 0x7f58e4161fa0 2023-11-18T20:42:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f58ea365640 1 --2- 172.21.15.60:0/5647244 >> v2:172.21.15.60:3300/0 conn(0x7f58e406df00 0x7f58e4147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f94be2e7640 1 -- 172.21.15.60:0/1256816878 shutdown_connections 2023-11-18T20:42:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f94be2e7640 1 --2- 172.21.15.60:0/1256816878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94b00df220 0x7f94b0038080 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f5b23e15640 1 Processor -- start 2023-11-18T20:42:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f0f127fc640 1 -- 172.21.15.60:0/487080160 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08031870 con 0x7f0f140b34a0 2023-11-18T20:42:42.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f0f127fc640 1 -- 172.21.15.60:0/487080160 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f080319d0 con 0x7f0f140b34a0 2023-11-18T20:42:42.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f94be2e7640 1 -- 172.21.15.60:0/1256816878 >> 172.21.15.60:0/1256816878 conn(0x7f94b0009fb0 msgr2=0x7f94b000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7fac0b59e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 0x7fac0c0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7fac0b59e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fac0c0722b0 0x7fac0c0b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7fac0b59e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0c0710a0 con 0x7fac0c161620 2023-11-18T20:42:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7fac0b59e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac0c06c910 con 0x7fac0c0722b0 2023-11-18T20:42:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7fac0b59e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c06cbf0 con 0x7fac0c071e70 2023-11-18T20:42:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f57fba4b640 1 -- 172.21.15.60:0/2250557727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e00039c0 msgr2=0x7f57e0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f57fba4b640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e00039c0 0x7f57e0003de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f57f00104f0 tx=0x7f57e00057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f57fba4b640 1 -- 172.21.15.60:0/2250557727 shutdown_connections 2023-11-18T20:42:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f57fba4b640 1 --2- 172.21.15.60:0/2250557727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e00039c0 0x7f57e0003de0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f57fba4b640 1 -- 172.21.15.60:0/2250557727 >> 172.21.15.60:0/2250557727 conn(0x7f57f40b0d60 msgr2=0x7f57f40b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f94be2e7640 1 -- 172.21.15.60:0/1256816878 shutdown_connections 2023-11-18T20:42:42.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f94be2e7640 1 -- 172.21.15.60:0/1256816878 wait complete. 2023-11-18T20:42:42.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f0f127fc640 1 -- 172.21.15.60:0/487080160 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08031d60 con 0x7f0f140b34a0 2023-11-18T20:42:42.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7fac09d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 0x7fac0c0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7fac09d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 0x7fac0c0b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56118/0 (socket says 172.21.15.60:56118) 2023-11-18T20:42:42.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7fac09d9b640 1 -- 172.21.15.60:0/1494189873 learned_addr learned my addr 172.21.15.60:0/1494189873 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7fac0ad9d640 1 --2- 172.21.15.60:0/1494189873 >> v2:172.21.15.100:3300/0 conn(0x7fac0c0722b0 0x7fac0c0b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f5b23e15640 1 -- start start 2023-11-18T20:42:42.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f9f44b8c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38008140 0x7f9f38035b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f9f44b8c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3804fbe0 0x7f9f38036050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f9f44b8c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f38050500 0x7f9f380e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f9f44b8c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38187e30 con 0x7f9f38050500 2023-11-18T20:42:42.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f9f44b8c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38188110 con 0x7f9f3804fbe0 2023-11-18T20:42:42.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f9f44b8c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f381883f0 con 0x7f9f38008140 2023-11-18T20:42:42.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f0726613640 1 -- start start 2023-11-18T20:42:42.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f0726613640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180eb030 0x7f07180552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.705 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7fac09d9b640 1 -- 172.21.15.60:0/1494189873 >> v2:172.21.15.100:3300/0 conn(0x7fac0c0722b0 msgr2=0x7fac0c0b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f58eab66640 1 --2- 172.21.15.60:0/5647244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 0x7f58e4071260 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f58e4067c90 tx=0x7f58dc02bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7694e8747bba28 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f9f3f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38008140 0x7f9f38035b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f0726613640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f07180eb410 0x7f07180eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f0726613640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0718057e00 con 0x7f07180e7a40 2023-11-18T20:42:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f0726613640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0718057970 con 0x7f07180eb410 2023-11-18T20:42:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f0726613640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07180387b0 con 0x7f07180eb030 2023-11-18T20:42:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f9f3f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38008140 0x7f9f38035b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56130/0 (socket says 172.21.15.60:56130) 2023-11-18T20:42:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f9f3f7fe640 1 -- 172.21.15.60:0/1490632692 learned_addr learned my addr 172.21.15.60:0/1490632692 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7fac09d9b640 1 --2- 172.21.15.60:0/1494189873 >> v2:172.21.15.100:3300/0 conn(0x7fac0c0722b0 0x7fac0c0b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7fac09d9b640 1 -- 172.21.15.60:0/1494189873 >> v1:172.21.15.100:6789/0 conn(0x7fac0c161620 legacy=0x7fac0c06d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f0f1a190640 1 -- 172.21.15.60:0/487080160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f140b34a0 msgr2=0x7f0f141661d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f0f1a190640 1 --2- 172.21.15.60:0/487080160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f140b34a0 0x7f0f141661d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009870 tx=0x7f0f0802b910 comp rx=0 tx=0).stop 2023-11-18T20:42:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7fac09d9b640 1 -- 172.21.15.60:0/1494189873 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c06ced0 con 0x7fac0c071e70 2023-11-18T20:42:42.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f0725611640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180eb030 0x7f07180552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f0f1a190640 1 -- 172.21.15.60:0/487080160 shutdown_connections 2023-11-18T20:42:42.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f0f1a190640 1 --2- 172.21.15.60:0/487080160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f140b34a0 0x7f0f141661d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f0f1a190640 1 -- 172.21.15.60:0/487080160 >> 172.21.15.60:0/487080160 conn(0x7f0f140b0d60 msgr2=0x7f0f140b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f5b23e15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b1400aaa0 0x7f5b140dd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f5b23e15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b140382c0 0x7f5b1404e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f5b23e15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b1404fa90 0x7f5b140466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f5b23e15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140d2600 con 0x7f5b140382c0 2023-11-18T20:42:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f5b23e15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14035320 con 0x7f5b1400aaa0 2023-11-18T20:42:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f5b23e15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140355c0 con 0x7f5b1404fa90 2023-11-18T20:42:42.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f0725611640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180eb030 0x7f07180552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56134/0 (socket says 172.21.15.60:56134) 2023-11-18T20:42:42.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f0725611640 1 -- 172.21.15.60:0/1748587557 learned_addr learned my addr 172.21.15.60:0/1748587557 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f5b213b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b140382c0 0x7f5b1404e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f5b213b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b140382c0 0x7f5b1404e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41090/0 (socket says 172.21.15.60:41090) 2023-11-18T20:42:42.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f5b213b0640 1 -- 172.21.15.60:0/2100125991 learned_addr learned my addr 172.21.15.60:0/2100125991 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f5b223b2640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b1404fa90 0x7f5b140466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f5b21bb1640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b1400aaa0 0x7f5b140dd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f9f3f7fe640 1 -- 172.21.15.60:0/1490632692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3804fbe0 msgr2=0x7f9f38036050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-18T20:42:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f9f3f7fe640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f3804fbe0 0x7f9f38036050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f9f3f7fe640 1 -- 172.21.15.60:0/1490632692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f38050500 msgr2=0x7f9f380e3d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-18T20:42:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f9f3f7fe640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f38050500 0x7f9f380e3d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f9f3f7fe640 1 -- 172.21.15.60:0/1490632692 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c0094d0 con 0x7f9f38008140 2023-11-18T20:42:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077404723 0 0) 0x7f0718057e00 con 0x7f07180e7a40 2023-11-18T20:42:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f070c003520 con 0x7f07180e7a40 2023-11-18T20:42:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f57fba4b640 1 -- 172.21.15.60:0/2250557727 shutdown_connections 2023-11-18T20:42:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.675+0000 7f57fba4b640 1 -- 172.21.15.60:0/2250557727 wait complete. 2023-11-18T20:42:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f5b223b2640 1 -- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b1400aaa0 msgr2=0x7f5b140dd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f5b223b2640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b1400aaa0 0x7f5b140dd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f5b223b2640 1 -- 172.21.15.60:0/2100125991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b140382c0 msgr2=0x7f5b1404e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f5b223b2640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b140382c0 0x7f5b1404e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f5b223b2640 1 -- 172.21.15.60:0/2100125991 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0094d0 con 0x7f5b1404fa90 2023-11-18T20:42:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f9f3f7fe640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38008140 0x7f9f38035b10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0097c0 tx=0x7f9f2c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f0724e10640 1 --2- 172.21.15.60:0/1748587557 >> v2:172.21.15.100:3300/0 conn(0x7f07180eb410 0x7f07180eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f94be2e7640 1 Processor -- start 2023-11-18T20:42:42.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1556834223 0 0) 0x7f070c003520 con 0x7f07180e7a40 2023-11-18T20:42:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.676+0000 7f58e9363640 1 -- 172.21.15.60:0/5647244 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc02ca80 con 0x7f58e4161fa0 2023-11-18T20:42:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f58e9363640 1 -- 172.21.15.60:0/5647244 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc02cbe0 con 0x7f58e4161fa0 2023-11-18T20:42:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f58eb367640 1 -- 172.21.15.60:0/5647244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 msgr2=0x7f58e4071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180eb030 msgr2=0x7f07180552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180eb030 0x7f07180552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 >> v2:172.21.15.100:3300/0 conn(0x7f07180eb410 msgr2=0x7f07180eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 --2- 172.21.15.60:0/1748587557 >> v2:172.21.15.100:3300/0 conn(0x7f07180eb410 0x7f07180eb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718056d60 con 0x7f07180e7a40 2023-11-18T20:42:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f071c001650 con 0x7f07180e7a40 2023-11-18T20:42:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f58eb367640 1 --2- 172.21.15.60:0/5647244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 0x7f58e4071260 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f58e4067c90 tx=0x7f58dc02bee0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0f1a190640 1 -- 172.21.15.60:0/487080160 shutdown_connections 2023-11-18T20:42:42.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 >> v1:172.21.15.60:6789/0 conn(0x7f07180e7a40 legacy=0x7f07180e7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f94be2e7640 1 -- start start 2023-11-18T20:42:42.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f5b223b2640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b1404fa90 0x7f5b140466b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c064180 tx=0x7f5b1c0678b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0f1a190640 1 -- 172.21.15.60:0/487080160 wait complete. 2023-11-18T20:42:42.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f57fba4b640 1 Processor -- start 2023-11-18T20:42:42.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.677+0000 7f57fba4b640 1 -- start start 2023-11-18T20:42:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180eb030 0x7f070c0039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c003f50 0x7f070c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c004d00 0x7f070c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0718057970 con 0x7f070c003f50 2023-11-18T20:42:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07180387b0 con 0x7f070c004d00 2023-11-18T20:42:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c00b9b0 con 0x7f07180eb030 2023-11-18T20:42:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f58eb367640 1 -- 172.21.15.60:0/5647244 shutdown_connections 2023-11-18T20:42:42.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f58eb367640 1 --2- 172.21.15.60:0/5647244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 0x7f58e4071260 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f58eb367640 1 --2- 172.21.15.60:0/5647244 >> v2:172.21.15.100:3300/0 conn(0x7f58e406e2e0 0x7f58e4161a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f58eb367640 1 --2- 172.21.15.60:0/5647244 >> v2:172.21.15.60:3300/0 conn(0x7f58e406df00 0x7f58e4147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f58eb367640 1 -- 172.21.15.60:0/5647244 >> 172.21.15.60:0/5647244 conn(0x7f58e40b0c60 msgr2=0x7f58e40b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f5b127fc640 1 -- 172.21.15.60:0/2100125991 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c06bb30 con 0x7f5b1404fa90 2023-11-18T20:42:42.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f9f2bfff640 1 -- 172.21.15.60:0/1490632692 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c043600 con 0x7f9f38008140 2023-11-18T20:42:42.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f9f2bfff640 1 -- 172.21.15.60:0/1490632692 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c043760 con 0x7f9f38008140 2023-11-18T20:42:42.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f9f2bfff640 1 -- 172.21.15.60:0/1490632692 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c043ad0 con 0x7f9f38008140 2023-11-18T20:42:42.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f9f44b8c640 1 -- 172.21.15.60:0/1490632692 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f381886d0 con 0x7f9f38008140 2023-11-18T20:42:42.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f9f44b8c640 1 -- 172.21.15.60:0/1490632692 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f38049480 con 0x7f9f38008140 2023-11-18T20:42:42.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f0725611640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180eb030 0x7f070c0039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f0725e12640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c004d00 0x7f070c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f57fba4b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57e00039c0 0x7f57f415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f0724e10640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c003f50 0x7f070c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7fac09d9b640 1 --2- 172.21.15.60:0/1494189873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 0x7fac0c0b3000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fac04009e50 tx=0x7fac0402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca86c225805381c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7fac0959a640 1 -- 172.21.15.60:0/1494189873 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0402cb00 con 0x7fac0c071e70 2023-11-18T20:42:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7fac0959a640 1 -- 172.21.15.60:0/1494189873 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0402cc60 con 0x7fac0c071e70 2023-11-18T20:42:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7fac0959a640 1 -- 172.21.15.60:0/1494189873 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac040343f0 con 0x7fac0c071e70 2023-11-18T20:42:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f94be2e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94b00501a0 0x7f94b0168f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f94be2e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b0053c70 0x7f94b00d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f94be2e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94b00df220 0x7f94b004c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f94be2e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b004bb90 con 0x7f94b0053c70 2023-11-18T20:42:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f94be2e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b0169560 con 0x7f94b00df220 2023-11-18T20:42:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f94be2e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b0169840 con 0x7f94b00501a0 2023-11-18T20:42:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.678+0000 7f5b127fc640 1 -- 172.21.15.60:0/2100125991 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c06bc90 con 0x7f5b1404fa90 2023-11-18T20:42:42.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f0725611640 1 -- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c004d00 msgr2=0x7f070c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f0725611640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f070c004d00 0x7f070c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f0725611640 1 -- 172.21.15.60:0/1748587557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c003f50 msgr2=0x7f070c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:42.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f5b127fc640 1 -- 172.21.15.60:0/2100125991 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c072400 con 0x7f5b1404fa90 2023-11-18T20:42:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f94b77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b0053c70 0x7f94b00d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f57fba4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57f415a2a0 0x7f57f406da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f94b77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b0053c70 0x7f94b00d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41102/0 (socket says 172.21.15.60:41102) 2023-11-18T20:42:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f94b77fe640 1 -- 172.21.15.60:0/1138456521 learned_addr learned my addr 172.21.15.60:0/1138456521 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f94bc884640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94b00df220 0x7f94b004c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f94b77fe640 1 -- 172.21.15.60:0/1138456521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94b00501a0 msgr2=0x7f94b0168f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f94b77fe640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94b00501a0 0x7f94b0168f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f9f44b8c640 1 -- 172.21.15.60:0/1490632692 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9f38188280 con 0x7f9f38008140 2023-11-18T20:42:42.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f94b77fe640 1 -- 172.21.15.60:0/1138456521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94b00df220 msgr2=0x7f94b004c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f94b77fe640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94b00df220 0x7f94b004c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f0f1a190640 1 Processor -- start 2023-11-18T20:42:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f94b77fe640 1 -- 172.21.15.60:0/1138456521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a40107f0 con 0x7f94b0053c70 2023-11-18T20:42:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f0725611640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c003f50 0x7f070c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f57fba4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f414e820 0x7f57f414eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f57f97e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57e00039c0 0x7f57f415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f57f97e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57e00039c0 0x7f57f415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56170/0 (socket says 172.21.15.60:56170) 2023-11-18T20:42:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f57f97e7640 1 -- 172.21.15.60:0/202457891 learned_addr learned my addr 172.21.15.60:0/202457891 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f41eebb0 con 0x7f57f415a2a0 2023-11-18T20:42:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f57f9fe8640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f414e820 0x7f57f414eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f415b700 con 0x7f57f414e820 2023-11-18T20:42:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f414f1e0 con 0x7f57e00039c0 2023-11-18T20:42:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f57f9fe8640 1 -- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57e00039c0 msgr2=0x7f57f415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f57f9fe8640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57e00039c0 0x7f57f415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f57f9fe8640 1 -- 172.21.15.60:0/202457891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57f415a2a0 msgr2=0x7f57f406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f57f9fe8640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57f415a2a0 0x7f57f406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f57f9fe8640 1 -- 172.21.15.60:0/202457891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f415b930 con 0x7f57f414e820 2023-11-18T20:42:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f57f9fe8640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f414e820 0x7f57f414eca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f57f40cd230 tx=0x7f57ec00d900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f94b77fe640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b0053c70 0x7f94b00d13e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c410 tx=0x7f94ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f5b23e15640 1 -- 172.21.15.60:0/2100125991 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b140474c0 con 0x7f5b1404fa90 2023-11-18T20:42:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f5b23e15640 1 -- 172.21.15.60:0/2100125991 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b140d2da0 con 0x7f5b1404fa90 2023-11-18T20:42:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f0f1a190640 1 -- start start 2023-11-18T20:42:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.679+0000 7f0725611640 1 -- 172.21.15.60:0/1748587557 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718056d60 con 0x7f07180eb030 2023-11-18T20:42:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f0724e10640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f070c003f50 0x7f070c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f9f2bfff640 1 -- 172.21.15.60:0/1490632692 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f2c043c30 con 0x7f9f38008140 2023-11-18T20:42:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f5b23e15640 1 -- 172.21.15.60:0/2100125991 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b14007390 con 0x7f5b1404fa90 2023-11-18T20:42:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.680+0000 7f94b4ff9640 1 -- 172.21.15.60:0/1138456521 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac005ce0 con 0x7f94b0053c70 2023-11-18T20:42:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f58eb367640 1 -- 172.21.15.60:0/5647244 shutdown_connections 2023-11-18T20:42:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f94be2e7640 1 -- 172.21.15.60:0/1138456521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b004bde0 con 0x7f94b0053c70 2023-11-18T20:42:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f94b4ff9640 1 -- 172.21.15.60:0/1138456521 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac005e40 con 0x7f94b0053c70 2023-11-18T20:42:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f57e9ffb640 1 -- 172.21.15.60:0/202457891 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00b070 con 0x7f57f414e820 2023-11-18T20:42:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f0725611640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180eb030 0x7f070c0039d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0720050020 tx=0x7f070c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6607394330980c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200a6b90 con 0x7f07180eb030 2023-11-18T20:42:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07200aa070 con 0x7f07180eb030 2023-11-18T20:42:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f0717fff640 1 -- 172.21.15.60:0/1748587557 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200b0400 con 0x7f07180eb030 2023-11-18T20:42:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f9f2bfff640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f2406ae90 0x7f9f2406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f9f2bfff640 1 -- 172.21.15.60:0/1490632692 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f2c03e070 con 0x7f9f38008140 2023-11-18T20:42:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f9f2bfff640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] conn(0x7f9f240730b0 0x7f9f240754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f9f2bfff640 1 -- 172.21.15.60:0/1490632692 --> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f24075b80 con 0x7f9f240730b0 2023-11-18T20:42:42.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f9f2bfff640 1 -- 172.21.15.60:0/1490632692 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f2c0aea20 con 0x7f9f38008140 2023-11-18T20:42:42.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7fac0b59e640 1 -- 172.21.15.60:0/1494189873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 msgr2=0x7fac0c0b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7fac0b59e640 1 --2- 172.21.15.60:0/1494189873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 0x7fac0c0b3000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fac04009e50 tx=0x7fac0402bf60 comp rx=0 tx=0).stop 2023-11-18T20:42:42.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f9f3effd640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f2406ae90 0x7f9f2406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f57e9ffb640 1 -- 172.21.15.60:0/202457891 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec003da0 con 0x7f57f414e820 2023-11-18T20:42:42.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f57e9ffb640 1 -- 172.21.15.60:0/202457891 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec005310 con 0x7f57f414e820 2023-11-18T20:42:42.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0f1a190640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f140b1880 0x7f0f141435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7fac0b59e640 1 -- 172.21.15.60:0/1494189873 shutdown_connections 2023-11-18T20:42:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7fac0b59e640 1 --2- 172.21.15.60:0/1494189873 >> v2:172.21.15.100:3300/0 conn(0x7fac0c0722b0 0x7fac0c0b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7fac0b59e640 1 --2- 172.21.15.60:0/1494189873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 0x7fac0c0b3000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7fac0b59e640 1 -- 172.21.15.60:0/1494189873 >> 172.21.15.60:0/1494189873 conn(0x7fac0c06a4d0 msgr2=0x7fac0c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f415bc10 con 0x7f57f414e820 2023-11-18T20:42:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0f1a190640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f140b34a0 0x7f0f141545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7fac0b59e640 1 -- 172.21.15.60:0/1494189873 shutdown_connections 2023-11-18T20:42:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f9f3ffff640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] conn(0x7f9f240730b0 0x7f9f240754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0f137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f140b1880 0x7f0f141435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0f12ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f140b34a0 0x7f0f141545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0f12ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f140b34a0 0x7f0f141545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41128/0 (socket says 172.21.15.60:41128) 2023-11-18T20:42:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0726613640 1 -- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180eb030 msgr2=0x7f070c0039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0726613640 1 --2- 172.21.15.60:0/1748587557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180eb030 0x7f070c0039d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0720050020 tx=0x7f070c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0725e12640 1 -- 172.21.15.60:0/1748587557 reap_dead start 2023-11-18T20:42:42.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0f137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f140b1880 0x7f0f141435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49682/0 (socket says 172.21.15.60:49682) 2023-11-18T20:42:42.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0f12ffd640 1 -- 172.21.15.60:0/3073393643 learned_addr learned my addr 172.21.15.60:0/3073393643 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f0f1a190640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f14159180 0x7f0f141472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f415c1d0 con 0x7f57f414e820 2023-11-18T20:42:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.683+0000 7f9f3ffff640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] conn(0x7f9f240730b0 0x7f9f240754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.683+0000 7f9f3effd640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f2406ae90 0x7f9f2406d350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9f380ce650 tx=0x7f9f34006bd0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.683+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1415b8c0 con 0x7f0f140b34a0 2023-11-18T20:42:42.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7fac0b59e640 1 -- 172.21.15.60:0/1494189873 wait complete. 2023-11-18T20:42:42.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f94b4ff9640 1 -- 172.21.15.60:0/1138456521 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac007310 con 0x7f94b0053c70 2023-11-18T20:42:42.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.682+0000 7f94be2e7640 1 -- 172.21.15.60:0/1138456521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b00d1920 con 0x7f94b0053c70 2023-11-18T20:42:42.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.683+0000 7f0726613640 1 -- 172.21.15.60:0/1748587557 shutdown_connections 2023-11-18T20:42:42.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.683+0000 7f0726613640 1 -- 172.21.15.60:0/1748587557 >> 172.21.15.60:0/1748587557 conn(0x7f07180559b0 msgr2=0x7f0718055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.683+0000 7f94a27fc640 1 -- 172.21.15.60:0/1138456521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94b00df600 con 0x7f94b0053c70 2023-11-18T20:42:42.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.681+0000 7f58eb367640 1 -- 172.21.15.60:0/5647244 wait complete. 2023-11-18T20:42:42.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.683+0000 7f58eb367640 1 Processor -- start 2023-11-18T20:42:42.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.683+0000 7f58eb367640 1 -- start start 2023-11-18T20:42:42.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.683+0000 7f0726613640 1 -- 172.21.15.60:0/1748587557 shutdown_connections 2023-11-18T20:42:42.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f9f2bfff640 1 -- 172.21.15.60:0/1490632692 <== osd.2 v2:172.21.15.60:6808/479181725 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9f24075b80 con 0x7f9f240730b0 2023-11-18T20:42:42.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f58eb367640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e406df00 0x7f58e4157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f58eb367640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e406e2e0 0x7f58e41590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f58eb367640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 0x7f58e40cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f58eb367640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e413f7d0 con 0x7f58e406df00 2023-11-18T20:42:42.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f58eb367640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40cd020 con 0x7f58e406e2e0 2023-11-18T20:42:42.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f57d77fe640 1 -- 172.21.15.60:0/202457891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57d0000fc0 con 0x7f57f414e820 2023-11-18T20:42:42.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7fac0b59e640 1 Processor -- start 2023-11-18T20:42:42.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7fac0b59e640 1 -- start start 2023-11-18T20:42:42.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f58eb367640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e4070bd0 con 0x7f58e4161fa0 2023-11-18T20:42:42.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f58ea365640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e406df00 0x7f58e4157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f58ea365640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e406df00 0x7f58e4157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41140/0 (socket says 172.21.15.60:41140) 2023-11-18T20:42:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f58ea365640 1 -- 172.21.15.60:0/1716408051 learned_addr learned my addr 172.21.15.60:0/1716408051 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f58eab66640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 0x7f58e40cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.683+0000 7f0f13fff640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f14159180 0x7f0f141472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f14143af0 con 0x7f0f140b1880 2023-11-18T20:42:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1415ac40 con 0x7f0f14159180 2023-11-18T20:42:42.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f5b127fc640 1 -- 172.21.15.60:0/2100125991 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5b1c072560 con 0x7f5b1404fa90 2023-11-18T20:42:42.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f5b127fc640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5b0006ae40 0x7f5b0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f94b4ff9640 1 -- 172.21.15.60:0/1138456521 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94ac007470 con 0x7f94b0053c70 2023-11-18T20:42:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f5b127fc640 1 -- 172.21.15.60:0/2100125991 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b1c1035c0 con 0x7f5b1404fa90 2023-11-18T20:42:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f5b127fc640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] conn(0x7f5b00072ef0 0x7f5b00075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f5b21bb1640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5b0006ae40 0x7f5b0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f94b4ff9640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f949c06ae70 0x7f949c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f5b127fc640 1 -- 172.21.15.60:0/2100125991 --> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b000759c0 con 0x7f5b00072ef0 2023-11-18T20:42:42.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f58e9b64640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e406e2e0 0x7f58e41590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f5b127fc640 1 -- 172.21.15.60:0/2100125991 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b1c103970 con 0x7f5b1404fa90 2023-11-18T20:42:42.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f94b4ff9640 1 -- 172.21.15.60:0/1138456521 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94ac088990 con 0x7f94b0053c70 2023-11-18T20:42:42.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f94b7fff640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f949c06ae70 0x7f949c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f94b4ff9640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] conn(0x7f949c073090 0x7f949c0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f94b4ff9640 1 -- 172.21.15.60:0/1138456521 --> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f949c075b60 con 0x7f949c073090 2023-11-18T20:42:42.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f94b4ff9640 1 -- 172.21.15.60:0/1138456521 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94ac088c90 con 0x7f94b0053c70 2023-11-18T20:42:42.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f58ea365640 1 -- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 msgr2=0x7f58e40cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f58ea365640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e4161fa0 0x7f58e40cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f58ea365640 1 -- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e406e2e0 msgr2=0x7f58e41590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f58ea365640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e406e2e0 0x7f58e41590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f58ea365640 1 -- 172.21.15.60:0/1716408051 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc009b30 con 0x7f58e406df00 2023-11-18T20:42:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f58e9b64640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e406e2e0 0x7f58e41590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f5b213b0640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] conn(0x7f5b00072ef0 0x7f5b00075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.684+0000 7f0726613640 1 -- 172.21.15.60:0/1748587557 wait complete. 2023-11-18T20:42:42.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.685+0000 7f0726613640 1 Processor -- start 2023-11-18T20:42:42.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f94bc884640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] conn(0x7f949c073090 0x7f949c0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f0f12ffd640 1 -- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f14159180 msgr2=0x7f0f141472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f58ea365640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e406df00 0x7f58e4157a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f58d400e7e0 tx=0x7f58d400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f5b21bb1640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5b0006ae40 0x7f5b0006d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c036000 tx=0x7f5b0c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f0f12ffd640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f14159180 0x7f0f141472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f0f12ffd640 1 -- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f140b1880 msgr2=0x7f0f141435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f0f12ffd640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f140b1880 0x7f0f141435b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f94b7fff640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f949c06ae70 0x7f949c06d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f94a403a000 tx=0x7f94a4032000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f5b213b0640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] conn(0x7f5b00072ef0 0x7f5b00075310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7fac0b59e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 0x7fac0c155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f58daffd640 1 -- 172.21.15.60:0/1716408051 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4004030 con 0x7f58e406df00 2023-11-18T20:42:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e413f9e0 con 0x7f58e406df00 2023-11-18T20:42:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7fac0b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac0c0722b0 0x7fac0c149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f57e9ffb640 1 -- 172.21.15.60:0/202457891 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57ec038060 con 0x7f57f414e820 2023-11-18T20:42:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f0726613640 1 -- start start 2023-11-18T20:42:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7fac09d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac0c0722b0 0x7fac0c149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f57e9ffb640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57d806ae90 0x7f57d806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f0726613640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07180eb030 0x7f071803c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f57e9ffb640 1 -- 172.21.15.60:0/202457891 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57ec036070 con 0x7f57f414e820 2023-11-18T20:42:42.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f0726613640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f071804d130 0x7f07180d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f57e9ffb640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] conn(0x7f57d80730b0 0x7f57d80754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f0725611640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07180eb030 0x7f071803c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f0726613640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180e94b0 0x7f07180e9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f0725611640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07180eb030 0x7f071803c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41154/0 (socket says 172.21.15.60:41154) 2023-11-18T20:42:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f0f12ffd640 1 -- 172.21.15.60:0/3073393643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f080094d0 con 0x7f0f140b34a0 2023-11-18T20:42:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.686+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e40cd550 con 0x7f58e406df00 2023-11-18T20:42:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f57e9ffb640 1 -- 172.21.15.60:0/202457891 --> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57d8075b80 con 0x7f57d80730b0 2023-11-18T20:42:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f57e9ffb640 1 -- 172.21.15.60:0/202457891 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57ec035030 con 0x7f57f414e820 2023-11-18T20:42:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f0724e10640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f071804d130 0x7f07180d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f5b127fc640 1 -- 172.21.15.60:0/2100125991 <== osd.1 v2:172.21.15.60:6809/3839540220 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5b000759c0 con 0x7f5b00072ef0 2023-11-18T20:42:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7fac0a59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 0x7fac0c155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7fac0a59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 0x7fac0c155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56202/0 (socket says 172.21.15.60:56202) 2023-11-18T20:42:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7fac0a59c640 1 -- 172.21.15.60:0/2355923215 learned_addr learned my addr 172.21.15.60:0/2355923215 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.687+0000 7f58daffd640 1 -- 172.21.15.60:0/1716408051 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d4004190 con 0x7f58e406df00 2023-11-18T20:42:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7fac0b59e640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac0c161620 0x7fac0c15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7f58daffd640 1 -- 172.21.15.60:0/1716408051 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4004520 con 0x7f58e406df00 2023-11-18T20:42:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f58e4146d20 con 0x7f58e406df00 2023-11-18T20:42:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c1643f0 con 0x7fac0c0722b0 2023-11-18T20:42:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c163b00 con 0x7fac0c161620 2023-11-18T20:42:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c163da0 con 0x7fac0c071e70 2023-11-18T20:42:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7fac0ad9d640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac0c161620 0x7fac0c15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7f57f8fe6640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] conn(0x7f57d80730b0 0x7f57d80754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7f94bc884640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] conn(0x7f949c073090 0x7f949c0754b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7f57f8fe6640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] conn(0x7f57d80730b0 0x7f57d80754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f58daffd640 1 -- 172.21.15.60:0/1716408051 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58d400c730 con 0x7f58e406df00 2023-11-18T20:42:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7f0726613640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071803d230 con 0x7f07180eb030 2023-11-18T20:42:42.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7f0726613640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07180e9e70 con 0x7f071804d130 2023-11-18T20:42:42.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7f0726613640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07180d8400 con 0x7f07180e94b0 2023-11-18T20:42:42.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.688+0000 7f0725611640 1 -- 172.21.15.60:0/3897117002 learned_addr learned my addr 172.21.15.60:0/3897117002 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:42.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f0725611640 1 -- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180e94b0 msgr2=0x7f07180e9930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-18T20:42:42.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f0725611640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07180e94b0 0x7f07180e9930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f0725611640 1 -- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f071804d130 msgr2=0x7f07180d7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f58daffd640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c806af10 0x7f58c806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f57f97e7640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57d806ae90 0x7f57d806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f94b4ff9640 1 -- 172.21.15.60:0/1138456521 <== osd.0 v2:172.21.15.60:6800/2078754623 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f949c075b60 con 0x7f949c073090 2023-11-18T20:42:42.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f0725611640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f071804d130 0x7f07180d7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f0725611640 1 -- 172.21.15.60:0/3897117002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071804d8a0 con 0x7f07180eb030 2023-11-18T20:42:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f0724e10640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f071804d130 0x7f07180d7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f0f12ffd640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f140b34a0 0x7f0f141545d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400b3e0 tx=0x7f0f0400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f0efbfff640 1 -- 172.21.15.60:0/3073393643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04011830 con 0x7f0f140b34a0 2023-11-18T20:42:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.689+0000 7f58e9b64640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c806af10 0x7f58c806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f0efbfff640 1 -- 172.21.15.60:0/3073393643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f040119b0 con 0x7f0f140b34a0 2023-11-18T20:42:42.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f58daffd640 1 -- 172.21.15.60:0/1716408051 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58d40891b0 con 0x7f58e406df00 2023-11-18T20:42:42.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f57e9ffb640 1 -- 172.21.15.60:0/202457891 <== osd.3 v2:172.21.15.60:6824/4034119923 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f57d8075b80 con 0x7f57d80730b0 2023-11-18T20:42:42.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f0725611640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07180eb030 0x7f071803c680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f07200681a0 tx=0x7f07200686a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f0715ffb640 1 -- 172.21.15.60:0/3897117002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200bb470 con 0x7f07180eb030 2023-11-18T20:42:42.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f58daffd640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] conn(0x7f58c8073130 0x7f58c8075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f58daffd640 1 -- 172.21.15.60:0/1716408051 --> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58c8075c00 con 0x7f58c8073130 2023-11-18T20:42:42.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f58daffd640 1 -- 172.21.15.60:0/1716408051 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58d408b770 con 0x7f58e406df00 2023-11-18T20:42:42.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7fac09d9b640 1 -- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 msgr2=0x7fac0c155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f0715ffb640 1 -- 172.21.15.60:0/3897117002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07200bb5d0 con 0x7f07180eb030 2023-11-18T20:42:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7fac09d9b640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac0c071e70 0x7fac0c155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7fac09d9b640 1 -- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac0c161620 msgr2=0x7fac0c15fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7fac09d9b640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac0c161620 0x7fac0c15fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f0715ffb640 1 -- 172.21.15.60:0/3897117002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200bb940 con 0x7f07180eb030 2023-11-18T20:42:42.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f58eab66640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] conn(0x7f58c8073130 0x7f58c8075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7fac09d9b640 1 -- 172.21.15.60:0/2355923215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac04009b30 con 0x7fac0c0722b0 2023-11-18T20:42:42.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f57f97e7640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57d806ae90 0x7f57d806d350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57f0001050 tx=0x7f57f0006000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.691+0000 7f58e9b64640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c806af10 0x7f58c806d3d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009a40 tx=0x7f58e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f0efbfff640 1 -- 172.21.15.60:0/3073393643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04011d60 con 0x7f0f140b34a0 2023-11-18T20:42:42.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1415af20 con 0x7f0f140b34a0 2023-11-18T20:42:42.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.691+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071804db80 con 0x7f07180eb030 2023-11-18T20:42:42.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.691+0000 7f58eab66640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] conn(0x7f58c8073130 0x7f58c8075550 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.690+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1415b4e0 con 0x7f0f140b34a0 2023-11-18T20:42:42.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.691+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071804e140 con 0x7f07180eb030 2023-11-18T20:42:42.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.691+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ef4000fc0 con 0x7f0f140b34a0 2023-11-18T20:42:42.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.692+0000 7f58daffd640 1 -- 172.21.15.60:0/1716408051 <== osd.7 v2:172.21.15.100:6824/1780039900 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f58c8075c00 con 0x7f58c8073130 2023-11-18T20:42:42.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.693+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0718057600 con 0x7f07180eb030 2023-11-18T20:42:42.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.693+0000 7f0715ffb640 1 -- 172.21.15.60:0/3897117002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f07200bbaa0 con 0x7f07180eb030 2023-11-18T20:42:42.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.693+0000 7fac09d9b640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac0c0722b0 0x7fac0c149fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac04032000 tx=0x7fac04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.694+0000 7f0efbfff640 1 -- 172.21.15.60:0/3073393643 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f0401a400 con 0x7f0f140b34a0 2023-11-18T20:42:42.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.694+0000 7f0efbfff640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0f0006af10 0x7f0f0006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.694+0000 7f0efbfff640 1 -- 172.21.15.60:0/3073393643 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f04089c10 con 0x7f0f140b34a0 2023-11-18T20:42:42.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.694+0000 7f0efbfff640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] conn(0x7f0f00073130 0x7f0f00075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.694+0000 7f0efbfff640 1 -- 172.21.15.60:0/3073393643 --> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f00075c00 con 0x7f0f00073130 2023-11-18T20:42:42.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.694+0000 7f0efbfff640 1 -- 172.21.15.60:0/3073393643 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f04089fc0 con 0x7f0f140b34a0 2023-11-18T20:42:42.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.694+0000 7fabfaffd640 1 -- 172.21.15.60:0/2355923215 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0403e450 con 0x7fac0c0722b0 2023-11-18T20:42:42.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.694+0000 7fabfaffd640 1 -- 172.21.15.60:0/2355923215 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0403e5d0 con 0x7fac0c0722b0 2023-11-18T20:42:42.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.694+0000 7f0715ffb640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f070406ae40 0x7f070406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.694+0000 7f0715ffb640 1 -- 172.21.15.60:0/3897117002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0720125f70 con 0x7f07180eb030 2023-11-18T20:42:42.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.695+0000 7f0724e10640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f070406ae40 0x7f070406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.695+0000 7f0715ffb640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] conn(0x7f0704073060 0x7f0704075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.695+0000 7f0715ffb640 1 -- 172.21.15.60:0/3897117002 --> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0704075b30 con 0x7f0704073060 2023-11-18T20:42:42.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.695+0000 7fabfaffd640 1 -- 172.21.15.60:0/2355923215 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0403e980 con 0x7fac0c0722b0 2023-11-18T20:42:42.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.695+0000 7f0725e12640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] conn(0x7f0704073060 0x7f0704075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.695+0000 7f0724e10640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f070406ae40 0x7f070406d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07180e77a0 tx=0x7f0710008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.695+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0c164080 con 0x7fac0c0722b0 2023-11-18T20:42:42.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.695+0000 7f0715ffb640 1 -- 172.21.15.60:0/3897117002 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0720128590 con 0x7f07180eb030 2023-11-18T20:42:42.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.695+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0c153e80 con 0x7fac0c0722b0 2023-11-18T20:42:42.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.696+0000 7f0f137fe640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0f0006af10 0x7f0f0006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.696+0000 7f0725e12640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] conn(0x7f0704073060 0x7f0704075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.696+0000 7f0f13fff640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] conn(0x7f0f00073130 0x7f0f00075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.696+0000 7fabf8ff9640 1 -- 172.21.15.60:0/2355923215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac0c0cd3d0 con 0x7fac0c0722b0 2023-11-18T20:42:42.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.696+0000 7f0f13fff640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] conn(0x7f0f00073130 0x7f0f00075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.696+0000 7f0f137fe640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0f0006af10 0x7f0f0006d3d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c001ef0 tx=0x7f0f0c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.697+0000 7f0715ffb640 1 -- 172.21.15.60:0/3897117002 <== osd.6 v2:172.21.15.100:6802/1390121871 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0704075b30 con 0x7f0704073060 2023-11-18T20:42:42.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.698+0000 7f0efbfff640 1 -- 172.21.15.60:0/3073393643 <== osd.4 v2:172.21.15.100:6800/454905857 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0f00075c00 con 0x7f0f00073130 2023-11-18T20:42:42.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.698+0000 7fabfaffd640 1 -- 172.21.15.60:0/2355923215 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac0403eae0 con 0x7fac0c0722b0 2023-11-18T20:42:42.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.698+0000 7fabfaffd640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fabec06af10 0x7fabec06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.699+0000 7fabfaffd640 1 -- 172.21.15.60:0/2355923215 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac040ac810 con 0x7fac0c0722b0 2023-11-18T20:42:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.699+0000 7fabfaffd640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] conn(0x7fabec073130 0x7fabec075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.699+0000 7fac0a59c640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fabec06af10 0x7fabec06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.699+0000 7fabfaffd640 1 -- 172.21.15.60:0/2355923215 --> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabec075c00 con 0x7fabec073130 2023-11-18T20:42:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.699+0000 7fabfaffd640 1 -- 172.21.15.60:0/2355923215 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac040acbc0 con 0x7fac0c0722b0 2023-11-18T20:42:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.700+0000 7fac0ad9d640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] conn(0x7fabec073130 0x7fabec075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.700+0000 7fac0a59c640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fabec06af10 0x7fabec06d3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fabfc007fd0 tx=0x7fabfc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.700+0000 7fac0ad9d640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] conn(0x7fabec073130 0x7fabec075550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.701+0000 7fabfaffd640 1 -- 172.21.15.60:0/2355923215 <== osd.5 v2:172.21.15.100:6816/4243789067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fabec075c00 con 0x7fabec073130 2023-11-18T20:42:42.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.715+0000 7f9f44b8c640 1 -- 172.21.15.60:0/1490632692 --> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9f38188530 con 0x7f9f240730b0 2023-11-18T20:42:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.716+0000 7f9f2bfff640 1 -- 172.21.15.60:0/1490632692 <== osd.2 v2:172.21.15.60:6808/479181725 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9f38188530 con 0x7f9f240730b0 2023-11-18T20:42:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.716+0000 7f9f29ffb640 1 -- 172.21.15.60:0/1490632692 >> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] conn(0x7f9f240730b0 msgr2=0x7f9f240754d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.716+0000 7f9f29ffb640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] conn(0x7f9f240730b0 0x7f9f240754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.717+0000 7f9f29ffb640 1 -- 172.21.15.60:0/1490632692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f2406ae90 msgr2=0x7f9f2406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.717+0000 7f9f29ffb640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f2406ae90 0x7f9f2406d350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9f380ce650 tx=0x7f9f34006bd0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.717+0000 7f9f29ffb640 1 -- 172.21.15.60:0/1490632692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38008140 msgr2=0x7f9f38035b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.717+0000 7f9f29ffb640 1 --2- 172.21.15.60:0/1490632692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f38008140 0x7f9f38035b10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0097c0 tx=0x7f9f2c004730 comp rx=0 tx=0).stop 2023-11-18T20:42:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.717+0000 7f9f3ffff640 1 -- 172.21.15.60:0/1490632692 reap_dead start 2023-11-18T20:42:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.717+0000 7f9f29ffb640 1 -- 172.21.15.60:0/1490632692 shutdown_connections 2023-11-18T20:42:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.717+0000 7f9f29ffb640 1 -- 172.21.15.60:0/1490632692 >> 172.21.15.60:0/1490632692 conn(0x7f9f38055320 msgr2=0x7f9f38048c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.717+0000 7f9f29ffb640 1 -- 172.21.15.60:0/1490632692 shutdown_connections 2023-11-18T20:42:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.717+0000 7f9f29ffb640 1 -- 172.21.15.60:0/1490632692 wait complete. 2023-11-18T20:42:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.720+0000 7f94a27fc640 1 -- 172.21.15.60:0/1138456521 --> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94b004c300 con 0x7f949c073090 2023-11-18T20:42:42.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.720+0000 7f94b4ff9640 1 -- 172.21.15.60:0/1138456521 <== osd.0 v2:172.21.15.60:6800/2078754623 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f94b004c300 con 0x7f949c073090 2023-11-18T20:42:42.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.720+0000 7f57d77fe640 1 -- 172.21.15.60:0/202457891 --> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57d0002cb0 con 0x7f57d80730b0 2023-11-18T20:42:42.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.721+0000 7f57e9ffb640 1 -- 172.21.15.60:0/202457891 <== osd.3 v2:172.21.15.60:6824/4034119923 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f57d0002cb0 con 0x7f57d80730b0 2023-11-18T20:42:42.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 >> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] conn(0x7f57d80730b0 msgr2=0x7f57d80754d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f94a27fc640 1 -- 172.21.15.60:0/1138456521 >> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] conn(0x7f949c073090 msgr2=0x7f949c0754b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f94a27fc640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] conn(0x7f949c073090 0x7f949c0754b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f5b23e15640 1 -- 172.21.15.60:0/2100125991 --> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5b140070b0 con 0x7f5b00072ef0 2023-11-18T20:42:42.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f57fba4b640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] conn(0x7f57d80730b0 0x7f57d80754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f94a27fc640 1 -- 172.21.15.60:0/1138456521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f949c06ae70 msgr2=0x7f949c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f94a27fc640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f949c06ae70 0x7f949c06d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f94a403a000 tx=0x7f94a4032000 comp rx=0 tx=0).stop 2023-11-18T20:42:42.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f94a27fc640 1 -- 172.21.15.60:0/1138456521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b0053c70 msgr2=0x7f94b00d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f94a27fc640 1 --2- 172.21.15.60:0/1138456521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94b0053c70 0x7f94b00d13e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c410 tx=0x7f94ac00c8d0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f94bc884640 1 -- 172.21.15.60:0/1138456521 reap_dead start 2023-11-18T20:42:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f94a27fc640 1 -- 172.21.15.60:0/1138456521 shutdown_connections 2023-11-18T20:42:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f94a27fc640 1 -- 172.21.15.60:0/1138456521 >> 172.21.15.60:0/1138456521 conn(0x7f94b0009fb0 msgr2=0x7f94b00d4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57d806ae90 msgr2=0x7f57d806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f57fba4b640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57d806ae90 0x7f57d806d350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57f0001050 tx=0x7f57f0006000 comp rx=0 tx=0).stop 2023-11-18T20:42:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f414e820 msgr2=0x7f57f414eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f5b127fc640 1 -- 172.21.15.60:0/2100125991 <== osd.1 v2:172.21.15.60:6809/3839540220 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5b18009e90 con 0x7f5b00072ef0 2023-11-18T20:42:42.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f57fba4b640 1 --2- 172.21.15.60:0/202457891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57f414e820 0x7f57f414eca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f57f40cd230 tx=0x7f57ec00d900 comp rx=0 tx=0).stop 2023-11-18T20:42:42.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f57f9fe8640 1 -- 172.21.15.60:0/202457891 reap_dead start 2023-11-18T20:42:42.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 shutdown_connections 2023-11-18T20:42:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.722+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 >> 172.21.15.60:0/202457891 conn(0x7f57f40b0d60 msgr2=0x7f57f40cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.723+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 shutdown_connections 2023-11-18T20:42:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.723+0000 7f57fba4b640 1 -- 172.21.15.60:0/202457891 wait complete. 2023-11-18T20:42:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.723+0000 7f94a27fc640 1 -- 172.21.15.60:0/1138456521 shutdown_connections 2023-11-18T20:42:42.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.723+0000 7f94a27fc640 1 -- 172.21.15.60:0/1138456521 wait complete. 2023-11-18T20:42:42.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.724+0000 7f5afffff640 1 -- 172.21.15.60:0/2100125991 >> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] conn(0x7f5b00072ef0 msgr2=0x7f5b00075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.724+0000 7f5afffff640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] conn(0x7f5b00072ef0 0x7f5b00075310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.724+0000 7f5afffff640 1 -- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5b0006ae40 msgr2=0x7f5b0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.724+0000 7f5afffff640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5b0006ae40 0x7f5b0006d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c036000 tx=0x7f5b0c008030 comp rx=0 tx=0).stop 2023-11-18T20:42:42.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.724+0000 7f5afffff640 1 -- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b1404fa90 msgr2=0x7f5b140466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.725+0000 7f5afffff640 1 --2- 172.21.15.60:0/2100125991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b1404fa90 0x7f5b140466b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c064180 tx=0x7f5b1c0678b0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.725+0000 7f5b223b2640 1 -- 172.21.15.60:0/2100125991 reap_dead start 2023-11-18T20:42:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.725+0000 7f5afffff640 1 -- 172.21.15.60:0/2100125991 shutdown_connections 2023-11-18T20:42:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.725+0000 7f5afffff640 1 -- 172.21.15.60:0/2100125991 >> 172.21.15.60:0/2100125991 conn(0x7f5b14009fb0 msgr2=0x7f5b14036590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.725+0000 7f5afffff640 1 -- 172.21.15.60:0/2100125991 shutdown_connections 2023-11-18T20:42:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.725+0000 7f5afffff640 1 -- 172.21.15.60:0/2100125991 wait complete. 2023-11-18T20:42:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.726+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 --> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f58e4162720 con 0x7f58c8073130 2023-11-18T20:42:42.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.727+0000 7f58daffd640 1 -- 172.21.15.60:0/1716408051 <== osd.7 v2:172.21.15.100:6824/1780039900 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f58dc02cd50 con 0x7f58c8073130 2023-11-18T20:42:42.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.728+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] conn(0x7f58c8073130 msgr2=0x7f58c8075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.728+0000 7f58eb367640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] conn(0x7f58c8073130 0x7f58c8075550 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.728+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c806af10 msgr2=0x7f58c806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.728+0000 7f58eb367640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58c806af10 0x7f58c806d3d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009a40 tx=0x7f58e0008040 comp rx=0 tx=0).stop 2023-11-18T20:42:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.728+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e406df00 msgr2=0x7f58e4157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.728+0000 7f58eb367640 1 --2- 172.21.15.60:0/1716408051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e406df00 0x7f58e4157a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f58d400e7e0 tx=0x7f58d400eca0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.728+0000 7f58eab66640 1 -- 172.21.15.60:0/1716408051 reap_dead start 2023-11-18T20:42:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.728+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 shutdown_connections 2023-11-18T20:42:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.728+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 >> 172.21.15.60:0/1716408051 conn(0x7f58e40b0c60 msgr2=0x7f58e41407e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.728+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 shutdown_connections 2023-11-18T20:42:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.729+0000 7f58eb367640 1 -- 172.21.15.60:0/1716408051 wait complete. 2023-11-18T20:42:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.729+0000 7fabf8ff9640 1 -- 172.21.15.60:0/2355923215 --> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fac0c06c5e0 con 0x7fabec073130 2023-11-18T20:42:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.730+0000 7fabfaffd640 1 -- 172.21.15.60:0/2355923215 <== osd.5 v2:172.21.15.100:6816/4243789067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fac0000b560 con 0x7fabec073130 2023-11-18T20:42:42.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] conn(0x7fabec073130 msgr2=0x7fabec075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7fac0b59e640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] conn(0x7fabec073130 0x7fabec075550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fabec06af10 msgr2=0x7fabec06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7fac0b59e640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fabec06af10 0x7fabec06d3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fabfc007fd0 tx=0x7fabfc009040 comp rx=0 tx=0).stop 2023-11-18T20:42:42.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 --> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0718057320 con 0x7f0704073060 2023-11-18T20:42:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac0c0722b0 msgr2=0x7fac0c149fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7fac0b59e640 1 --2- 172.21.15.60:0/2355923215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac0c0722b0 0x7fac0c149fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac04032000 tx=0x7fac04004a40 comp rx=0 tx=0).stop 2023-11-18T20:42:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7fac0ad9d640 1 -- 172.21.15.60:0/2355923215 reap_dead start 2023-11-18T20:42:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 shutdown_connections 2023-11-18T20:42:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 >> 172.21.15.60:0/2355923215 conn(0x7fac0c06a4d0 msgr2=0x7fac0c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7f0715ffb640 1 -- 172.21.15.60:0/3897117002 <== osd.6 v2:172.21.15.100:6802/1390121871 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f071c007b40 con 0x7f0704073060 2023-11-18T20:42:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.731+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 --> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ef4002cb0 con 0x7f0f00073130 2023-11-18T20:42:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.732+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 shutdown_connections 2023-11-18T20:42:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.732+0000 7fac0b59e640 1 -- 172.21.15.60:0/2355923215 wait complete. 2023-11-18T20:42:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.732+0000 7f0efbfff640 1 -- 172.21.15.60:0/3073393643 <== osd.4 v2:172.21.15.100:6800/454905857 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ef4002cb0 con 0x7f0f00073130 2023-11-18T20:42:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.732+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] conn(0x7f0704073060 msgr2=0x7f0704075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.732+0000 7f0726613640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] conn(0x7f0704073060 0x7f0704075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.732+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] conn(0x7f0f00073130 msgr2=0x7f0f00075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.732+0000 7f0f1a190640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] conn(0x7f0f00073130 0x7f0f00075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.732+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f070406ae40 msgr2=0x7f070406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.732+0000 7f0726613640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f070406ae40 0x7f070406d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07180e77a0 tx=0x7f0710008040 comp rx=0 tx=0).stop 2023-11-18T20:42:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07180eb030 msgr2=0x7f071803c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0726613640 1 --2- 172.21.15.60:0/3897117002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07180eb030 0x7f071803c680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f07200681a0 tx=0x7f07200686a0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0725e12640 1 -- 172.21.15.60:0/3897117002 reap_dead start 2023-11-18T20:42:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0f0006af10 msgr2=0x7f0f0006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0f1a190640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0f0006af10 0x7f0f0006d3d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c001ef0 tx=0x7f0f0c009040 comp rx=0 tx=0).stop 2023-11-18T20:42:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 shutdown_connections 2023-11-18T20:42:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 >> 172.21.15.60:0/3897117002 conn(0x7f07180559b0 msgr2=0x7f07180e8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f140b34a0 msgr2=0x7f0f141545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0f1a190640 1 --2- 172.21.15.60:0/3073393643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f140b34a0 0x7f0f141545d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400b3e0 tx=0x7f0f0400b8a0 comp rx=0 tx=0).stop 2023-11-18T20:42:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0f13fff640 1 -- 172.21.15.60:0/3073393643 reap_dead start 2023-11-18T20:42:42.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 shutdown_connections 2023-11-18T20:42:42.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0726613640 1 -- 172.21.15.60:0/3897117002 wait complete. 2023-11-18T20:42:42.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.733+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 shutdown_connections 2023-11-18T20:42:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.734+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 >> 172.21.15.60:0/3073393643 conn(0x7f0f140b0d60 msgr2=0x7f0f14071610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.734+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 shutdown_connections 2023-11-18T20:42:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:42.734+0000 7f0f1a190640 1 -- 172.21.15.60:0/3073393643 wait complete. 2023-11-18T20:42:42.816 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-11-18T20:42:42.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-18T20:42:42.818 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-11-18T20:42:42.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-18T20:42:42.819 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-11-18T20:42:42.819 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-18T20:42:42.820 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-11-18T20:42:42.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-18T20:42:42.821 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-11-18T20:42:42.821 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-18T20:42:42.822 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-11-18T20:42:42.822 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-18T20:42:42.823 INFO:teuthology.orchestra.run.smithi060.stdout:55834574855 2023-11-18T20:42:42.823 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-18T20:42:42.824 INFO:teuthology.orchestra.run.smithi060.stdout:55834574855 2023-11-18T20:42:42.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-18T20:42:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.041+0000 7fadfdf73640 1 Processor -- start 2023-11-18T20:42:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.041+0000 7fadfdf73640 1 -- start start 2023-11-18T20:42:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.040+0000 7f0340d1d640 1 Processor -- start 2023-11-18T20:42:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.040+0000 7f0340d1d640 1 -- start start 2023-11-18T20:42:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.041+0000 7f0340d1d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0334037c40 0x7f0334053d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7fadfdf73640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf00e12d0 0x7fadf003a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7f042ef1d640 1 Processor -- start 2023-11-18T20:42:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7f042ef1d640 1 -- start start 2023-11-18T20:42:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7fadfdf73640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fadf000a420 0x7fadf0055d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7fadfdf73640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf00558e0 con 0x7fadf000ab00 2023-11-18T20:42:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7fadfdf73640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf00e16b0 con 0x7fadf000a420 2023-11-18T20:42:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.041+0000 7f0340d1d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0334052cf0 con 0x7f0334033920 2023-11-18T20:42:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7fadfdf73640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf003a670 con 0x7fadf00e12d0 2023-11-18T20:42:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7f0340d1d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0334052f60 con 0x7f0334038060 2023-11-18T20:42:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7fadfcf71640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf00e12d0 0x7fadf003a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7f0340d1d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0334054970 con 0x7f0334037c40 2023-11-18T20:42:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7f033a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0334033920 0x7f0334100da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39118/0 (socket says 172.21.15.60:39118) 2023-11-18T20:42:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7fadfcf71640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf00e12d0 0x7fadf003a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56212/0 (socket says 172.21.15.60:56212) 2023-11-18T20:42:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7f033a59c640 1 -- 172.21.15.60:0/3939065269 learned_addr learned my addr 172.21.15.60:0/3939065269 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7fadfcf71640 1 -- 172.21.15.60:0/2122442171 learned_addr learned my addr 172.21.15.60:0/2122442171 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7fadfd772640 1 --2- 172.21.15.60:0/2122442171 >> v2:172.21.15.60:3300/0 conn(0x7fadf000a420 0x7fadf0055d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7f90d759e640 1 Processor -- start 2023-11-18T20:42:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7f033959a640 1 -- 172.21.15.60:0/3939065269 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687544702 0 0) 0x7f0334052f60 con 0x7f0334038060 2023-11-18T20:42:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7f033959a640 1 -- 172.21.15.60:0/3939065269 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0324003520 con 0x7f0334038060 2023-11-18T20:42:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7fb3c51e7640 1 Processor -- start 2023-11-18T20:42:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.042+0000 7f0339d9b640 1 --2- 172.21.15.60:0/3939065269 >> v2:172.21.15.60:3300/0 conn(0x7f0334037c40 0x7f0334053d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7fa900190640 1 Processor -- start 2023-11-18T20:42:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f033959a640 1 -- 172.21.15.60:0/3939065269 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174306700 0 0) 0x7f0334052cf0 con 0x7f0334033920 2023-11-18T20:42:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7fac7f55a640 1 Processor -- start 2023-11-18T20:42:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f033959a640 1 -- 172.21.15.60:0/3939065269 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0334052f60 con 0x7f0334033920 2023-11-18T20:42:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f90d759e640 1 -- start start 2023-11-18T20:42:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7fadfcf71640 1 -- 172.21.15.60:0/2122442171 >> v2:172.21.15.60:3300/0 conn(0x7fadf000a420 msgr2=0x7fadf0055d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7fb3c51e7640 1 -- start start 2023-11-18T20:42:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fb3c51e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb3c00b40b0 0x7fb3c00b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fb3c51e7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb3c00b2490 0x7fb3c00b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fb3c51e7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c00cd020 con 0x7fb3c0159210 2023-11-18T20:42:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fb3c51e7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c00b4a70 con 0x7fb3c00b40b0 2023-11-18T20:42:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fb3c51e7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c00cd300 con 0x7fb3c00b2490 2023-11-18T20:42:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7fadfcf71640 1 --2- 172.21.15.60:0/2122442171 >> v2:172.21.15.60:3300/0 conn(0x7fadf000a420 0x7fadf0055d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7fadfcf71640 1 -- 172.21.15.60:0/2122442171 >> v1:172.21.15.60:6789/0 conn(0x7fadf000ab00 legacy=0x7fadf0009d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fb3bf59e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb3c00b2490 0x7fb3c00b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7fa900190640 1 -- start start 2023-11-18T20:42:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fa900190640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f8159c40 0x7fa8f815cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fa900190640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa8f80b38e0 0x7fa8f815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fa900190640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f815dbe0 con 0x7fa8f815a020 2023-11-18T20:42:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fa900190640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f815aa30 con 0x7fa8f80b38e0 2023-11-18T20:42:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fa900190640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f815d340 con 0x7fa8f8159c40 2023-11-18T20:42:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f0339d9b640 1 -- 172.21.15.60:0/3939065269 >> v1:172.21.15.100:6789/0 conn(0x7f0334038060 legacy=0x7f0334054260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f0339d9b640 1 -- 172.21.15.60:0/3939065269 >> v1:172.21.15.60:6789/0 conn(0x7f0334033920 legacy=0x7f0334100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f0339d9b640 1 -- 172.21.15.60:0/3939065269 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0334053240 con 0x7f0334037c40 2023-11-18T20:42:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fa8fd72b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa8f815a020 0x7fa8f80b31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39144/0 (socket says 172.21.15.60:39144) 2023-11-18T20:42:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fa8fd72b640 1 -- 172.21.15.60:0/2038843791 learned_addr learned my addr 172.21.15.60:0/2038843791 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7fac7f55a640 1 -- start start 2023-11-18T20:42:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7fadfcf71640 1 -- 172.21.15.60:0/2122442171 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf0054cd0 con 0x7fadf00e12d0 2023-11-18T20:42:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fadfcf71640 1 --2- 172.21.15.60:0/2122442171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf00e12d0 0x7fadf003a130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fadec009870 tx=0x7fadec02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f6e39ba1d6791b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fadf77fe640 1 -- 172.21.15.60:0/2122442171 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec02e070 con 0x7fadf00e12d0 2023-11-18T20:42:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fadf77fe640 1 -- 172.21.15.60:0/2122442171 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec0315a0 con 0x7fadf00e12d0 2023-11-18T20:42:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fadf77fe640 1 -- 172.21.15.60:0/2122442171 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec031930 con 0x7fadf00e12d0 2023-11-18T20:42:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fb3bed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb3c0159210 0x7fb3c015a2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39138/0 (socket says 172.21.15.60:39138) 2023-11-18T20:42:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fb3bed9d640 1 -- 172.21.15.60:0/2187992745 learned_addr learned my addr 172.21.15.60:0/2187992745 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fa8fdf2c640 1 --2- 172.21.15.60:0/2038843791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f8159c40 0x7fa8f815cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fac7f55a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fac700ead00 0x7fac7004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fac7f55a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fac70006ea0 0x7fac70007310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fac7f55a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac70054330 con 0x7fac7004ffd0 2023-11-18T20:42:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fac7f55a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac70007850 con 0x7fac700ead00 2023-11-18T20:42:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7fac7f55a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac70053720 con 0x7fac70006ea0 2023-11-18T20:42:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fac7d2f6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fac700ead00 0x7fac7004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fac7d2f6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fac700ead00 0x7fac7004fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41204/0 (socket says 172.21.15.60:41204) 2023-11-18T20:42:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fac7d2f6640 1 -- 172.21.15.60:0/1266199355 learned_addr learned my addr 172.21.15.60:0/1266199355 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fac7daf7640 1 --2- 172.21.15.60:0/1266199355 >> v2:172.21.15.100:3300/0 conn(0x7fac70006ea0 0x7fac70007310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fa8fe72d640 1 --2- 172.21.15.60:0/2038843791 >> v2:172.21.15.100:3300/0 conn(0x7fa8f80b38e0 0x7fa8f815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f042ef1d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042814b2e0 0x7f042806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f042ef1d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f042806f7a0 0x7f042806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f042ef1d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042806db30 con 0x7f042806aa10 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f042ef1d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042806ea10 con 0x7f042806f7a0 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.043+0000 7f042ef1d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042806aea0 con 0x7f042814b2e0 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7f0427fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f042806f7a0 0x7f042806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7f0427fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f042806f7a0 0x7f042806a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41194/0 (socket says 172.21.15.60:41194) 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7f0427fff640 1 -- 172.21.15.60:0/4245743566 learned_addr learned my addr 172.21.15.60:0/4245743566 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7f0427fff640 1 -- 172.21.15.60:0/4245743566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042814b2e0 msgr2=0x7f042806d170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7f0427fff640 1 --2- 172.21.15.60:0/4245743566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042814b2e0 0x7f042806d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7f0427fff640 1 -- 172.21.15.60:0/4245743566 >> v1:172.21.15.60:6789/0 conn(0x7f042806aa10 legacy=0x7f04281651a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-18T20:42:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.044+0000 7f0427fff640 1 -- 172.21.15.60:0/4245743566 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042806e340 con 0x7f042806f7a0 2023-11-18T20:42:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7f0427fff640 1 --2- 172.21.15.60:0/4245743566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042806f7a0 0x7f042806a4d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f041c009ad0 tx=0x7f041c02ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9ec353efd9d5d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f90d759e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f90d00392a0 0x7f90d00e9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f04277fe640 1 -- 172.21.15.60:0/4245743566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c037680 con 0x7f042806f7a0 2023-11-18T20:42:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7fadfdf73640 1 -- 172.21.15.60:0/2122442171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf00e12d0 msgr2=0x7fadf003a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2122442171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf00e12d0 0x7fadf003a130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fadec009870 tx=0x7fadec02b840 comp rx=0 tx=0).stop 2023-11-18T20:42:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f90d759e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d0006870 con 0x7f90d004fa60 2023-11-18T20:42:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f90d759e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d0006ac0 con 0x7f90d00df3f0 2023-11-18T20:42:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f90d759e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d0007740 con 0x7f90d00392a0 2023-11-18T20:42:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fac7d2f6640 1 -- 172.21.15.60:0/1266199355 >> v2:172.21.15.100:3300/0 conn(0x7fac70006ea0 msgr2=0x7fac70007310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fac7d2f6640 1 --2- 172.21.15.60:0/1266199355 >> v2:172.21.15.100:3300/0 conn(0x7fac70006ea0 0x7fac70007310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fac7d2f6640 1 -- 172.21.15.60:0/1266199355 >> v1:172.21.15.60:6789/0 conn(0x7fac7004ffd0 legacy=0x7fac70037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f90d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f90d00392a0 0x7f90d00e9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f90d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f90d00392a0 0x7f90d00e9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41212/0 (socket says 172.21.15.60:41212) 2023-11-18T20:42:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f90d6d9d640 1 -- 172.21.15.60:0/1955178935 learned_addr learned my addr 172.21.15.60:0/1955178935 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fac7d2f6640 1 -- 172.21.15.60:0/1266199355 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac70053a00 con 0x7fac700ead00 2023-11-18T20:42:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7f0339d9b640 1 --2- 172.21.15.60:0/3939065269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0334037c40 0x7f0334053d20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f032c0097a0 tx=0x7f032c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0fa4aed3cc8b97 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f033959a640 1 -- 172.21.15.60:0/3939065269 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300015a0 con 0x7f0334037c40 2023-11-18T20:42:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f033959a640 1 -- 172.21.15.60:0/3939065269 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c02e050 con 0x7f0334037c40 2023-11-18T20:42:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f033959a640 1 -- 172.21.15.60:0/3939065269 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c004250 con 0x7f0334037c40 2023-11-18T20:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7fadfdf73640 1 -- 172.21.15.60:0/2122442171 shutdown_connections 2023-11-18T20:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2122442171 >> v2:172.21.15.60:3300/0 conn(0x7fadf000a420 0x7fadf0055d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2122442171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf00e12d0 0x7fadf003a130 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7fadfdf73640 1 -- 172.21.15.60:0/2122442171 >> 172.21.15.60:0/2122442171 conn(0x7fadf0056c90 msgr2=0x7fadf00570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7fadfdf73640 1 -- 172.21.15.60:0/2122442171 shutdown_connections 2023-11-18T20:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fb3bf59e640 1 -- 172.21.15.60:0/2187992745 >> v2:172.21.15.60:3300/0 conn(0x7fb3c00b40b0 msgr2=0x7fb3c00b4530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-18T20:42:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7fadfdf73640 1 -- 172.21.15.60:0/2122442171 wait complete. 2023-11-18T20:42:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fa8fe72d640 1 -- 172.21.15.60:0/2038843791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f8159c40 msgr2=0x7fa8f815cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fa8fe72d640 1 --2- 172.21.15.60:0/2038843791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f8159c40 0x7fa8f815cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fa8fe72d640 1 -- 172.21.15.60:0/2038843791 >> v1:172.21.15.60:6789/0 conn(0x7fa8f815a020 legacy=0x7fa8f80b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fa8fe72d640 1 -- 172.21.15.60:0/2038843791 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f815d620 con 0x7fa8f80b38e0 2023-11-18T20:42:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7f04277fe640 1 -- 172.21.15.60:0/4245743566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041c0377e0 con 0x7f042806f7a0 2023-11-18T20:42:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fb3bf59e640 1 --2- 172.21.15.60:0/2187992745 >> v2:172.21.15.60:3300/0 conn(0x7fb3c00b40b0 0x7fb3c00b4530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fb3bf59e640 1 -- 172.21.15.60:0/2187992745 >> v1:172.21.15.60:6789/0 conn(0x7fb3c0159210 legacy=0x7fb3c015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.045+0000 7fb3bf59e640 1 -- 172.21.15.60:0/2187992745 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c00cd5e0 con 0x7fb3c00b2490 2023-11-18T20:42:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.046+0000 7fa8fe72d640 1 --2- 172.21.15.60:0/2038843791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f80b38e0 0x7fa8f815aec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400a940 tx=0x7fa8f402cd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fcfc56c44165b9e server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f04277fe640 1 -- 172.21.15.60:0/4245743566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c037b70 con 0x7f042806f7a0 2023-11-18T20:42:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7fa8fcf2a640 1 -- 172.21.15.60:0/2038843791 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f402db40 con 0x7fa8f80b38e0 2023-11-18T20:42:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7fb3bf59e640 1 --2- 172.21.15.60:0/2187992745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3c00b2490 0x7fb3c00b2920 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00aeb20 tx=0x7fb3b002db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2620a95dbc1d3cf server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7fa8fcf2a640 1 -- 172.21.15.60:0/2038843791 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f402dca0 con 0x7fa8f80b38e0 2023-11-18T20:42:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7fa8fcf2a640 1 -- 172.21.15.60:0/2038843791 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4035660 con 0x7fa8f80b38e0 2023-11-18T20:42:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f90d6d9d640 1 -- 172.21.15.60:0/1955178935 >> v1:172.21.15.100:6789/0 conn(0x7f90d00df3f0 legacy=0x7f90d0053230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f90d6d9d640 1 -- 172.21.15.60:0/1955178935 >> v1:172.21.15.60:6789/0 conn(0x7f90d004fa60 legacy=0x7f90d004fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f90d6d9d640 1 -- 172.21.15.60:0/1955178935 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0006da0 con 0x7f90d00392a0 2023-11-18T20:42:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7fb3bdd9b640 1 -- 172.21.15.60:0/2187992745 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0008df0 con 0x7fb3c00b2490 2023-11-18T20:42:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f0340d1d640 1 -- 172.21.15.60:0/3939065269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0334037c40 msgr2=0x7f0334053d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f0340d1d640 1 --2- 172.21.15.60:0/3939065269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0334037c40 0x7f0334053d20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f032c0097a0 tx=0x7f032c02b7e0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7fb3bdd9b640 1 -- 172.21.15.60:0/2187992745 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b000a0d0 con 0x7fb3c00b2490 2023-11-18T20:42:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7fb3bdd9b640 1 -- 172.21.15.60:0/2187992745 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000a460 con 0x7fb3c00b2490 2023-11-18T20:42:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f0340d1d640 1 -- 172.21.15.60:0/3939065269 shutdown_connections 2023-11-18T20:42:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f0340d1d640 1 --2- 172.21.15.60:0/3939065269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0334037c40 0x7f0334053d20 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f032c0097a0 tx=0x7f032c02b7e0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f0340d1d640 1 -- 172.21.15.60:0/3939065269 >> 172.21.15.60:0/3939065269 conn(0x7f0334009fb0 msgr2=0x7f033400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7fac7d2f6640 1 --2- 172.21.15.60:0/1266199355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac700ead00 0x7fac7004fa90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac700528a0 tx=0x7fac6802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d57077a03fe0433 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f042ef1d640 1 -- 172.21.15.60:0/4245743566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042806f7a0 msgr2=0x7f042806a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f042ef1d640 1 --2- 172.21.15.60:0/4245743566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042806f7a0 0x7f042806a4d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f041c009ad0 tx=0x7f041c02ba60 comp rx=0 tx=0).stop 2023-11-18T20:42:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f042ef1d640 1 -- 172.21.15.60:0/4245743566 shutdown_connections 2023-11-18T20:42:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f042ef1d640 1 --2- 172.21.15.60:0/4245743566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042806f7a0 0x7f042806a4d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f042ef1d640 1 --2- 172.21.15.60:0/4245743566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042814b2e0 0x7f042806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f042ef1d640 1 -- 172.21.15.60:0/4245743566 >> 172.21.15.60:0/4245743566 conn(0x7f04280cfd80 msgr2=0x7f04280d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7f0340d1d640 1 -- 172.21.15.60:0/3939065269 shutdown_connections 2023-11-18T20:42:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7f0340d1d640 1 -- 172.21.15.60:0/3939065269 wait complete. 2023-11-18T20:42:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f2b705b0640 1 Processor -- start 2023-11-18T20:42:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.047+0000 7f2b705b0640 1 -- start start 2023-11-18T20:42:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fac6ffff640 1 -- 172.21.15.60:0/1266199355 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68031870 con 0x7fac700ead00 2023-11-18T20:42:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fac6ffff640 1 -- 172.21.15.60:0/1266199355 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac680319d0 con 0x7fac700ead00 2023-11-18T20:42:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fac6ffff640 1 -- 172.21.15.60:0/1266199355 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68031d60 con 0x7fac700ead00 2023-11-18T20:42:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fa900190640 1 -- 172.21.15.60:0/2038843791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f80b38e0 msgr2=0x7fa8f815aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fa900190640 1 --2- 172.21.15.60:0/2038843791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f80b38e0 0x7fa8f815aec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400a940 tx=0x7fa8f402cd90 comp rx=0 tx=0).stop 2023-11-18T20:42:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7f042ef1d640 1 -- 172.21.15.60:0/4245743566 shutdown_connections 2023-11-18T20:42:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fa900190640 1 -- 172.21.15.60:0/2038843791 shutdown_connections 2023-11-18T20:42:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fa900190640 1 --2- 172.21.15.60:0/2038843791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f80b38e0 0x7fa8f815aec0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fb3c51e7640 1 -- 172.21.15.60:0/2187992745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3c00b2490 msgr2=0x7fb3c00b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fb3c51e7640 1 --2- 172.21.15.60:0/2187992745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3c00b2490 0x7fb3c00b2920 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00aeb20 tx=0x7fb3b002db10 comp rx=0 tx=0).stop 2023-11-18T20:42:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fa900190640 1 --2- 172.21.15.60:0/2038843791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f8159c40 0x7fa8f815cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fa900190640 1 -- 172.21.15.60:0/2038843791 >> 172.21.15.60:0/2038843791 conn(0x7fa8f80b0b10 msgr2=0x7fa8f80b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fadfdf73640 1 Processor -- start 2023-11-18T20:42:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fb3c51e7640 1 -- 172.21.15.60:0/2187992745 shutdown_connections 2023-11-18T20:42:43.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fb3c51e7640 1 --2- 172.21.15.60:0/2187992745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3c00b2490 0x7fb3c00b2920 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fb3c51e7640 1 --2- 172.21.15.60:0/2187992745 >> v2:172.21.15.60:3300/0 conn(0x7fb3c00b40b0 0x7fb3c00b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fb3c51e7640 1 -- 172.21.15.60:0/2187992745 >> 172.21.15.60:0/2187992745 conn(0x7fb3c00b0d60 msgr2=0x7fb3c00b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fadfdf73640 1 -- start start 2023-11-18T20:42:43.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7f90d6d9d640 1 --2- 172.21.15.60:0/1955178935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00392a0 0x7f90d00e9d10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f90d80730e0 tx=0x7f90d80a5dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a317a2fe40ffe8 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fa900190640 1 -- 172.21.15.60:0/2038843791 shutdown_connections 2023-11-18T20:42:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7f90d559a640 1 -- 172.21.15.60:0/1955178935 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80a6c40 con 0x7f90d00392a0 2023-11-18T20:42:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7f90d559a640 1 -- 172.21.15.60:0/1955178935 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d80a6e20 con 0x7f90d00392a0 2023-11-18T20:42:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7f90d559a640 1 -- 172.21.15.60:0/1955178935 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80ae640 con 0x7f90d00392a0 2023-11-18T20:42:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7fb3c51e7640 1 -- 172.21.15.60:0/2187992745 shutdown_connections 2023-11-18T20:42:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7f2b705b0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b681627b0 0x7f2b680cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7f2b705b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b680d0e30 con 0x7f2b6806c0e0 2023-11-18T20:42:43.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7fac7f55a640 1 -- 172.21.15.60:0/1266199355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac700ead00 msgr2=0x7fac7004fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7fac7f55a640 1 --2- 172.21.15.60:0/1266199355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac700ead00 0x7fac7004fa90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac700528a0 tx=0x7fac6802b910 comp rx=0 tx=0).stop 2023-11-18T20:42:43.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7f2b705b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b680d4070 con 0x7f2b680cd9b0 2023-11-18T20:42:43.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f90d759e640 1 -- 172.21.15.60:0/1955178935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00392a0 msgr2=0x7f90d00e9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7f0340d1d640 1 Processor -- start 2023-11-18T20:42:43.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f0340d1d640 1 -- start start 2023-11-18T20:42:43.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.048+0000 7fa900190640 1 -- 172.21.15.60:0/2038843791 wait complete. 2023-11-18T20:42:43.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f90d759e640 1 --2- 172.21.15.60:0/1955178935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00392a0 0x7f90d00e9d10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f90d80730e0 tx=0x7f90d80a5dd0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7f042ef1d640 1 -- 172.21.15.60:0/4245743566 wait complete. 2023-11-18T20:42:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fadfdf73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf000a420 0x7fadf00d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7fb3c51e7640 1 -- 172.21.15.60:0/2187992745 wait complete. 2023-11-18T20:42:43.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fadfcf71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf000a420 0x7fadf00d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fadfcf71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf000a420 0x7fadf00d5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41226/0 (socket says 172.21.15.60:41226) 2023-11-18T20:42:43.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fadfcf71640 1 -- 172.21.15.60:0/2034891441 learned_addr learned my addr 172.21.15.60:0/2034891441 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf000ab00 0x7fadf01744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fac7f55a640 1 -- 172.21.15.60:0/1266199355 shutdown_connections 2023-11-18T20:42:43.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fac7f55a640 1 --2- 172.21.15.60:0/1266199355 >> v2:172.21.15.100:3300/0 conn(0x7fac70006ea0 0x7fac70007310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fac7f55a640 1 --2- 172.21.15.60:0/1266199355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac700ead00 0x7fac7004fa90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fac7f55a640 1 -- 172.21.15.60:0/1266199355 >> 172.21.15.60:0/1266199355 conn(0x7fac70009fb0 msgr2=0x7fac7000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.049+0000 7f2b705b0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b68165360 con 0x7f2b681627b0 2023-11-18T20:42:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f2b6e34c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2b680cd9b0 0x7f2b6806b9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:53044/0 (socket says 172.21.15.60:53044) 2023-11-18T20:42:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f2b6e34c640 1 -- 172.21.15.60:0/986339596 learned_addr learned my addr 172.21.15.60:0/986339596 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f2b6eb4d640 1 --2- 172.21.15.60:0/986339596 >> v2:172.21.15.100:3300/0 conn(0x7f2b681627b0 0x7f2b680cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf00e12d0 0x7fadf0174a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f90d759e640 1 -- 172.21.15.60:0/1955178935 shutdown_connections 2023-11-18T20:42:43.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f90d759e640 1 --2- 172.21.15.60:0/1955178935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d00392a0 0x7f90d00e9d10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359870490 0 0) 0x7f2b680d0e30 con 0x7f2b6806c0e0 2023-11-18T20:42:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b54003520 con 0x7f2b6806c0e0 2023-11-18T20:42:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7f90d759e640 1 -- 172.21.15.60:0/1955178935 >> 172.21.15.60:0/1955178935 conn(0x7f90d0009f80 msgr2=0x7f90d000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf0175030 con 0x7fadf000a420 2023-11-18T20:42:43.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf0175310 con 0x7fadf00e12d0 2023-11-18T20:42:43.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf01755f0 con 0x7fadf000ab00 2023-11-18T20:42:43.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f0340d1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0334033920 0x7f03340ef030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f0340d1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0334038060 0x7f033404b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f0340d1d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033404b650 0x7f03340495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f0340d1d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03340ef570 con 0x7f0334038060 2023-11-18T20:42:43.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f0340d1d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03340ec9a0 con 0x7f0334033920 2023-11-18T20:42:43.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f0340d1d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033404ac80 con 0x7f033404b650 2023-11-18T20:42:43.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.050+0000 7fadf7fff640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf000ab00 0x7fadf01744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fadfd772640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf00e12d0 0x7fadf0174a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fa900190640 1 Processor -- start 2023-11-18T20:42:43.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fac7f55a640 1 -- 172.21.15.60:0/1266199355 shutdown_connections 2023-11-18T20:42:43.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f033ad9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033404b650 0x7f03340495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f033ad9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033404b650 0x7f03340495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56234/0 (socket says 172.21.15.60:56234) 2023-11-18T20:42:43.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f033ad9d640 1 -- 172.21.15.60:0/3667019342 learned_addr learned my addr 172.21.15.60:0/3667019342 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fadfcf71640 1 -- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf000ab00 msgr2=0x7fadf01744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f0339d9b640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0334038060 0x7f033404b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fadfcf71640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf000ab00 0x7fadf01744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fadfcf71640 1 -- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf00e12d0 msgr2=0x7fadf0174a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fa900190640 1 -- start start 2023-11-18T20:42:43.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fadfcf71640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf00e12d0 0x7fadf0174a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fb3c51e7640 1 Processor -- start 2023-11-18T20:42:43.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f033ad9d640 1 -- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0334033920 msgr2=0x7f03340ef030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-18T20:42:43.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f033ad9d640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0334033920 0x7f03340ef030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f033ad9d640 1 -- 172.21.15.60:0/3667019342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0334038060 msgr2=0x7f033404b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f033ad9d640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0334038060 0x7f033404b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f033ad9d640 1 -- 172.21.15.60:0/3667019342 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c009400 con 0x7f033404b650 2023-11-18T20:42:43.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7fb3c51e7640 1 -- start start 2023-11-18T20:42:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fadfcf71640 1 -- 172.21.15.60:0/2034891441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec0094d0 con 0x7fadf000a420 2023-11-18T20:42:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fadf7fff640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf000ab00 0x7fadf01744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:43.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7fadfcf71640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf000a420 0x7fadf00d5b00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fadec0090b0 tx=0x7fadec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f033ad9d640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033404b650 0x7f03340495b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f033c079980 tx=0x7f033c0677f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f90d759e640 1 -- 172.21.15.60:0/1955178935 shutdown_connections 2023-11-18T20:42:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7fadf57fa640 1 -- 172.21.15.60:0/2034891441 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec032a00 con 0x7fadf000a420 2023-11-18T20:42:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f032affd640 1 -- 172.21.15.60:0/3667019342 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c07d030 con 0x7f033404b650 2023-11-18T20:42:43.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f032affd640 1 -- 172.21.15.60:0/3667019342 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c06aa60 con 0x7f033404b650 2023-11-18T20:42:43.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f032affd640 1 -- 172.21.15.60:0/3667019342 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c06ae50 con 0x7f033404b650 2023-11-18T20:42:43.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f90d759e640 1 -- 172.21.15.60:0/1955178935 wait complete. 2023-11-18T20:42:43.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7fadf57fa640 1 -- 172.21.15.60:0/2034891441 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec032b60 con 0x7fadf000a420 2023-11-18T20:42:43.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa900190640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f80b38e0 0x7fa8f8140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa900190640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f8159c40 0x7fa8f80c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa900190640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f815a020 0x7fa8f8158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f0340d1d640 1 -- 172.21.15.60:0/3667019342 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033404a070 con 0x7f033404b650 2023-11-18T20:42:43.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa900190640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f8156d20 con 0x7fa8f80b38e0 2023-11-18T20:42:43.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa900190640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f8141b50 con 0x7fa8f8159c40 2023-11-18T20:42:43.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa900190640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f8141e30 con 0x7fa8f815a020 2023-11-18T20:42:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa8fe72d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f815a020 0x7fa8f8158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa8fdf2c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f80b38e0 0x7fa8f8140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa8fe72d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f815a020 0x7fa8f8158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56244/0 (socket says 172.21.15.60:56244) 2023-11-18T20:42:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa8fe72d640 1 -- 172.21.15.60:0/627579437 learned_addr learned my addr 172.21.15.60:0/627579437 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fb3c51e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3c00b2490 0x7fb3c01f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7fac7f55a640 1 -- 172.21.15.60:0/1266199355 wait complete. 2023-11-18T20:42:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fac7f55a640 1 Processor -- start 2023-11-18T20:42:43.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f042ef1d640 1 Processor -- start 2023-11-18T20:42:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f042ef1d640 1 -- start start 2023-11-18T20:42:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f042ef1d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042806aa10 0x7f042815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f042ef1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042806f7a0 0x7f042815d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f042ef1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042814b2e0 0x7f04281fa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f042ef1d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042815f6a0 con 0x7f042806f7a0 2023-11-18T20:42:43.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f042ef1d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04281fb550 con 0x7f042814b2e0 2023-11-18T20:42:43.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f042ef1d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04281faf90 con 0x7f042806aa10 2023-11-18T20:42:43.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042ccb9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042806aa10 0x7f042815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042ccb9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042806aa10 0x7f042815ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56240/0 (socket says 172.21.15.60:56240) 2023-11-18T20:42:43.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042ccb9640 1 -- 172.21.15.60:0/1885712610 learned_addr learned my addr 172.21.15.60:0/1885712610 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f0427fff640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042806f7a0 0x7f042815d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fb3c51e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3c00b40b0 0x7fb3c0070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042d4ba640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042814b2e0 0x7f04281fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042ccb9640 1 -- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042814b2e0 msgr2=0x7f04281fa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042ccb9640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042814b2e0 0x7f04281fa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042ccb9640 1 -- 172.21.15.60:0/1885712610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042806f7a0 msgr2=0x7f042815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042ccb9640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042806f7a0 0x7f042815d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042ccb9640 1 -- 172.21.15.60:0/1885712610 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0094d0 con 0x7f042806aa10 2023-11-18T20:42:43.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f0340d1d640 1 -- 172.21.15.60:0/3667019342 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033404a630 con 0x7f033404b650 2023-11-18T20:42:43.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fb3c51e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3c0159210 0x7fb3c0071150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042d4ba640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042814b2e0 0x7f04281fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:43.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fa8fd72b640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f8159c40 0x7fa8f80c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fadf57fa640 1 -- 172.21.15.60:0/2034891441 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec0453f0 con 0x7fadf000a420 2023-11-18T20:42:43.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf01758d0 con 0x7fadf000a420 2023-11-18T20:42:43.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf00d7bd0 con 0x7fadf000a420 2023-11-18T20:42:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7f0339d9b640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0334038060 0x7f033404b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fb3be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3c00b40b0 0x7fb3c0070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fb3be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3c00b40b0 0x7fb3c0070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49796/0 (socket says 172.21.15.60:49796) 2023-11-18T20:42:43.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fb3be59c640 1 -- 172.21.15.60:0/1100560123 learned_addr learned my addr 172.21.15.60:0/1100560123 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fb3bf59e640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3c0159210 0x7fb3c0071150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c0153330 con 0x7fb3c00b2490 2023-11-18T20:42:43.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c01545e0 con 0x7fb3c00b40b0 2023-11-18T20:42:43.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c01f8040 con 0x7fb3c0159210 2023-11-18T20:42:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7f042ccb9640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042806aa10 0x7f042815ccb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f041400b3e0 tx=0x7f041400b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fb3bed9d640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3c00b2490 0x7fb3c01f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607018157 0 0) 0x7f2b680d4070 con 0x7f2b680cd9b0 2023-11-18T20:42:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b680d0e30 con 0x7f2b680cd9b0 2023-11-18T20:42:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3609074126 0 0) 0x7f2b54003520 con 0x7f2b6806c0e0 2023-11-18T20:42:43.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 >> v2:172.21.15.100:3300/0 conn(0x7f2b681627b0 msgr2=0x7f2b680cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:43.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 --2- 172.21.15.60:0/986339596 >> v2:172.21.15.100:3300/0 conn(0x7f2b681627b0 0x7f2b680cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 >> v1:172.21.15.100:6789/0 conn(0x7f2b680cd9b0 legacy=0x7f2b6806b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68162bc0 con 0x7f2b6806c0e0 2023-11-18T20:42:43.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f2b600019f0 con 0x7f2b6806c0e0 2023-11-18T20:42:43.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 >> v1:172.21.15.60:6789/0 conn(0x7f2b6806c0e0 legacy=0x7f2b681620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 --2- 172.21.15.60:0/986339596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 0x7f2b54003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 --2- 172.21.15.60:0/986339596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b540047c0 0x7f2b54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 --2- 172.21.15.60:0/986339596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b54005180 0x7f2b5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68165360 con 0x7f2b54003b00 2023-11-18T20:42:43.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5401d010 con 0x7f2b540047c0 2023-11-18T20:42:43.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5401d1d0 con 0x7f2b54005180 2023-11-18T20:42:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6e34c640 1 --2- 172.21.15.60:0/986339596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 0x7f2b54003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.051+0000 7f2b6db4b640 1 --2- 172.21.15.60:0/986339596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b540047c0 0x7f2b54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f2b6e34c640 1 -- 172.21.15.60:0/986339596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b54005180 msgr2=0x7f2b5400b980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-18T20:42:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f2b6e34c640 1 --2- 172.21.15.60:0/986339596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b54005180 0x7f2b5400b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f2b6e34c640 1 -- 172.21.15.60:0/986339596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b540047c0 msgr2=0x7f2b54004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f2b6e34c640 1 --2- 172.21.15.60:0/986339596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b540047c0 0x7f2b54004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.052+0000 7f2b6e34c640 1 -- 172.21.15.60:0/986339596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68162bc0 con 0x7f2b54003b00 2023-11-18T20:42:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7f2b6eb4d640 1 -- 172.21.15.60:0/986339596 reap_dead start 2023-11-18T20:42:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fb3bf59e640 1 -- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3c00b40b0 msgr2=0x7fb3c0070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7f90d759e640 1 Processor -- start 2023-11-18T20:42:43.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fb3bf59e640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3c00b40b0 0x7fb3c0070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.053+0000 7fac7f55a640 1 -- start start 2023-11-18T20:42:43.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fb3bf59e640 1 -- 172.21.15.60:0/1100560123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3c00b2490 msgr2=0x7fb3c01f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fb3bf59e640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3c00b2490 0x7fb3c01f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fb3bf59e640 1 -- 172.21.15.60:0/1100560123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b000ae30 con 0x7fb3c0159210 2023-11-18T20:42:43.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fac7f55a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac70006ea0 0x7fac70039970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fac7f55a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac7004ffd0 0x7fac7004dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7f0340d1d640 1 -- 172.21.15.60:0/3667019342 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03340089d0 con 0x7f033404b650 2023-11-18T20:42:43.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fac7d2f6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac70006ea0 0x7fac70039970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf0007de0 con 0x7fadf000a420 2023-11-18T20:42:43.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7f04257fa640 1 -- 172.21.15.60:0/1885712610 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414011830 con 0x7f042806aa10 2023-11-18T20:42:43.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7f04257fa640 1 -- 172.21.15.60:0/1885712610 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0414011990 con 0x7f042806aa10 2023-11-18T20:42:43.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fa8fdf2c640 1 -- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f815a020 msgr2=0x7fa8f8158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fa8fdf2c640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f815a020 0x7fa8f8158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fa8fdf2c640 1 -- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f8159c40 msgr2=0x7fa8f80c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fa8fdf2c640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f8159c40 0x7fa8f80c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7fa8fdf2c640 1 -- 172.21.15.60:0/627579437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f400a5a0 con 0x7fa8f80b38e0 2023-11-18T20:42:43.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fa8fdf2c640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f80b38e0 0x7fa8f8140e40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e850 tx=0x7fa8e800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fa8ee7fc640 1 -- 172.21.15.60:0/627579437 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e800c940 con 0x7fa8f80b38e0 2023-11-18T20:42:43.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fa900190640 1 -- 172.21.15.60:0/627579437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f8142110 con 0x7fa8f80b38e0 2023-11-18T20:42:43.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fa900190640 1 -- 172.21.15.60:0/627579437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f81426d0 con 0x7fa8f80b38e0 2023-11-18T20:42:43.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fb3bf59e640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3c0159210 0x7fb3c0071150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0034000 tx=0x7fb3b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fb3a77fe640 1 -- 172.21.15.60:0/1100560123 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b003c070 con 0x7fb3c0159210 2023-11-18T20:42:43.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fb3a77fe640 1 -- 172.21.15.60:0/1100560123 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b002f6d0 con 0x7fb3c0159210 2023-11-18T20:42:43.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fac7d2f6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac70006ea0 0x7fac70039970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56268/0 (socket says 172.21.15.60:56268) 2023-11-18T20:42:43.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fac7d2f6640 1 -- 172.21.15.60:0/2858520402 learned_addr learned my addr 172.21.15.60:0/2858520402 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7fac7caf5640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac7004ffd0 0x7fac7004dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7fac7f55a640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac700ead00 0x7fac700db450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7004ea50 con 0x7fac7004ffd0 2023-11-18T20:42:43.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7004e260 con 0x7fac700ead00 2023-11-18T20:42:43.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7004e490 con 0x7fac70006ea0 2023-11-18T20:42:43.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7f042ef1d640 1 -- 172.21.15.60:0/1885712610 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04281fb270 con 0x7f042806aa10 2023-11-18T20:42:43.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7f042ef1d640 1 -- 172.21.15.60:0/1885712610 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04280ccb00 con 0x7f042806aa10 2023-11-18T20:42:43.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7f04257fa640 1 -- 172.21.15.60:0/1885712610 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414011d40 con 0x7f042806aa10 2023-11-18T20:42:43.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7fb3a77fe640 1 -- 172.21.15.60:0/1100560123 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b002fa60 con 0x7fb3c0159210 2023-11-18T20:42:43.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c0153540 con 0x7fb3c0159210 2023-11-18T20:42:43.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c0153b00 con 0x7fb3c0159210 2023-11-18T20:42:43.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7f90d759e640 1 -- start start 2023-11-18T20:42:43.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d759e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00392a0 0x7f90d00cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d004fa60 0x7f90d004cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d00df3f0 0x7f90d00485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d759e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d004dec0 con 0x7f90d004fa60 2023-11-18T20:42:43.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d759e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d004d900 con 0x7f90d00df3f0 2023-11-18T20:42:43.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d759e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d004dbe0 con 0x7f90d00392a0 2023-11-18T20:42:43.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.054+0000 7f2b6e34c640 1 --2- 172.21.15.60:0/986339596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 0x7f2b54003f00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b580019a0 tx=0x7f2b540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5058490ee6e92459 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.055+0000 7f032affd640 1 -- 172.21.15.60:0/3667019342 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f033c080070 con 0x7f033404b650 2023-11-18T20:42:43.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f032affd640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f031c06af10 0x7f031c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f032affd640 1 -- 172.21.15.60:0/3667019342 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f033c07e070 con 0x7f033404b650 2023-11-18T20:42:43.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d004fa60 0x7f90d004cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d004fa60 0x7f90d004cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41298/0 (socket says 172.21.15.60:41298) 2023-11-18T20:42:43.139 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d5d9b640 1 -- 172.21.15.60:0/3427351683 learned_addr learned my addr 172.21.15.60:0/3427351683 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d6d9d640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d00df3f0 0x7f90d00485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.056+0000 7f90d659c640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00392a0 0x7f90d00cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f033a59c640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f031c06af10 0x7f031c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f042ef1d640 1 -- 172.21.15.60:0/1885712610 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0428067940 con 0x7f042806aa10 2023-11-18T20:42:43.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7fac7daf7640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac700ead00 0x7fac700db450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7fac7caf5640 1 -- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac70006ea0 msgr2=0x7fac70039970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7fac7caf5640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac70006ea0 0x7fac70039970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7fa8ee7fc640 1 -- 172.21.15.60:0/627579437 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e800caa0 con 0x7fa8f80b38e0 2023-11-18T20:42:43.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7fac7caf5640 1 -- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac700ead00 msgr2=0x7fac700db450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90d6d9d640 1 -- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00392a0 msgr2=0x7f90d00cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90d6d9d640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00392a0 0x7f90d00cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90d6d9d640 1 -- 172.21.15.60:0/3427351683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d004fa60 msgr2=0x7f90d004cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90d6d9d640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d004fa60 0x7f90d004cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90d6d9d640 1 -- 172.21.15.60:0/3427351683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d004bc70 con 0x7f90d00df3f0 2023-11-18T20:42:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7fa8ee7fc640 1 -- 172.21.15.60:0/627579437 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e800ce30 con 0x7fa8f80b38e0 2023-11-18T20:42:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7fac7caf5640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac700ead00 0x7fac700db450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7fac7caf5640 1 -- 172.21.15.60:0/2858520402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac680094d0 con 0x7fac7004ffd0 2023-11-18T20:42:43.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b580048a0 con 0x7f2b54003b00 2023-11-18T20:42:43.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b58004a20 con 0x7f2b54003b00 2023-11-18T20:42:43.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f2b6d34a640 1 -- 172.21.15.60:0/986339596 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b58004dd0 con 0x7f2b54003b00 2023-11-18T20:42:43.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7fac7caf5640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac7004ffd0 0x7fac7004dd20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac7400c410 tx=0x7fac7400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.058+0000 7fadf57fa640 1 -- 172.21.15.60:0/2034891441 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadec04e450 con 0x7fadf000a420 2023-11-18T20:42:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.058+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c0069400 con 0x7fb3c0159210 2023-11-18T20:42:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.058+0000 7f2b705b0640 1 -- 172.21.15.60:0/986339596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 msgr2=0x7f2b54003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.058+0000 7f2b705b0640 1 --2- 172.21.15.60:0/986339596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 0x7f2b54003f00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b580019a0 tx=0x7f2b540057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90d6d9d640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d00df3f0 0x7f90d00485c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f90d8068f20 tx=0x7f90d80a8070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.058+0000 7fadf57fa640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faddc06af40 0x7faddc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90c6ffd640 1 -- 172.21.15.60:0/3427351683 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80b8610 con 0x7f90d00df3f0 2023-11-18T20:42:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90c6ffd640 1 -- 172.21.15.60:0/3427351683 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d80b8770 con 0x7f90d00df3f0 2023-11-18T20:42:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90d759e640 1 -- 172.21.15.60:0/3427351683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d004bf50 con 0x7f90d00df3f0 2023-11-18T20:42:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90d759e640 1 -- 172.21.15.60:0/3427351683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d004d4f0 con 0x7f90d00df3f0 2023-11-18T20:42:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.057+0000 7f90c6ffd640 1 -- 172.21.15.60:0/3427351683 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80b8b20 con 0x7f90d00df3f0 2023-11-18T20:42:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.058+0000 7fadf57fa640 1 -- 172.21.15.60:0/2034891441 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadec0ab660 con 0x7fadf000a420 2023-11-18T20:42:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.058+0000 7f90d759e640 1 -- 172.21.15.60:0/3427351683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d004f8d0 con 0x7f90d00df3f0 2023-11-18T20:42:43.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7fa900190640 1 -- 172.21.15.60:0/627579437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e0005250 con 0x7fa8f80b38e0 2023-11-18T20:42:43.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7fac6dffb640 1 -- 172.21.15.60:0/2858520402 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac740035f0 con 0x7fac7004ffd0 2023-11-18T20:42:43.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7004e770 con 0x7fac7004ffd0 2023-11-18T20:42:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7f2b705b0640 1 -- 172.21.15.60:0/986339596 shutdown_connections 2023-11-18T20:42:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7f2b705b0640 1 --2- 172.21.15.60:0/986339596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 0x7f2b54003f00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7f2b705b0640 1 -- 172.21.15.60:0/986339596 >> 172.21.15.60:0/986339596 conn(0x7f2b68072020 msgr2=0x7f2b68072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7fadf7fff640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faddc06af40 0x7faddc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7002c940 con 0x7fac7004ffd0 2023-11-18T20:42:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7fa8ee7fc640 1 -- 172.21.15.60:0/627579437 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa8e8013470 con 0x7fa8f80b38e0 2023-11-18T20:42:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7f2b705b0640 1 -- 172.21.15.60:0/986339596 shutdown_connections 2023-11-18T20:42:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7f2b705b0640 1 -- 172.21.15.60:0/986339596 wait complete. 2023-11-18T20:42:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7f90c6ffd640 1 -- 172.21.15.60:0/3427351683 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f90d80b5050 con 0x7f90d00df3f0 2023-11-18T20:42:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.059+0000 7fadf7fff640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faddc06af40 0x7faddc06d400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fadf00362b0 tx=0x7fade4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.060+0000 7fa8ee7fc640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8dc06ae90 0x7fa8dc06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.060+0000 7f033a59c640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f031c06af10 0x7f031c06d3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0330000f60 tx=0x7f03300014f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.060+0000 7fa8ee7fc640 1 -- 172.21.15.60:0/627579437 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8e80896e0 con 0x7fa8f80b38e0 2023-11-18T20:42:43.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.060+0000 7fa8fd72b640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8dc06ae90 0x7fa8dc06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.060+0000 7f90c6ffd640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f90b806ae90 0x7f90b806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.060+0000 7f90c6ffd640 1 -- 172.21.15.60:0/3427351683 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90d8125450 con 0x7f90d00df3f0 2023-11-18T20:42:43.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.060+0000 7f90d659c640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f90b806ae90 0x7f90b806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.060+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7002c4c0 con 0x7fac7004ffd0 2023-11-18T20:42:43.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.060+0000 7fa8fd72b640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8dc06ae90 0x7fa8dc06d350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80b4180 tx=0x7fa8f00013e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.061+0000 7f90d659c640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f90b806ae90 0x7f90b806d350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f90d0035760 tx=0x7f90cc007e30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.061+0000 7f2b705b0640 1 Processor -- start 2023-11-18T20:42:43.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.061+0000 7f2b705b0640 1 -- start start 2023-11-18T20:42:43.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.061+0000 7f032affd640 1 -- 172.21.15.60:0/3667019342 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f033c0bcb40 con 0x7f033404b650 2023-11-18T20:42:43.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f2b705b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 0x7f2b680b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f2b705b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b6814f670 0x7f2b6814d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f2b705b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b6814db40 0x7f2b6814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f04257fa640 1 -- 172.21.15.60:0/1885712610 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f041401a400 con 0x7f042806aa10 2023-11-18T20:42:43.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f2b705b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b680c5770 con 0x7f2b54003b00 2023-11-18T20:42:43.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f2b705b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68158aa0 con 0x7f2b6814db40 2023-11-18T20:42:43.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f2b705b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b681587c0 con 0x7f2b6814f670 2023-11-18T20:42:43.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f2b6eb4d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b6814db40 0x7f2b6814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f2b6eb4d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b6814db40 0x7f2b6814dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49814/0 (socket says 172.21.15.60:49814) 2023-11-18T20:42:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f2b6eb4d640 1 -- 172.21.15.60:0/3641417111 learned_addr learned my addr 172.21.15.60:0/3641417111 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.062+0000 7f2b6db4b640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b6814f670 0x7f2b6814d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7f2b6e34c640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 0x7f2b680b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7f2b6eb4d640 1 -- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b6814f670 msgr2=0x7f2b6814d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7f2b6eb4d640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b6814f670 0x7f2b6814d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7f04257fa640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f040c06ae90 0x7f040c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7f2b6eb4d640 1 -- 172.21.15.60:0/3641417111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 msgr2=0x7f2b680b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7f04257fa640 1 -- 172.21.15.60:0/1885712610 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0414089df0 con 0x7f042806aa10 2023-11-18T20:42:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7fb3a77fe640 1 -- 172.21.15.60:0/1100560123 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb3b002fbc0 con 0x7fb3c0159210 2023-11-18T20:42:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7f2b6eb4d640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 0x7f2b680b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7f2b6eb4d640 1 -- 172.21.15.60:0/3641417111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680c4ac0 con 0x7f2b6814db40 2023-11-18T20:42:43.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7f0427fff640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f040c06ae90 0x7f040c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.063+0000 7f2b6eb4d640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b6814db40 0x7f2b6814dff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b680ca2c0 tx=0x7f2b6400ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.064+0000 7f2b6e34c640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 0x7f2b680b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:43.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.064+0000 7fb3a77fe640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb3ac06af10 0x7fb3ac06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.064+0000 7fb3a77fe640 1 -- 172.21.15.60:0/1100560123 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3b0033070 con 0x7fb3c0159210 2023-11-18T20:42:43.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.064+0000 7fb3bed9d640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb3ac06af10 0x7fb3ac06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.064+0000 7f0427fff640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f040c06ae90 0x7f040c06d350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f041c036000 tx=0x7f041c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.064+0000 7fb3bed9d640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb3ac06af10 0x7fb3ac06d3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4005880 tx=0x7fb3b4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.065+0000 7f2b5effd640 1 -- 172.21.15.60:0/3641417111 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400ee50 con 0x7f2b6814db40 2023-11-18T20:42:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.065+0000 7f2b5effd640 1 -- 172.21.15.60:0/3641417111 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6400c890 con 0x7f2b6814db40 2023-11-18T20:42:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.065+0000 7f2b5effd640 1 -- 172.21.15.60:0/3641417111 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400cc40 con 0x7f2b6814db40 2023-11-18T20:42:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.065+0000 7f2b705b0640 1 -- 172.21.15.60:0/3641417111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b680c4da0 con 0x7f2b6814db40 2023-11-18T20:42:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.065+0000 7f2b705b0640 1 -- 172.21.15.60:0/3641417111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b680b4580 con 0x7f2b6814db40 2023-11-18T20:42:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.065+0000 7f90c6ffd640 1 -- 172.21.15.60:0/3427351683 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f90d80f0690 con 0x7f90d00df3f0 2023-11-18T20:42:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.066+0000 7fac6dffb640 1 -- 172.21.15.60:0/2858520402 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac74003750 con 0x7fac7004ffd0 2023-11-18T20:42:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.066+0000 7fac6dffb640 1 -- 172.21.15.60:0/2858520402 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac74003b20 con 0x7fac7004ffd0 2023-11-18T20:42:43.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.066+0000 7fadf57fa640 1 -- 172.21.15.60:0/2034891441 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadf0007de0 con 0x7fadf000a420 2023-11-18T20:42:43.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.067+0000 7f04257fa640 1 -- 172.21.15.60:0/1885712610 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0414054e80 con 0x7f042806aa10 2023-11-18T20:42:43.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.067+0000 7f2b5effd640 1 -- 172.21.15.60:0/3641417111 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b64013470 con 0x7f2b6814db40 2023-11-18T20:42:43.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.068+0000 7f2b705b0640 1 -- 172.21.15.60:0/3641417111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b680b4740 con 0x7f2b6814db40 2023-11-18T20:42:43.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.068+0000 7f2b5effd640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b4c06ae70 0x7f2b4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.068+0000 7f2b5effd640 1 -- 172.21.15.60:0/3641417111 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b64088f00 con 0x7f2b6814db40 2023-11-18T20:42:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.068+0000 7f2b6e34c640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b4c06ae70 0x7f2b4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.069+0000 7f2b6e34c640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b4c06ae70 0x7f2b4c06d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b58033000 tx=0x7f2b58009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.071+0000 7fb3a77fe640 1 -- 172.21.15.60:0/1100560123 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3b00d7e90 con 0x7fb3c0159210 2023-11-18T20:42:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.073+0000 7fa8ee7fc640 1 -- 172.21.15.60:0/627579437 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8e8054890 con 0x7fa8f80b38e0 2023-11-18T20:42:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.073+0000 7f2b5effd640 1 -- 172.21.15.60:0/3641417111 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b64053f10 con 0x7f2b6814db40 2023-11-18T20:42:43.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.074+0000 7fac6dffb640 1 -- 172.21.15.60:0/2858520402 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac74003c80 con 0x7fac7004ffd0 2023-11-18T20:42:43.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.075+0000 7fac6dffb640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fac5c06ae40 0x7fac5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:43.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.075+0000 7fac6dffb640 1 -- 172.21.15.60:0/2858520402 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac74089190 con 0x7fac7004ffd0 2023-11-18T20:42:43.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.079+0000 7fac7d2f6640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fac5c06ae40 0x7fac5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:43.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.081+0000 7fac6dffb640 1 -- 172.21.15.60:0/2858520402 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fac740542d0 con 0x7fac7004ffd0 2023-11-18T20:42:43.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.082+0000 7fac7d2f6640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fac5c06ae40 0x7fac5c06d300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac680090b0 tx=0x7fac6803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:43.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.484+0000 7f90d759e640 1 -- 172.21.15.60:0/3427351683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f90d004dd20 con 0x7f90d00df3f0 2023-11-18T20:42:43.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.484+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb3c014d100 con 0x7fb3c0159210 2023-11-18T20:42:43.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.485+0000 7f0340d1d640 1 -- 172.21.15.60:0/3667019342 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f033410dbd0 con 0x7f033404b650 2023-11-18T20:42:43.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.485+0000 7f90c6ffd640 1 -- 172.21.15.60:0/3427351683 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f90d004dd20 con 0x7f90d00df3f0 2023-11-18T20:42:43.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.485+0000 7fb3a77fe640 1 -- 172.21.15.60:0/1100560123 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3b0076260 con 0x7fb3c0159210 2023-11-18T20:42:43.487 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-11-18T20:42:43.488 INFO:teuthology.orchestra.run.smithi060.stdout:55834574852 2023-11-18T20:42:43.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.485+0000 7f042ef1d640 1 -- 172.21.15.60:0/1885712610 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f042804e8e0 con 0x7f042806aa10 2023-11-18T20:42:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.486+0000 7f04257fa640 1 -- 172.21.15.60:0/1885712610 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0414015090 con 0x7f042806aa10 2023-11-18T20:42:43.489 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-11-18T20:42:43.489 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-11-18T20:42:43.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.486+0000 7f032affd640 1 -- 172.21.15.60:0/3667019342 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f033c0c0160 con 0x7f033404b650 2023-11-18T20:42:43.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.488+0000 7f90d759e640 1 -- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f90b806ae90 msgr2=0x7f90b806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.488+0000 7f90d759e640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f90b806ae90 0x7f90b806d350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f90d0035760 tx=0x7f90cc007e30 comp rx=0 tx=0).stop 2023-11-18T20:42:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.488+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb3ac06af10 msgr2=0x7fb3ac06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.488+0000 7f90d759e640 1 -- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d00df3f0 msgr2=0x7f90d00485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f90d759e640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d00df3f0 0x7f90d00485c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f90d8068f20 tx=0x7f90d80a8070 comp rx=0 tx=0).stop 2023-11-18T20:42:43.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb3ac06af10 0x7fb3ac06d3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4005880 tx=0x7fb3b4009040 comp rx=0 tx=0).stop 2023-11-18T20:42:43.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f90d759e640 1 -- 172.21.15.60:0/3427351683 shutdown_connections 2023-11-18T20:42:43.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f90d759e640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f90b806ae90 0x7f90b806d350 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f90d759e640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90d00df3f0 0x7f90d00485c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f90d759e640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90d004fa60 0x7f90d004cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f90d759e640 1 --2- 172.21.15.60:0/3427351683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90d00392a0 0x7f90d00cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f90d759e640 1 -- 172.21.15.60:0/3427351683 >> 172.21.15.60:0/3427351683 conn(0x7f90d0009f80 msgr2=0x7f90d004fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fadf0007de0 con 0x7fadf000a420 2023-11-18T20:42:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3c0159210 msgr2=0x7fb3c0071150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f90d759e640 1 -- 172.21.15.60:0/3427351683 shutdown_connections 2023-11-18T20:42:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3c0159210 0x7fb3c0071150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0034000 tx=0x7fb3b00046c0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f90d759e640 1 -- 172.21.15.60:0/3427351683 wait complete. 2023-11-18T20:42:43.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 shutdown_connections 2023-11-18T20:42:43.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f042ef1d640 1 -- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f040c06ae90 msgr2=0x7f040c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb3ac06af10 0x7fb3ac06d3d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f042ef1d640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f040c06ae90 0x7f040c06d350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f041c036000 tx=0x7f041c008030 comp rx=0 tx=0).stop 2023-11-18T20:42:43.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3c0159210 0x7fb3c0071150 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3c00b40b0 0x7fb3c0070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 --2- 172.21.15.60:0/1100560123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3c00b2490 0x7fb3c01f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 >> 172.21.15.60:0/1100560123 conn(0x7fb3c00b0d60 msgr2=0x7fb3c0151e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f042ef1d640 1 -- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042806aa10 msgr2=0x7f042815ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7f042ef1d640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042806aa10 0x7f042815ccb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f041400b3e0 tx=0x7f041400b8a0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 shutdown_connections 2023-11-18T20:42:43.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.489+0000 7fb3c51e7640 1 -- 172.21.15.60:0/1100560123 wait complete. 2023-11-18T20:42:43.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f042ef1d640 1 -- 172.21.15.60:0/1885712610 shutdown_connections 2023-11-18T20:42:43.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f042ef1d640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f040c06ae90 0x7f040c06d350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f042ef1d640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042814b2e0 0x7f04281fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f042ef1d640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042806f7a0 0x7f042815d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f042ef1d640 1 --2- 172.21.15.60:0/1885712610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f042806aa10 0x7f042815ccb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f042ef1d640 1 -- 172.21.15.60:0/1885712610 >> 172.21.15.60:0/1885712610 conn(0x7f04280cfd80 msgr2=0x7f04280d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f042ef1d640 1 -- 172.21.15.60:0/1885712610 shutdown_connections 2023-11-18T20:42:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7fadf57fa640 1 -- 172.21.15.60:0/2034891441 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fadf0007de0 con 0x7fadf000a420 2023-11-18T20:42:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f042ef1d640 1 -- 172.21.15.60:0/1885712610 wait complete. 2023-11-18T20:42:43.502 INFO:teuthology.orchestra.run.smithi060.stdout:55834574852 2023-11-18T20:42:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f0340d1d640 1 -- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f031c06af10 msgr2=0x7f031c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f0340d1d640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f031c06af10 0x7f031c06d3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0330000f60 tx=0x7f03300014f0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f0340d1d640 1 -- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033404b650 msgr2=0x7f03340495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7f0340d1d640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033404b650 0x7f03340495b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f033c079980 tx=0x7f033c0677f0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.490+0000 7fa900190640 1 -- 172.21.15.60:0/627579437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa8e00057d0 con 0x7fa8f80b38e0 2023-11-18T20:42:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.491+0000 7f0340d1d640 1 -- 172.21.15.60:0/3667019342 shutdown_connections 2023-11-18T20:42:43.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.491+0000 7f0340d1d640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f031c06af10 0x7f031c06d3d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.491+0000 7f0340d1d640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033404b650 0x7f03340495b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.491+0000 7f0340d1d640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0334038060 0x7f033404b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.491+0000 7f0340d1d640 1 --2- 172.21.15.60:0/3667019342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0334033920 0x7f03340ef030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.491+0000 7f0340d1d640 1 -- 172.21.15.60:0/3667019342 >> 172.21.15.60:0/3667019342 conn(0x7f0334009fb0 msgr2=0x7f0334035750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.491+0000 7f0340d1d640 1 -- 172.21.15.60:0/3667019342 shutdown_connections 2023-11-18T20:42:43.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.491+0000 7fa8ee7fc640 1 -- 172.21.15.60:0/627579437 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa8e8017020 con 0x7fa8f80b38e0 2023-11-18T20:42:43.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.491+0000 7f0340d1d640 1 -- 172.21.15.60:0/3667019342 wait complete. 2023-11-18T20:42:43.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.491+0000 7f2b705b0640 1 -- 172.21.15.60:0/3641417111 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2b680c5260 con 0x7f2b6814db40 2023-11-18T20:42:43.506 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-11-18T20:42:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.492+0000 7f2b5effd640 1 -- 172.21.15.60:0/3641417111 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b64000e60 con 0x7f2b6814db40 2023-11-18T20:42:43.506 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-11-18T20:42:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.494+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faddc06af40 msgr2=0x7faddc06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.494+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faddc06af40 0x7faddc06d400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fadf00362b0 tx=0x7fade4009040 comp rx=0 tx=0).stop 2023-11-18T20:42:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.494+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf000a420 msgr2=0x7fadf00d5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.494+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf000a420 0x7fadf00d5b00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fadec0090b0 tx=0x7fadec0046c0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.494+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 shutdown_connections 2023-11-18T20:42:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faddc06af40 0x7faddc06d400 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadf00e12d0 0x7fadf0174a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadf000ab00 0x7fadf01744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fadfdf73640 1 --2- 172.21.15.60:0/2034891441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadf000a420 0x7fadf00d5b00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 >> 172.21.15.60:0/2034891441 conn(0x7fadf0056c90 msgr2=0x7fadf003c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 shutdown_connections 2023-11-18T20:42:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fadfdf73640 1 -- 172.21.15.60:0/2034891441 wait complete. 2023-11-18T20:42:43.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fac7002cb80 con 0x7fac7004ffd0 2023-11-18T20:42:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fa900190640 1 -- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8dc06ae90 msgr2=0x7fa8dc06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fa900190640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8dc06ae90 0x7fa8dc06d350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80b4180 tx=0x7fa8f00013e0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7f2b705b0640 1 -- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b4c06ae70 msgr2=0x7f2b4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fa900190640 1 -- 172.21.15.60:0/627579437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f80b38e0 msgr2=0x7fa8f8140e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7fa900190640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f80b38e0 0x7fa8f8140e40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e850 tx=0x7fa8e800ed10 comp rx=0 tx=0).stop 2023-11-18T20:42:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.495+0000 7f2b705b0640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b4c06ae70 0x7f2b4c06d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b58033000 tx=0x7f2b58009030 comp rx=0 tx=0).stop 2023-11-18T20:42:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7f2b705b0640 1 -- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b6814db40 msgr2=0x7f2b6814dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7f2b705b0640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b6814db40 0x7f2b6814dff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b680ca2c0 tx=0x7f2b6400ec00 comp rx=0 tx=0).stop 2023-11-18T20:42:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7fa900190640 1 -- 172.21.15.60:0/627579437 shutdown_connections 2023-11-18T20:42:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7fa900190640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8dc06ae90 0x7fa8dc06d350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7fa900190640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f815a020 0x7fa8f8158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7fa900190640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f8159c40 0x7fa8f80c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7fa900190640 1 --2- 172.21.15.60:0/627579437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f80b38e0 0x7fa8f8140e40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7fa900190640 1 -- 172.21.15.60:0/627579437 >> 172.21.15.60:0/627579437 conn(0x7fa8f80b0b10 msgr2=0x7fa8f806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7f2b705b0640 1 -- 172.21.15.60:0/3641417111 shutdown_connections 2023-11-18T20:42:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7f2b705b0640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2b4c06ae70 0x7f2b4c06d330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7f2b705b0640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b6814db40 0x7f2b6814dff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7f2b705b0640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b6814f670 0x7f2b6814d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7fa900190640 1 -- 172.21.15.60:0/627579437 shutdown_connections 2023-11-18T20:42:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7fa900190640 1 -- 172.21.15.60:0/627579437 wait complete. 2023-11-18T20:42:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7f2b705b0640 1 --2- 172.21.15.60:0/3641417111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b54003b00 0x7f2b680b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7f2b705b0640 1 -- 172.21.15.60:0/3641417111 >> 172.21.15.60:0/3641417111 conn(0x7f2b68072020 msgr2=0x7f2b680d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.496+0000 7fac6dffb640 1 -- 172.21.15.60:0/2858520402 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fac7002cb80 con 0x7fac7004ffd0 2023-11-18T20:42:43.516 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-11-18T20:42:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.497+0000 7f2b705b0640 1 -- 172.21.15.60:0/3641417111 shutdown_connections 2023-11-18T20:42:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.497+0000 7f2b705b0640 1 -- 172.21.15.60:0/3641417111 wait complete. 2023-11-18T20:42:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.500+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fac5c06ae40 msgr2=0x7fac5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.500+0000 7fac7f55a640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fac5c06ae40 0x7fac5c06d300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac680090b0 tx=0x7fac6803c040 comp rx=0 tx=0).stop 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.500+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac7004ffd0 msgr2=0x7fac7004dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.500+0000 7fac7f55a640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac7004ffd0 0x7fac7004dd20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac7400c410 tx=0x7fac7400c810 comp rx=0 tx=0).stop 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.500+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 shutdown_connections 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.500+0000 7fac7f55a640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fac5c06ae40 0x7fac5c06d300 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac680090b0 tx=0x7fac6803c040 comp rx=0 tx=0).stop 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.500+0000 7fac7f55a640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac700ead00 0x7fac700db450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.500+0000 7fac7f55a640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac7004ffd0 0x7fac7004dd20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.500+0000 7fac7f55a640 1 --2- 172.21.15.60:0/2858520402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac70006ea0 0x7fac70039970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.500+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 >> 172.21.15.60:0/2858520402 conn(0x7fac70009fb0 msgr2=0x7fac7004a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.505+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 shutdown_connections 2023-11-18T20:42:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:43.505+0000 7fac7f55a640 1 -- 172.21.15.60:0/2858520402 wait complete. 2023-11-18T20:42:43.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-11-18T20:42:43.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2023-11-18T20:42:43.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-11-18T20:42:43.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-11-18T20:42:43.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-11-18T20:42:43.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-11-18T20:42:43.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-11-18T20:42:43.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-11-18T20:42:44.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-18T20:42:44.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-18T20:42:44.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-18T20:42:44.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-18T20:42:44.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-18T20:42:44.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-18T20:42:44.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-18T20:42:44.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-18T20:42:44.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.716+0000 7ff05a1b0640 1 Processor -- start 2023-11-18T20:42:44.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.716+0000 7ff05a1b0640 1 -- start start 2023-11-18T20:42:44.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.717+0000 7fb7bed73640 1 Processor -- start 2023-11-18T20:42:44.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.717+0000 7efc85ba9640 1 Processor -- start 2023-11-18T20:42:44.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.717+0000 7fb7bed73640 1 -- start start 2023-11-18T20:42:44.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7ff05a1b0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff05406f000 0x7ff05406a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7ff05a1b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0540cf000 con 0x7ff054143b50 2023-11-18T20:42:44.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7ff05a1b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0540cf260 con 0x7ff05406f630 2023-11-18T20:42:44.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.717+0000 7efc85ba9640 1 -- start start 2023-11-18T20:42:44.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7fa3783d5640 1 Processor -- start 2023-11-18T20:42:44.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.717+0000 7f606f3b0640 1 Processor -- start 2023-11-18T20:42:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7ff05a1b0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0540cfde0 con 0x7ff05406f000 2023-11-18T20:42:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7ff0537fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff054143b50 0x7ff0540ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39182/0 (socket says 172.21.15.60:39182) 2023-11-18T20:42:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7ff0537fe640 1 -- 172.21.15.60:0/1921168624 learned_addr learned my addr 172.21.15.60:0/1921168624 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7ff053fff640 1 --2- 172.21.15.60:0/1921168624 >> v2:172.21.15.100:3300/0 conn(0x7ff05406f000 0x7ff05406a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7fa3783d5640 1 -- start start 2023-11-18T20:42:44.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7fb7bed73640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb7b00e3090 0x7fb7b0057d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7fb7bed73640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7b00542d0 0x7fb7b00068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7fb7bed73640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b00582a0 con 0x7fb7b003e810 2023-11-18T20:42:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7fb7bed73640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b0006170 con 0x7fb7b00542d0 2023-11-18T20:42:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.718+0000 7fb7bed73640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b0058540 con 0x7fb7b00e3090 2023-11-18T20:42:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc85ba9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efc8015d330 0x7efc800b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205739860 0 0) 0x7ff0540cf000 con 0x7ff054143b50 2023-11-18T20:42:44.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff03c003520 con 0x7ff054143b50 2023-11-18T20:42:44.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc7f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efc8015d330 0x7efc800b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc7effd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7efc800d2910 0x7efc800d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:53068/0 (socket says 172.21.15.60:53068) 2023-11-18T20:42:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc7f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efc8015d330 0x7efc800b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49848/0 (socket says 172.21.15.60:49848) 2023-11-18T20:42:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc7effd640 1 -- 172.21.15.60:0/326270905 learned_addr learned my addr 172.21.15.60:0/326270905 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc85ba9640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc800d3900 0x7efc800b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc85ba9640 1 -- 172.21.15.60:0/326270905 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc800b2b10 con 0x7efc800d2910 2023-11-18T20:42:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc85ba9640 1 -- 172.21.15.60:0/326270905 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc800d1590 con 0x7efc8015d330 2023-11-18T20:42:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc85ba9640 1 -- 172.21.15.60:0/326270905 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc800d1870 con 0x7efc800d3900 2023-11-18T20:42:44.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc7ffff640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc800d3900 0x7efc800b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588927688 0 0) 0x7efc800b2b10 con 0x7efc800d2910 2023-11-18T20:42:44.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc68003520 con 0x7efc800d2910 2023-11-18T20:42:44.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7fb7be572640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7b00542d0 0x7fb7b00068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7fb7be572640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb7b00542d0 0x7fb7b00068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41302/0 (socket says 172.21.15.60:41302) 2023-11-18T20:42:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7fb7be572640 1 -- 172.21.15.60:0/315433459 learned_addr learned my addr 172.21.15.60:0/315433459 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7fb7bdd71640 1 --2- 172.21.15.60:0/315433459 >> v2:172.21.15.100:3300/0 conn(0x7fb7b00e3090 0x7fb7b0057d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7fb7be572640 1 -- 172.21.15.60:0/315433459 >> v2:172.21.15.100:3300/0 conn(0x7fb7b00e3090 msgr2=0x7fb7b0057d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7f57c6ef8640 1 Processor -- start 2023-11-18T20:42:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187153174 0 0) 0x7ff03c003520 con 0x7ff054143b50 2023-11-18T20:42:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 >> v2:172.21.15.100:3300/0 conn(0x7ff05406f000 msgr2=0x7ff05406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 --2- 172.21.15.60:0/1921168624 >> v2:172.21.15.100:3300/0 conn(0x7ff05406f000 0x7ff05406a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 >> v1:172.21.15.100:6789/0 conn(0x7ff05406f630 legacy=0x7ff05406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540cf540 con 0x7ff054143b50 2023-11-18T20:42:44.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7ff044001690 con 0x7ff054143b50 2023-11-18T20:42:44.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 >> v1:172.21.15.60:6789/0 conn(0x7ff054143b50 legacy=0x7ff0540ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 --2- 172.21.15.60:0/1921168624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff03c003970 0x7ff03c003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 --2- 172.21.15.60:0/1921168624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff03c004370 0x7ff03c004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 --2- 172.21.15.60:0/1921168624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff03c0050e0 0x7ff03c014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540cfde0 con 0x7ff03c003970 2023-11-18T20:42:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c014830 con 0x7ff03c004370 2023-11-18T20:42:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0149f0 con 0x7ff03c0050e0 2023-11-18T20:42:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7fa3783d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa3700b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7f606f3b0640 1 -- start start 2023-11-18T20:42:44.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7fa3783d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa370162d80 con 0x7fa3700b0cf0 2023-11-18T20:42:44.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7fa3783d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa370165fd0 con 0x7fa3700b1150 2023-11-18T20:42:44.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7fa3783d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa370070390 con 0x7fa370158690 2023-11-18T20:42:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7fa376171640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa3700b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7fa376171640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa3700b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56324/0 (socket says 172.21.15.60:56324) 2023-11-18T20:42:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7fa376171640 1 -- 172.21.15.60:0/1185734801 learned_addr learned my addr 172.21.15.60:0/1185734801 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7fb7be572640 1 --2- 172.21.15.60:0/315433459 >> v2:172.21.15.100:3300/0 conn(0x7fb7b00e3090 0x7fb7b0057d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7fb7be572640 1 -- 172.21.15.60:0/315433459 >> v1:172.21.15.100:6789/0 conn(0x7fb7b003e810 legacy=0x7fb7b0053bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.719+0000 7fb7be572640 1 -- 172.21.15.60:0/315433459 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b0058820 con 0x7fb7b00542d0 2023-11-18T20:42:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f87309fb640 1 Processor -- start 2023-11-18T20:42:44.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7fa376171640 1 -- 172.21.15.60:0/1185734801 >> v1:172.21.15.100:6789/0 conn(0x7fa3700b1150 legacy=0x7fa3701658c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-18T20:42:44.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7fa376171640 1 -- 172.21.15.60:0/1185734801 >> v1:172.21.15.60:6789/0 conn(0x7fa3700b0cf0 legacy=0x7fa3701651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7fa376171640 1 -- 172.21.15.60:0/1185734801 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3700705a0 con 0x7fa370158690 2023-11-18T20:42:44.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 715435997 0 0) 0x7efc68003520 con 0x7efc800d2910 2023-11-18T20:42:44.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc800d3900 msgr2=0x7efc800b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc800d3900 0x7efc800b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 >> v2:172.21.15.100:3300/0 conn(0x7efc8015d330 msgr2=0x7efc800b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 --2- 172.21.15.60:0/326270905 >> v2:172.21.15.100:3300/0 conn(0x7efc8015d330 0x7efc800b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800d1b50 con 0x7efc800d2910 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7efc74001670 con 0x7efc800d2910 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 >> v1:172.21.15.100:6789/0 conn(0x7efc800d2910 legacy=0x7efc800d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc68003970 0x7efc68003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc680047c0 0x7efc68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc68005180 0x7efc6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc800d1590 con 0x7efc68005180 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc800d1870 con 0x7efc68003970 2023-11-18T20:42:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6800bfa0 con 0x7efc680047c0 2023-11-18T20:42:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7efc7effd640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc680047c0 0x7efc68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7efc7effd640 1 -- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc68003970 msgr2=0x7efc68003d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-18T20:42:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7efc7effd640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc68003970 0x7efc68003d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7efc7effd640 1 -- 172.21.15.60:0/326270905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc68005180 msgr2=0x7efc6800b980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-18T20:42:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7efc7effd640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc68005180 0x7efc6800b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7efc7effd640 1 -- 172.21.15.60:0/326270905 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800d1b50 con 0x7efc680047c0 2023-11-18T20:42:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7ff052ffd640 1 --2- 172.21.15.60:0/1921168624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff03c004370 0x7ff03c004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f606f3b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815bbe0 0x7f60680cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.720+0000 7f57c6ef8640 1 -- start start 2023-11-18T20:42:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f57c6ef8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57b80e76d0 0x7f57b804c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f57c6ef8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57b8053c00 0x7f57b8006fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f57c6ef8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b8057d50 con 0x7f57b8058980 2023-11-18T20:42:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f606d14c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815bbe0 0x7f60680cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f57c6ef8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b8006220 con 0x7f57b80e76d0 2023-11-18T20:42:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f57c66f7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57b8053c00 0x7f57b8006fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f57c66f7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57b8053c00 0x7f57b8006fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49874/0 (socket says 172.21.15.60:49874) 2023-11-18T20:42:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f57c66f7640 1 -- 172.21.15.60:0/3585319207 learned_addr learned my addr 172.21.15.60:0/3585319207 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f57c6ef8640 1 -- 172.21.15.60:0/3585319207 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b8006500 con 0x7f57b8053c00 2023-11-18T20:42:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f606d14c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815bbe0 0x7f60680cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56338/0 (socket says 172.21.15.60:56338) 2023-11-18T20:42:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f606f3b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f606815f280 0x7f606815f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f606f3b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60680cf0f0 con 0x7f606815bfc0 2023-11-18T20:42:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f606f3b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60680c7ee0 con 0x7f606815f280 2023-11-18T20:42:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f606f3b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60680cf330 con 0x7f606815bbe0 2023-11-18T20:42:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f606d14c640 1 -- 172.21.15.60:0/1776251930 learned_addr learned my addr 172.21.15.60:0/1776251930 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7fa5d2ebb640 1 Processor -- start 2023-11-18T20:42:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.721+0000 7f87309fb640 1 -- start start 2023-11-18T20:42:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f57c66f7640 1 -- 172.21.15.60:0/3585319207 >> v2:172.21.15.60:3300/0 conn(0x7f57b80e76d0 msgr2=0x7f57b804c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f57c66f7640 1 --2- 172.21.15.60:0/3585319207 >> v2:172.21.15.60:3300/0 conn(0x7f57b80e76d0 0x7f57b804c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f57c66f7640 1 -- 172.21.15.60:0/3585319207 >> v1:172.21.15.60:6789/0 conn(0x7f57b8058980 legacy=0x7f57b80534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f57c66f7640 1 -- 172.21.15.60:0/3585319207 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80067e0 con 0x7f57b8053c00 2023-11-18T20:42:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f606d94d640 1 --2- 172.21.15.60:0/1776251930 >> v2:172.21.15.60:3300/0 conn(0x7f606815f280 0x7f606815f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7ff053fff640 1 --2- 172.21.15.60:0/1921168624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff03c0050e0 0x7ff03c014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7ff052ffd640 1 -- 172.21.15.60:0/1921168624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff03c0050e0 msgr2=0x7ff03c014190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7ff052ffd640 1 --2- 172.21.15.60:0/1921168624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff03c0050e0 0x7ff03c014190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7ff052ffd640 1 -- 172.21.15.60:0/1921168624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff03c003970 msgr2=0x7ff03c003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7ff052ffd640 1 --2- 172.21.15.60:0/1921168624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff03c003970 0x7ff03c003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7ff053fff640 1 -- 172.21.15.60:0/1921168624 reap_dead start 2023-11-18T20:42:44.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7ff052ffd640 1 -- 172.21.15.60:0/1921168624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540cf540 con 0x7ff03c004370 2023-11-18T20:42:44.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7efc7effd640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc680047c0 0x7efc68004c40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efc74007fd0 tx=0x7efc680057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0f1e9caf0fc5d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f87309fb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87240e73f0 0x7f872405b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f87309fb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87240e77d0 0x7f8724058080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f87309fb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872405be40 con 0x7f872405b930 2023-11-18T20:42:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f87309fb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872405c120 con 0x7f87240e77d0 2023-11-18T20:42:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f87309fb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872405c400 con 0x7f87240e73f0 2023-11-18T20:42:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f872b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87240e73f0 0x7f872405b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7402cb10 con 0x7efc680047c0 2023-11-18T20:42:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7402cc90 con 0x7efc680047c0 2023-11-18T20:42:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7efc7e7fc640 1 -- 172.21.15.60:0/326270905 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74005b10 con 0x7efc680047c0 2023-11-18T20:42:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7fa376171640 1 --2- 172.21.15.60:0/1185734801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa3700b44f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0098a0 tx=0x7fa36c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a019e0db860128 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7fa37516f640 1 -- 172.21.15.60:0/1185734801 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c030840 con 0x7fa370158690 2023-11-18T20:42:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7fa37516f640 1 -- 172.21.15.60:0/1185734801 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c0309a0 con 0x7fa370158690 2023-11-18T20:42:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f872b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87240e73f0 0x7f872405b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49880/0 (socket says 172.21.15.60:49880) 2023-11-18T20:42:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f872b7fe640 1 -- 172.21.15.60:0/3310186290 learned_addr learned my addr 172.21.15.60:0/3310186290 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa37516f640 1 -- 172.21.15.60:0/1185734801 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c030d30 con 0x7fa370158690 2023-11-18T20:42:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7f57c66f7640 1 --2- 172.21.15.60:0/3585319207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57b8053c00 0x7f57b8006fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f57c0050140 tx=0x7f57c00a5db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e5f584611dd90b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f606d14c640 1 -- 172.21.15.60:0/1776251930 >> v2:172.21.15.60:3300/0 conn(0x7f606815f280 msgr2=0x7f606815f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f606d14c640 1 --2- 172.21.15.60:0/1776251930 >> v2:172.21.15.60:3300/0 conn(0x7f606815f280 0x7f606815f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f606d14c640 1 -- 172.21.15.60:0/1776251930 >> v1:172.21.15.100:6789/0 conn(0x7f606815bfc0 legacy=0x7f60680c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7f606d14c640 1 -- 172.21.15.60:0/1776251930 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60680c7920 con 0x7f606815bbe0 2023-11-18T20:42:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7f606d94d640 1 --2- 172.21.15.60:0/1776251930 >> v2:172.21.15.60:3300/0 conn(0x7f606815f280 0x7f606815f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa3783d5640 1 -- 172.21.15.60:0/1185734801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 msgr2=0x7fa3700b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa3783d5640 1 --2- 172.21.15.60:0/1185734801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa3700b44f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0098a0 tx=0x7fa36c02b980 comp rx=0 tx=0).stop 2023-11-18T20:42:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fb7be572640 1 --2- 172.21.15.60:0/315433459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b00542d0 0x7fb7b00068a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb7b003be30 tx=0x7fb7a802b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7d4ab0b77192b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7efc85ba9640 1 -- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc680047c0 msgr2=0x7efc68004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.722+0000 7fa5d2ebb640 1 -- start start 2023-11-18T20:42:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa5d2ebb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4054b10 0x7fa5c4051ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa5d2ebb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5c4053bc0 0x7fa5c4051ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa5d2ebb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c40e2660 con 0x7fa5c40e1270 2023-11-18T20:42:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa5d2ebb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c4055c20 con 0x7fa5c4053bc0 2023-11-18T20:42:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa5d2ebb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c4054fa0 con 0x7fa5c4054b10 2023-11-18T20:42:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7f872affd640 1 --2- 172.21.15.60:0/3310186290 >> v2:172.21.15.60:3300/0 conn(0x7f87240e77d0 0x7f8724058080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7f872b7fe640 1 -- 172.21.15.60:0/3310186290 >> v2:172.21.15.60:3300/0 conn(0x7f87240e77d0 msgr2=0x7f8724058080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7f872b7fe640 1 --2- 172.21.15.60:0/3310186290 >> v2:172.21.15.60:3300/0 conn(0x7f87240e77d0 0x7f8724058080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7f872b7fe640 1 -- 172.21.15.60:0/3310186290 >> v1:172.21.15.60:6789/0 conn(0x7f872405b930 legacy=0x7f872400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7f872b7fe640 1 -- 172.21.15.60:0/3310186290 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872405c6e0 con 0x7f87240e73f0 2023-11-18T20:42:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa3783d5640 1 -- 172.21.15.60:0/1185734801 shutdown_connections 2023-11-18T20:42:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa3783d5640 1 --2- 172.21.15.60:0/1185734801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa3700b44f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa5d26ba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5c4053bc0 0x7fa5c4051ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7fa3783d5640 1 -- 172.21.15.60:0/1185734801 >> 172.21.15.60:0/1185734801 conn(0x7fa37006a4d0 msgr2=0x7fa37006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7fa5d26ba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5c4053bc0 0x7fa5c4051ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41336/0 (socket says 172.21.15.60:41336) 2023-11-18T20:42:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7fa5d26ba640 1 -- 172.21.15.60:0/1130738988 learned_addr learned my addr 172.21.15.60:0/1130738988 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7ff052ffd640 1 --2- 172.21.15.60:0/1921168624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff03c004370 0x7ff03c004840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff048001860 tx=0x7ff03c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=876776b715aa4c55 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7f57c4ef4640 1 -- 172.21.15.60:0/3585319207 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00a6c20 con 0x7f57b8053c00 2023-11-18T20:42:44.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7efc85ba9640 1 --2- 172.21.15.60:0/326270905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc680047c0 0x7efc68004c40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efc74007fd0 tx=0x7efc680057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7efc7ffff640 1 -- 172.21.15.60:0/326270905 reap_dead start 2023-11-18T20:42:44.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7efc85ba9640 1 -- 172.21.15.60:0/326270905 shutdown_connections 2023-11-18T20:42:44.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7efc85ba9640 1 -- 172.21.15.60:0/326270905 >> 172.21.15.60:0/326270905 conn(0x7efc80072020 msgr2=0x7efc80072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7fa3783d5640 1 -- 172.21.15.60:0/1185734801 shutdown_connections 2023-11-18T20:42:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7f57c4ef4640 1 -- 172.21.15.60:0/3585319207 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c00a6dc0 con 0x7f57b8053c00 2023-11-18T20:42:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7f57c4ef4640 1 -- 172.21.15.60:0/3585319207 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c00ae640 con 0x7f57b8053c00 2023-11-18T20:42:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.723+0000 7f606d14c640 1 --2- 172.21.15.60:0/1776251930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815bbe0 0x7f60680cebb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6064009870 tx=0x7f606402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9be1cdd19b9ba87 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7fa5d16b8640 1 --2- 172.21.15.60:0/1130738988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4054b10 0x7fa5c4051ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7fb7bcd6f640 1 -- 172.21.15.60:0/315433459 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8030830 con 0x7fb7b00542d0 2023-11-18T20:42:44.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04802f050 con 0x7ff03c004370 2023-11-18T20:42:44.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff048033610 con 0x7ff03c004370 2023-11-18T20:42:44.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7ff0527fc640 1 -- 172.21.15.60:0/1921168624 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048033a00 con 0x7ff03c004370 2023-11-18T20:42:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7fa3783d5640 1 -- 172.21.15.60:0/1185734801 wait complete. 2023-11-18T20:42:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7f605ffff640 1 -- 172.21.15.60:0/1776251930 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064031640 con 0x7f606815bbe0 2023-11-18T20:42:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7f605ffff640 1 -- 172.21.15.60:0/1776251930 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60640317a0 con 0x7f606815bbe0 2023-11-18T20:42:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7f605ffff640 1 -- 172.21.15.60:0/1776251930 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064031b30 con 0x7f606815bbe0 2023-11-18T20:42:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7fa5d16b8640 1 -- 172.21.15.60:0/1130738988 >> v2:172.21.15.60:3300/0 conn(0x7fa5c4053bc0 msgr2=0x7fa5c4051ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7fa5d16b8640 1 --2- 172.21.15.60:0/1130738988 >> v2:172.21.15.60:3300/0 conn(0x7fa5c4053bc0 0x7fa5c4051ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7fa5d16b8640 1 -- 172.21.15.60:0/1130738988 >> v1:172.21.15.60:6789/0 conn(0x7fa5c40e1270 legacy=0x7fa5c40534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7fa5d16b8640 1 -- 172.21.15.60:0/1130738988 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c4055280 con 0x7fa5c4054b10 2023-11-18T20:42:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7efc85ba9640 1 -- 172.21.15.60:0/326270905 shutdown_connections 2023-11-18T20:42:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7efc85ba9640 1 -- 172.21.15.60:0/326270905 wait complete. 2023-11-18T20:42:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f57c6ef8640 1 -- 172.21.15.60:0/3585319207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57b8053c00 msgr2=0x7f57b8006fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f57c6ef8640 1 --2- 172.21.15.60:0/3585319207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57b8053c00 0x7f57b8006fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f57c0050140 tx=0x7f57c00a5db0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f57c6ef8640 1 -- 172.21.15.60:0/3585319207 shutdown_connections 2023-11-18T20:42:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f57c6ef8640 1 --2- 172.21.15.60:0/3585319207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57b8053c00 0x7f57b8006fe0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f57c6ef8640 1 --2- 172.21.15.60:0/3585319207 >> v2:172.21.15.60:3300/0 conn(0x7f57b80e76d0 0x7f57b804c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7ff05a1b0640 1 -- 172.21.15.60:0/1921168624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff03c004370 msgr2=0x7ff03c004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7ff05a1b0640 1 --2- 172.21.15.60:0/1921168624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff03c004370 0x7ff03c004840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff048001860 tx=0x7ff03c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f57c6ef8640 1 -- 172.21.15.60:0/3585319207 >> 172.21.15.60:0/3585319207 conn(0x7f57b8058ea0 msgr2=0x7f57b8059280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f606f3b0640 1 -- 172.21.15.60:0/1776251930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815bbe0 msgr2=0x7f60680cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f606f3b0640 1 --2- 172.21.15.60:0/1776251930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815bbe0 0x7f60680cebb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6064009870 tx=0x7f606402b8f0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.724+0000 7f872b7fe640 1 --2- 172.21.15.60:0/3310186290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240e73f0 0x7f872405b3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8720009870 tx=0x7f872002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4061f7d93e26f85d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f872a7fc640 1 -- 172.21.15.60:0/3310186290 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720031870 con 0x7f87240e73f0 2023-11-18T20:42:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f872a7fc640 1 -- 172.21.15.60:0/3310186290 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87200319d0 con 0x7f87240e73f0 2023-11-18T20:42:44.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f872a7fc640 1 -- 172.21.15.60:0/3310186290 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720031870 con 0x7f87240e73f0 2023-11-18T20:42:44.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7ff05a1b0640 1 -- 172.21.15.60:0/1921168624 shutdown_connections 2023-11-18T20:42:44.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7ff05a1b0640 1 --2- 172.21.15.60:0/1921168624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff03c004370 0x7ff03c004840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7ff05a1b0640 1 -- 172.21.15.60:0/1921168624 >> 172.21.15.60:0/1921168624 conn(0x7ff05406d3b0 msgr2=0x7ff05406d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:44.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7fb7bcd6f640 1 -- 172.21.15.60:0/315433459 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a8030990 con 0x7fb7b00542d0 2023-11-18T20:42:44.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7fb7bcd6f640 1 -- 172.21.15.60:0/315433459 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8030d20 con 0x7fb7b00542d0 2023-11-18T20:42:44.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7ff05a1b0640 1 -- 172.21.15.60:0/1921168624 shutdown_connections 2023-11-18T20:42:44.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f606f3b0640 1 -- 172.21.15.60:0/1776251930 shutdown_connections 2023-11-18T20:42:44.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f57c6ef8640 1 -- 172.21.15.60:0/3585319207 shutdown_connections 2023-11-18T20:42:44.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f606f3b0640 1 --2- 172.21.15.60:0/1776251930 >> v2:172.21.15.60:3300/0 conn(0x7f606815f280 0x7f606815f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f606f3b0640 1 --2- 172.21.15.60:0/1776251930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815bbe0 0x7f60680cebb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f606f3b0640 1 -- 172.21.15.60:0/1776251930 >> 172.21.15.60:0/1776251930 conn(0x7f60680b2c10 msgr2=0x7f60680b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7ff05a1b0640 1 -- 172.21.15.60:0/1921168624 wait complete. 2023-11-18T20:42:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7f606f3b0640 1 -- 172.21.15.60:0/1776251930 shutdown_connections 2023-11-18T20:42:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7f606f3b0640 1 -- 172.21.15.60:0/1776251930 wait complete. 2023-11-18T20:42:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fb7bed73640 1 -- 172.21.15.60:0/315433459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b00542d0 msgr2=0x7fb7b00068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fb7bed73640 1 --2- 172.21.15.60:0/315433459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b00542d0 0x7fb7b00068a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb7b003be30 tx=0x7fb7a802b970 comp rx=0 tx=0).stop 2023-11-18T20:42:44.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fa3783d5640 1 Processor -- start 2023-11-18T20:42:44.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fa3783d5640 1 -- start start 2023-11-18T20:42:44.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fb7bed73640 1 -- 172.21.15.60:0/315433459 shutdown_connections 2023-11-18T20:42:44.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fb7bed73640 1 --2- 172.21.15.60:0/315433459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b00542d0 0x7fb7b00068a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fb7bed73640 1 --2- 172.21.15.60:0/315433459 >> v2:172.21.15.100:3300/0 conn(0x7fb7b00e3090 0x7fb7b0057d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fb7bed73640 1 -- 172.21.15.60:0/315433459 >> 172.21.15.60:0/315433459 conn(0x7fb7b00559b0 msgr2=0x7fb7b0055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:44.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7f87309fb640 1 -- 172.21.15.60:0/3310186290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240e73f0 msgr2=0x7f872405b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7f87309fb640 1 --2- 172.21.15.60:0/3310186290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240e73f0 0x7f872405b3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8720009870 tx=0x7f872002b910 comp rx=0 tx=0).stop 2023-11-18T20:42:44.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fb7bed73640 1 -- 172.21.15.60:0/315433459 shutdown_connections 2023-11-18T20:42:44.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.725+0000 7f57c6ef8640 1 -- 172.21.15.60:0/3585319207 wait complete. 2023-11-18T20:42:44.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7efc85ba9640 1 Processor -- start 2023-11-18T20:42:44.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fa5d16b8640 1 --2- 172.21.15.60:0/1130738988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4054b10 0x7fa5c4051ab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc000b60 tx=0x7fa5cc0a5d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1f4a3324073fab server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.726+0000 7fa5d0eb7640 1 -- 172.21.15.60:0/1130738988 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0a6bf0 con 0x7fa5c4054b10 2023-11-18T20:42:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7f87309fb640 1 -- 172.21.15.60:0/3310186290 shutdown_connections 2023-11-18T20:42:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7f87309fb640 1 --2- 172.21.15.60:0/3310186290 >> v2:172.21.15.60:3300/0 conn(0x7f87240e77d0 0x7f8724058080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7f87309fb640 1 --2- 172.21.15.60:0/3310186290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240e73f0 0x7f872405b3f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7f87309fb640 1 -- 172.21.15.60:0/3310186290 >> 172.21.15.60:0/3310186290 conn(0x7f8724059d30 msgr2=0x7f872405a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7efc85ba9640 1 -- start start 2023-11-18T20:42:44.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7fa5d0eb7640 1 -- 172.21.15.60:0/1130738988 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc0a6d90 con 0x7fa5c4054b10 2023-11-18T20:42:44.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7fa5d0eb7640 1 -- 172.21.15.60:0/1130738988 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0a6bf0 con 0x7fa5c4054b10 2023-11-18T20:42:44.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7f87309fb640 1 -- 172.21.15.60:0/3310186290 shutdown_connections 2023-11-18T20:42:44.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7f87309fb640 1 -- 172.21.15.60:0/3310186290 wait complete. 2023-11-18T20:42:44.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7ff05a1b0640 1 Processor -- start 2023-11-18T20:42:44.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7f57c6ef8640 1 Processor -- start 2023-11-18T20:42:44.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7f57c6ef8640 1 -- start start 2023-11-18T20:42:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7ff05a1b0640 1 -- start start 2023-11-18T20:42:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7fa3783d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3700b0cf0 0x7fa37013df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa3783d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3700b1150 0x7fa37015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/1130738988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4054b10 msgr2=0x7fa5c4051ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa5d2ebb640 1 --2- 172.21.15.60:0/1130738988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4054b10 0x7fa5c4051ab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc000b60 tx=0x7fa5cc0a5d80 comp rx=0 tx=0).stop 2023-11-18T20:42:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.727+0000 7fb7bed73640 1 -- 172.21.15.60:0/315433459 wait complete. 2023-11-18T20:42:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7f606f3b0640 1 Processor -- start 2023-11-18T20:42:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7f606f3b0640 1 -- start start 2023-11-18T20:42:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa376171640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3700b0cf0 0x7fa37013df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/1130738988 shutdown_connections 2023-11-18T20:42:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa5d2ebb640 1 --2- 172.21.15.60:0/1130738988 >> v2:172.21.15.60:3300/0 conn(0x7fa5c4053bc0 0x7fa5c4051ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa5d2ebb640 1 --2- 172.21.15.60:0/1130738988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4054b10 0x7fa5c4051ab0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/1130738988 >> 172.21.15.60:0/1130738988 conn(0x7fa5c4020000 msgr2=0x7fa5c4056d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa375970640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3700b1150 0x7fa37015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa375970640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3700b1150 0x7fa37015a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41350/0 (socket says 172.21.15.60:41350) 2023-11-18T20:42:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7efc85ba9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8014d270 0x7efc8015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa375970640 1 -- 172.21.15.60:0/173200231 learned_addr learned my addr 172.21.15.60:0/173200231 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/1130738988 shutdown_connections 2023-11-18T20:42:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa3783d5640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa37015a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3701ebd30 con 0x7fa3700b1150 2023-11-18T20:42:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7efc85ba9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc8014d650 0x7efc8015ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7efc7f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8014d270 0x7efc8015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37015aad0 con 0x7fa3700b0cf0 2023-11-18T20:42:44.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f57c6ef8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57b8053c00 0x7f57b804d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3701eb120 con 0x7fa370158690 2023-11-18T20:42:44.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7efc7f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8014d270 0x7efc8015e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41358/0 (socket says 172.21.15.60:41358) 2023-11-18T20:42:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f57c6ef8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57b8058980 0x7f57b804eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f87309fb640 1 Processor -- start 2023-11-18T20:42:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7ff05a1b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff03c004370 0x7ff05415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7ff05a1b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff054159f40 0x7ff05413d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7ff05a1b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0540aff30 0x7ff05414e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f57c6ef8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57b80e76d0 0x7f57b804f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7efc7f7fe640 1 -- 172.21.15.60:0/1684012518 learned_addr learned my addr 172.21.15.60:0/1684012518 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7ff05a1b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05415ab30 con 0x7ff03c004370 2023-11-18T20:42:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7ff05a1b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05415b500 con 0x7ff0540aff30 2023-11-18T20:42:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7ff05a1b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05413ded0 con 0x7ff054159f40 2023-11-18T20:42:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7efc7effd640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc8014d650 0x7efc8015ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7efc85ba9640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc80155ea0 0x7efc80153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.728+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/1130738988 wait complete. 2023-11-18T20:42:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7fb7bed73640 1 Processor -- start 2023-11-18T20:42:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f57c5ef6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57b8053c00 0x7f57b804d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f57c5ef6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57b8053c00 0x7f57b804d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56368/0 (socket says 172.21.15.60:56368) 2023-11-18T20:42:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f87309fb640 1 -- start start 2023-11-18T20:42:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f57c5ef6640 1 -- 172.21.15.60:0/93724988 learned_addr learned my addr 172.21.15.60:0/93724988 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7ff053fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0540aff30 0x7ff05414e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f606f3b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f606815bbe0 0x7f6068149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc800cb380 con 0x7efc8014d270 2023-11-18T20:42:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f57c66f7640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57b80e76d0 0x7f57b804f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc800cbc20 con 0x7efc80155ea0 2023-11-18T20:42:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b804dfe0 con 0x7f57b80e76d0 2023-11-18T20:42:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b804da20 con 0x7f57b8058980 2023-11-18T20:42:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7efc7ffff640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc80155ea0 0x7efc80153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b804dd00 con 0x7f57b8053c00 2023-11-18T20:42:44.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7ff053fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0540aff30 0x7ff05414e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49892/0 (socket says 172.21.15.60:49892) 2023-11-18T20:42:44.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7ff053fff640 1 -- 172.21.15.60:0/461311794 learned_addr learned my addr 172.21.15.60:0/461311794 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7ff052ffd640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff054159f40 0x7ff05413d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8015efd0 con 0x7efc8014d650 2023-11-18T20:42:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f606f3b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f606815bfc0 0x7f606806cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7ff0537fe640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff03c004370 0x7ff05415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f606d14c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f606815bbe0 0x7f6068149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f606d14c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f606815bbe0 0x7f6068149970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49916/0 (socket says 172.21.15.60:49916) 2023-11-18T20:42:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f606f3b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815f280 0x7f60681f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7ff053fff640 1 -- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff054159f40 msgr2=0x7ff05413d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7ff053fff640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff054159f40 0x7ff05413d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7ff053fff640 1 -- 172.21.15.60:0/461311794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff03c004370 msgr2=0x7ff05415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7ff053fff640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff03c004370 0x7ff05415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7ff053fff640 1 -- 172.21.15.60:0/461311794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05413e1b0 con 0x7ff0540aff30 2023-11-18T20:42:44.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f606f3b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606806c770 con 0x7f606815bfc0 2023-11-18T20:42:44.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7fa376972640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa37015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7fa376972640 1 -- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3700b0cf0 msgr2=0x7fa37013df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7fa376972640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3700b0cf0 0x7fa37013df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.729+0000 7fa376972640 1 -- 172.21.15.60:0/173200231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3700b1150 msgr2=0x7fa37015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7fa376972640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3700b1150 0x7fa37015a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7fa376972640 1 -- 172.21.15.60:0/173200231 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0094d0 con 0x7fa370158690 2023-11-18T20:42:44.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f606f3b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606806d140 con 0x7f606815bbe0 2023-11-18T20:42:44.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f606f3b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606814a070 con 0x7f606815f280 2023-11-18T20:42:44.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7fb7bed73640 1 -- start start 2023-11-18T20:42:44.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f606d14c640 1 -- 172.21.15.60:0/3388648359 learned_addr learned my addr 172.21.15.60:0/3388648359 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f57c5ef6640 1 -- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57b8058980 msgr2=0x7f57b804eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f57c5ef6640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57b8058980 0x7f57b804eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f57c5ef6640 1 -- 172.21.15.60:0/93724988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57b80e76d0 msgr2=0x7f57b804f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f57c5ef6640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57b80e76d0 0x7f57b804f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f57c5ef6640 1 -- 172.21.15.60:0/93724988 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c00687b0 con 0x7f57b8053c00 2023-11-18T20:42:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7fa5d2ebb640 1 Processor -- start 2023-11-18T20:42:44.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7efc7f7fe640 1 -- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc8014d650 msgr2=0x7efc8015ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7efc7f7fe640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc8014d650 0x7efc8015ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7efc7f7fe640 1 -- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc80155ea0 msgr2=0x7efc80153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7efc7f7fe640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc80155ea0 0x7efc80153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7efc7f7fe640 1 -- 172.21.15.60:0/1684012518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8014dad0 con 0x7efc8014d270 2023-11-18T20:42:44.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7f606d94d640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815f280 0x7f60681f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f606c94b640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f606815bfc0 0x7f606806cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7fa5d2ebb640 1 -- start start 2023-11-18T20:42:44.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.730+0000 7fa376171640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3700b0cf0 0x7fa37013df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:44.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fa375970640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3700b1150 0x7fa37015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:44.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f57c5ef6640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57b8053c00 0x7f57b804d0b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00b410 tx=0x7f57bc00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7efc7f7fe640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8014d270 0x7efc8015e460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc7000e990 tx=0x7efc7000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7efc67fff640 1 -- 172.21.15.60:0/1684012518 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70009690 con 0x7efc8014d270 2023-11-18T20:42:44.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f606d14c640 1 -- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815f280 msgr2=0x7f60681f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f606d14c640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815f280 0x7f60681f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f606d14c640 1 -- 172.21.15.60:0/3388648359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f606815bfc0 msgr2=0x7f606806cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f606d14c640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f606815bfc0 0x7f606806cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f606d14c640 1 -- 172.21.15.60:0/3388648359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60640094d0 con 0x7f606815bbe0 2023-11-18T20:42:44.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f87309fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f872405b930 0x7f8724180450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f87309fb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240e73f0 0x7f87240e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fb7bed73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b003e810 0x7fb7b0009d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fb7bed73640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb7b00542d0 0x7fb7b000a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fb7bed73640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7b00e3090 0x7fb7b01812e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fb7bed73640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b000c650 con 0x7fb7b003e810 2023-11-18T20:42:44.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fb7bed73640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b000c880 con 0x7fb7b00e3090 2023-11-18T20:42:44.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fb7bed73640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b000cb60 con 0x7fb7b00542d0 2023-11-18T20:42:44.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7ff053fff640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0540aff30 0x7ff05414e430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff05406ff00 tx=0x7ff04c0115a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7ff03bfff640 1 -- 172.21.15.60:0/461311794 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00e070 con 0x7ff0540aff30 2023-11-18T20:42:44.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7ff03bfff640 1 -- 172.21.15.60:0/461311794 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c011e10 con 0x7ff0540aff30 2023-11-18T20:42:44.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7ff03bfff640 1 -- 172.21.15.60:0/461311794 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c003990 con 0x7ff0540aff30 2023-11-18T20:42:44.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fa376972640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa37015a590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa36800be50 tx=0x7fa36800bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7ff05a1b0640 1 -- 172.21.15.60:0/461311794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05415a4e0 con 0x7ff0540aff30 2023-11-18T20:42:44.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc801567f0 con 0x7efc8014d270 2023-11-18T20:42:44.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc800cb8b0 con 0x7efc8014d270 2023-11-18T20:42:44.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f606d14c640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f606815bbe0 0x7f6068149970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6064009870 tx=0x7f60640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f872b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f872405b930 0x7f8724180450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f872b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f872405b930 0x7f8724180450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49924/0 (socket says 172.21.15.60:49924) 2023-11-18T20:42:44.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f872b7fe640 1 -- 172.21.15.60:0/1293981709 learned_addr learned my addr 172.21.15.60:0/1293981709 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fb7be572640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7b00e3090 0x7fb7b01812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fb7be572640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7b00e3090 0x7fb7b01812e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49930/0 (socket says 172.21.15.60:49930) 2023-11-18T20:42:44.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7fb7be572640 1 -- 172.21.15.60:0/1316022358 learned_addr learned my addr 172.21.15.60:0/1316022358 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7ff05a1b0640 1 -- 172.21.15.60:0/461311794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05413db30 con 0x7ff0540aff30 2023-11-18T20:42:44.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7efc67fff640 1 -- 172.21.15.60:0/1684012518 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7000c960 con 0x7efc8014d270 2023-11-18T20:42:44.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7efc67fff640 1 -- 172.21.15.60:0/1684012518 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7000cd10 con 0x7efc8014d270 2023-11-18T20:42:44.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f87309fb640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240e77d0 0x7f8724034900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f872affd640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240e73f0 0x7f87240e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fb7bd570640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb7b00542d0 0x7fb7b000a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fa5d2ebb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4053bc0 0x7fa5c403c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fb7be572640 1 -- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb7b00542d0 msgr2=0x7fb7b000a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fb7be572640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb7b00542d0 0x7fb7b000a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fb7be572640 1 -- 172.21.15.60:0/1316022358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b003e810 msgr2=0x7fb7b0009d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-18T20:42:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fb7be572640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b003e810 0x7fb7b0009d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fa366ffd640 1 -- 172.21.15.60:0/173200231 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368014440 con 0x7fa370158690 2023-11-18T20:42:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fa366ffd640 1 -- 172.21.15.60:0/173200231 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3680145a0 con 0x7fa370158690 2023-11-18T20:42:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fb7be572640 1 -- 172.21.15.60:0/1316022358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a80094d0 con 0x7fb7b00e3090 2023-11-18T20:42:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f57b67fc640 1 -- 172.21.15.60:0/93724988 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc011830 con 0x7f57b8053c00 2023-11-18T20:42:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f57b67fc640 1 -- 172.21.15.60:0/93724988 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc0119b0 con 0x7f57b8053c00 2023-11-18T20:42:44.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.731+0000 7f57b67fc640 1 -- 172.21.15.60:0/93724988 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc011d60 con 0x7f57b8053c00 2023-11-18T20:42:44.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b804e310 con 0x7f57b8053c00 2023-11-18T20:42:44.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b804d5f0 con 0x7f57b8053c00 2023-11-18T20:42:44.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fa366ffd640 1 -- 172.21.15.60:0/173200231 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368014950 con 0x7fa370158690 2023-11-18T20:42:44.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3701eb400 con 0x7fa370158690 2023-11-18T20:42:44.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.732+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3701eb9c0 con 0x7fa370158690 2023-11-18T20:42:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7f605dffb640 1 -- 172.21.15.60:0/3388648359 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606403ab60 con 0x7f606815bbe0 2023-11-18T20:42:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc60005250 con 0x7efc8014d270 2023-11-18T20:42:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7f605dffb640 1 -- 172.21.15.60:0/3388648359 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606403acc0 con 0x7f606815bbe0 2023-11-18T20:42:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7f606f3b0640 1 -- 172.21.15.60:0/3388648359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60680c6710 con 0x7f606815bbe0 2023-11-18T20:42:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7f605dffb640 1 -- 172.21.15.60:0/3388648359 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064044630 con 0x7f606815bbe0 2023-11-18T20:42:44.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7f606f3b0640 1 -- 172.21.15.60:0/3388648359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606806c120 con 0x7f606815bbe0 2023-11-18T20:42:44.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7f872bfff640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240e77d0 0x7f8724034900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240e2170 con 0x7f87240e73f0 2023-11-18T20:42:44.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8724034e40 con 0x7f872405b930 2023-11-18T20:42:44.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240e3440 con 0x7f87240e77d0 2023-11-18T20:42:44.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7fa5d2ebb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c40e1270 0x7fa5c4180900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7fa5d2ebb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c40504b0 0x7fa5c40398c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7fa5d2ebb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c404f410 con 0x7fa5c40504b0 2023-11-18T20:42:44.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7fa5d2ebb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c404f6f0 con 0x7fa5c40e1270 2023-11-18T20:42:44.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7fa5d2ebb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c404f9d0 con 0x7fa5c4053bc0 2023-11-18T20:42:44.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7ff05a1b0640 1 -- 172.21.15.60:0/461311794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0540ce050 con 0x7ff0540aff30 2023-11-18T20:42:44.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7fb7be572640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7b00e3090 0x7fb7b01812e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7a802b9b0 tx=0x7fb7a8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.734+0000 7f872b7fe640 1 -- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240e77d0 msgr2=0x7f8724034900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.734+0000 7f872b7fe640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240e77d0 0x7f8724034900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.734+0000 7f872b7fe640 1 -- 172.21.15.60:0/1293981709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240e73f0 msgr2=0x7f87240e1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.734+0000 7f872b7fe640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240e73f0 0x7f87240e1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.734+0000 7f872b7fe640 1 -- 172.21.15.60:0/1293981709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87200094d0 con 0x7f872405b930 2023-11-18T20:42:44.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.734+0000 7f872bfff640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240e77d0 0x7f8724034900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:44.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.733+0000 7ff03bfff640 1 -- 172.21.15.60:0/461311794 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff04c003af0 con 0x7ff0540aff30 2023-11-18T20:42:44.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.734+0000 7ff03bfff640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff04006ae40 0x7ff04006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.734+0000 7ff03bfff640 1 -- 172.21.15.60:0/461311794 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff04c089c70 con 0x7ff0540aff30 2023-11-18T20:42:44.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7f872affd640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240e73f0 0x7f87240e1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:44.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7f60537fe640 1 -- 172.21.15.60:0/3388648359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6068069400 con 0x7f606815bbe0 2023-11-18T20:42:44.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3701ebee0 con 0x7fa370158690 2023-11-18T20:42:44.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7f872b7fe640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f872405b930 0x7f8724180450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8720000c00 tx=0x7f87200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7efc67fff640 1 -- 172.21.15.60:0/1684012518 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc70013470 con 0x7efc8014d270 2023-11-18T20:42:44.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7fa5d1eb9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4053bc0 0x7fa5c403c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.734+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a8005250 con 0x7f57b8053c00 2023-11-18T20:42:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7f57b67fc640 1 -- 172.21.15.60:0/93724988 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57bc01a400 con 0x7f57b8053c00 2023-11-18T20:42:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7fa5d26ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c40504b0 0x7fa5c40398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7fa5d26ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c40504b0 0x7fa5c40398c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41404/0 (socket says 172.21.15.60:41404) 2023-11-18T20:42:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7fa5d26ba640 1 -- 172.21.15.60:0/3271383447 learned_addr learned my addr 172.21.15.60:0/3271383447 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:44.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7fa5d16b8640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c40e1270 0x7fa5c4180900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7ff0537fe640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff04006ae40 0x7ff04006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7f605dffb640 1 -- 172.21.15.60:0/3388648359 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f606402f050 con 0x7f606815bbe0 2023-11-18T20:42:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7f57b67fc640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57a406af10 0x7f57a406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7fa366ffd640 1 -- 172.21.15.60:0/173200231 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa36801d3b0 con 0x7fa370158690 2023-11-18T20:42:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7fa5d26ba640 1 -- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4053bc0 msgr2=0x7fa5c403c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7fa5d26ba640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4053bc0 0x7fa5c403c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7fa5d26ba640 1 -- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c40e1270 msgr2=0x7fa5c4180900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:44.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7fa5d26ba640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c40e1270 0x7fa5c4180900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:44.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7fa5d26ba640 1 -- 172.21.15.60:0/3271383447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc0687b0 con 0x7fa5c40504b0 2023-11-18T20:42:44.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7efc67fff640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efc6c06ae70 0x7efc6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.735+0000 7f57b67fc640 1 -- 172.21.15.60:0/93724988 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57bc0896f0 con 0x7f57b8053c00 2023-11-18T20:42:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7fa5d16b8640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c40e1270 0x7fa5c4180900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7efc67fff640 1 -- 172.21.15.60:0/1684012518 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc700897b0 con 0x7efc8014d270 2023-11-18T20:42:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7fa366ffd640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa35406af10 0x7fa35406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7ff0537fe640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff04006ae40 0x7ff04006d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff054067ee0 tx=0x7ff044001950 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7f57c56f5640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57a406af10 0x7f57a406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7f605dffb640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f604c06ae90 0x7f604c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7efc7effd640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efc6c06ae70 0x7efc6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7fa366ffd640 1 -- 172.21.15.60:0/173200231 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa368089770 con 0x7fa370158690 2023-11-18T20:42:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7fa5d26ba640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c40504b0 0x7fa5c40398c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e870 tx=0x7fa5bc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7f605dffb640 1 -- 172.21.15.60:0/3388648359 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60640ab600 con 0x7f606815bbe0 2023-11-18T20:42:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7fa376171640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa35406af10 0x7fa35406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7f606c94b640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f604c06ae90 0x7f604c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7efc7effd640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efc6c06ae70 0x7efc6c06d330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc74034000 tx=0x7efc74007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.736+0000 7f57c56f5640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57a406af10 0x7f57a406d3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f57b80e0fe0 tx=0x7f57b0007990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.737+0000 7fa376171640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa35406af10 0x7fa35406d3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa36c036000 tx=0x7fa36c03b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.737+0000 7f606c94b640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f604c06ae90 0x7f604c06d350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60680b3520 tx=0x7f6058009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.740+0000 7fa5c27fc640 1 -- 172.21.15.60:0/3271383447 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00ee60 con 0x7fa5c40504b0 2023-11-18T20:42:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.740+0000 7fa5c27fc640 1 -- 172.21.15.60:0/3271383447 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc00c940 con 0x7fa5c40504b0 2023-11-18T20:42:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.740+0000 7fa5c27fc640 1 -- 172.21.15.60:0/3271383447 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00ccf0 con 0x7fa5c40504b0 2023-11-18T20:42:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.740+0000 7efc67fff640 1 -- 172.21.15.60:0/1684012518 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc70054870 con 0x7efc8014d270 2023-11-18T20:42:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.740+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/3271383447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c404fcb0 con 0x7fa5c40504b0 2023-11-18T20:42:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.740+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/3271383447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c403cac0 con 0x7fa5c40504b0 2023-11-18T20:42:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.741+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/3271383447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c40057a0 con 0x7fa5c40504b0 2023-11-18T20:42:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.742+0000 7fa5c27fc640 1 -- 172.21.15.60:0/3271383447 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa5bc013470 con 0x7fa5c40504b0 2023-11-18T20:42:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.742+0000 7fa5c27fc640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa5b006ae90 0x7fa5b006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.743+0000 7fa5c27fc640 1 -- 172.21.15.60:0/3271383447 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5bc089560 con 0x7fa5c40504b0 2023-11-18T20:42:44.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.743+0000 7fa5d1eb9640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa5b006ae90 0x7fa5b006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.743+0000 7fb7ae7fc640 1 -- 172.21.15.60:0/1316022358 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8039a90 con 0x7fb7b00e3090 2023-11-18T20:42:44.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.743+0000 7fb7bed73640 1 -- 172.21.15.60:0/1316022358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b000ce40 con 0x7fb7b00e3090 2023-11-18T20:42:44.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.743+0000 7ff03bfff640 1 -- 172.21.15.60:0/461311794 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff0540ce050 con 0x7ff0540aff30 2023-11-18T20:42:44.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.743+0000 7fb7bed73640 1 -- 172.21.15.60:0/1316022358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b000d640 con 0x7fb7b00e3090 2023-11-18T20:42:44.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.743+0000 7fa5d1eb9640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa5b006ae90 0x7fa5b006d350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4038f10 tx=0x7fa5c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.744+0000 7fb7bed73640 1 -- 172.21.15.60:0/1316022358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b00057b0 con 0x7fb7b00e3090 2023-11-18T20:42:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.746+0000 7f57b67fc640 1 -- 172.21.15.60:0/93724988 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57bc054810 con 0x7f57b8053c00 2023-11-18T20:42:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.747+0000 7fa5c27fc640 1 -- 172.21.15.60:0/3271383447 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa5bc054730 con 0x7fa5c40504b0 2023-11-18T20:42:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.752+0000 7f605dffb640 1 -- 172.21.15.60:0/3388648359 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6064076790 con 0x7f606815bbe0 2023-11-18T20:42:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.752+0000 7f8717fff640 1 -- 172.21.15.60:0/1293981709 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720031870 con 0x7f872405b930 2023-11-18T20:42:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.752+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87240e23a0 con 0x7f872405b930 2023-11-18T20:42:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.752+0000 7f8717fff640 1 -- 172.21.15.60:0/1293981709 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872003ad40 con 0x7f872405b930 2023-11-18T20:42:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.752+0000 7f8717fff640 1 -- 172.21.15.60:0/1293981709 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720031870 con 0x7f872405b930 2023-11-18T20:42:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.752+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87240e2960 con 0x7f872405b930 2023-11-18T20:42:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.752+0000 7fb7ae7fc640 1 -- 172.21.15.60:0/1316022358 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a8039bf0 con 0x7fb7b00e3090 2023-11-18T20:42:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.752+0000 7fb7ae7fc640 1 -- 172.21.15.60:0/1316022358 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a804d460 con 0x7fb7b00e3090 2023-11-18T20:42:44.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.754+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870c005250 con 0x7f872405b930 2023-11-18T20:42:44.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.754+0000 7fa366ffd640 1 -- 172.21.15.60:0/173200231 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3701ebee0 con 0x7fa370158690 2023-11-18T20:42:44.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.755+0000 7fb7ae7fc640 1 -- 172.21.15.60:0/1316022358 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb7a804d5c0 con 0x7fb7b00e3090 2023-11-18T20:42:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.755+0000 7f8717fff640 1 -- 172.21.15.60:0/1293981709 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f872004d450 con 0x7f872405b930 2023-11-18T20:42:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.755+0000 7f8717fff640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f871006af00 0x7f871006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.756+0000 7f872affd640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f871006af00 0x7f871006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.756+0000 7fb7ae7fc640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb79c06ae40 0x7fb79c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:44.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.756+0000 7fb7ae7fc640 1 -- 172.21.15.60:0/1316022358 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7a803e070 con 0x7fb7b00e3090 2023-11-18T20:42:44.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.756+0000 7f872affd640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f871006af00 0x7f871006d3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8718006fd0 tx=0x7f8718008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.756+0000 7fb7bdd71640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb79c06ae40 0x7fb79c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:44.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.757+0000 7fb7bdd71640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb79c06ae40 0x7fb79c06d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb7b802cee0 tx=0x7fb7b806d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:44.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.758+0000 7fb7ae7fc640 1 -- 172.21.15.60:0/1316022358 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7a8078120 con 0x7fb7b00e3090 2023-11-18T20:42:44.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.758+0000 7f8717fff640 1 -- 172.21.15.60:0/1293981709 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f872002d040 con 0x7f872405b930 2023-11-18T20:42:44.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:44.763+0000 7f8717fff640 1 -- 172.21.15.60:0/1293981709 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8720076810 con 0x7f872405b930 2023-11-18T20:42:45.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.146+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efc60005500 con 0x7efc8014d270 2023-11-18T20:42:45.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.147+0000 7efc67fff640 1 -- 172.21.15.60:0/1684012518 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efc70021020 con 0x7efc8014d270 2023-11-18T20:42:45.148 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-11-18T20:42:45.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.148+0000 7ff05a1b0640 1 -- 172.21.15.60:0/461311794 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff0540ce050 con 0x7ff0540aff30 2023-11-18T20:42:45.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.149+0000 7ff03bfff640 1 -- 172.21.15.60:0/461311794 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0540ce050 con 0x7ff0540aff30 2023-11-18T20:42:45.150 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-11-18T20:42:45.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.150+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efc6c06ae70 msgr2=0x7efc6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.150+0000 7efc85ba9640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efc6c06ae70 0x7efc6c06d330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc74034000 tx=0x7efc74007000 comp rx=0 tx=0).stop 2023-11-18T20:42:45.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.151+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8014d270 msgr2=0x7efc8015e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.151+0000 7efc85ba9640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8014d270 0x7efc8015e460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc7000e990 tx=0x7efc7000ee50 comp rx=0 tx=0).stop 2023-11-18T20:42:45.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.151+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 shutdown_connections 2023-11-18T20:42:45.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.151+0000 7efc85ba9640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efc6c06ae70 0x7efc6c06d330 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.151+0000 7efc85ba9640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc80155ea0 0x7efc80153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.151+0000 7efc85ba9640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc8014d650 0x7efc8015ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.151+0000 7efc85ba9640 1 --2- 172.21.15.60:0/1684012518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8014d270 0x7efc8015e460 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.151+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 >> 172.21.15.60:0/1684012518 conn(0x7efc80072020 msgr2=0x7efc800b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 shutdown_connections 2023-11-18T20:42:45.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7efc85ba9640 1 -- 172.21.15.60:0/1684012518 wait complete. 2023-11-18T20:42:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7ff039ffb640 1 -- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff04006ae40 msgr2=0x7ff04006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7ff039ffb640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff04006ae40 0x7ff04006d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff054067ee0 tx=0x7ff044001950 comp rx=0 tx=0).stop 2023-11-18T20:42:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7ff039ffb640 1 -- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0540aff30 msgr2=0x7ff05414e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7ff039ffb640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0540aff30 0x7ff05414e430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff05406ff00 tx=0x7ff04c0115a0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7ff039ffb640 1 -- 172.21.15.60:0/461311794 shutdown_connections 2023-11-18T20:42:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7ff039ffb640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff04006ae40 0x7ff04006d300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7ff039ffb640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0540aff30 0x7ff05414e430 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7ff039ffb640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff054159f40 0x7ff05413d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7ff039ffb640 1 --2- 172.21.15.60:0/461311794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff03c004370 0x7ff05415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.152+0000 7ff039ffb640 1 -- 172.21.15.60:0/461311794 >> 172.21.15.60:0/461311794 conn(0x7ff05406d3b0 msgr2=0x7ff05406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.153+0000 7ff039ffb640 1 -- 172.21.15.60:0/461311794 shutdown_connections 2023-11-18T20:42:45.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.153+0000 7ff039ffb640 1 -- 172.21.15.60:0/461311794 wait complete. 2023-11-18T20:42:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.159+0000 7f60537fe640 1 -- 172.21.15.60:0/3388648359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f606806d7f0 con 0x7f606815bbe0 2023-11-18T20:42:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.160+0000 7f605dffb640 1 -- 172.21.15.60:0/3388648359 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6064079db0 con 0x7f606815bbe0 2023-11-18T20:42:45.161 INFO:teuthology.orchestra.run.smithi060.stdout:55834574853 2023-11-18T20:42:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.163+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f57a8005500 con 0x7f57b8053c00 2023-11-18T20:42:45.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.163+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/3271383447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa5c40e4ad0 con 0x7fa5c40504b0 2023-11-18T20:42:45.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 -- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f604c06ae90 msgr2=0x7f604c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f604c06ae90 0x7f604c06d350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60680b3520 tx=0x7f6058009040 comp rx=0 tx=0).stop 2023-11-18T20:42:45.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 -- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f606815bbe0 msgr2=0x7f6068149970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f606815bbe0 0x7f6068149970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6064009870 tx=0x7f60640046c0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 -- 172.21.15.60:0/3388648359 shutdown_connections 2023-11-18T20:42:45.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f57b67fc640 1 -- 172.21.15.60:0/93724988 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f57bc057e30 con 0x7f57b8053c00 2023-11-18T20:42:45.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7fa5c27fc640 1 -- 172.21.15.60:0/3271383447 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5bc057d50 con 0x7fa5c40504b0 2023-11-18T20:42:45.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f604c06ae90 0x7f604c06d350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815f280 0x7f60681f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f606815bfc0 0x7f606806cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 --2- 172.21.15.60:0/3388648359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f606815bbe0 0x7f6068149970 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6064009870 tx=0x7f60640046c0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 -- 172.21.15.60:0/3388648359 >> 172.21.15.60:0/3388648359 conn(0x7f60680b2c10 msgr2=0x7f606806e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.167 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-11-18T20:42:45.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 -- 172.21.15.60:0/3388648359 shutdown_connections 2023-11-18T20:42:45.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.164+0000 7f60537fe640 1 -- 172.21.15.60:0/3388648359 wait complete. 2023-11-18T20:42:45.168 INFO:teuthology.orchestra.run.smithi060.stdout:55834574855 2023-11-18T20:42:45.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57a406af10 msgr2=0x7f57a406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7f57c6ef8640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57a406af10 0x7f57a406d3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f57b80e0fe0 tx=0x7f57b0007990 comp rx=0 tx=0).stop 2023-11-18T20:42:45.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57b8053c00 msgr2=0x7f57b804d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7f57c6ef8640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57b8053c00 0x7f57b804d0b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00b410 tx=0x7f57bc00b8d0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 shutdown_connections 2023-11-18T20:42:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7f57c6ef8640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f57a406af10 0x7f57a406d3d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7f57c6ef8640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57b80e76d0 0x7f57b804f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa5b006ae90 msgr2=0x7fa5b006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7fa5d2ebb640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa5b006ae90 0x7fa5b006d350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4038f10 tx=0x7fa5c8009040 comp rx=0 tx=0).stop 2023-11-18T20:42:45.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/3271383447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c40504b0 msgr2=0x7fa5c40398c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7fa5d2ebb640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c40504b0 0x7fa5c40398c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e870 tx=0x7fa5bc00ed30 comp rx=0 tx=0).stop 2023-11-18T20:42:45.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/3271383447 shutdown_connections 2023-11-18T20:42:45.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7fa5d2ebb640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa5b006ae90 0x7fa5b006d350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7fa5d2ebb640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c40504b0 0x7fa5c40398c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7fa5d2ebb640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5c40e1270 0x7fa5c4180900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.168+0000 7fa5d2ebb640 1 --2- 172.21.15.60:0/3271383447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5c4053bc0 0x7fa5c403c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.169+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/3271383447 >> 172.21.15.60:0/3271383447 conn(0x7fa5c4020000 msgr2=0x7fa5c4008f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.169+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/3271383447 shutdown_connections 2023-11-18T20:42:45.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.169+0000 7fa5d2ebb640 1 -- 172.21.15.60:0/3271383447 wait complete. 2023-11-18T20:42:45.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.169+0000 7f57c6ef8640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57b8058980 0x7f57b804eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.169+0000 7f57c6ef8640 1 --2- 172.21.15.60:0/93724988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57b8053c00 0x7f57b804d0b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.169+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 >> 172.21.15.60:0/93724988 conn(0x7f57b8058ea0 msgr2=0x7f57b8059280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.169+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 shutdown_connections 2023-11-18T20:42:45.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.170+0000 7f57c6ef8640 1 -- 172.21.15.60:0/93724988 wait complete. 2023-11-18T20:42:45.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.171+0000 7fb7bed73640 1 -- 172.21.15.60:0/1316022358 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb7b000c9c0 con 0x7fb7b00e3090 2023-11-18T20:42:45.174 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-11-18T20:42:45.174 DEBUG:teuthology.parallel:result is None 2023-11-18T20:42:45.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2023-11-18T20:42:45.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.171+0000 7fb7ae7fc640 1 -- 172.21.15.60:0/1316022358 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb7b000c9c0 con 0x7fb7b00e3090 2023-11-18T20:42:45.175 DEBUG:teuthology.parallel:result is None 2023-11-18T20:42:45.175 INFO:teuthology.orchestra.run.smithi060.stdout:55834574855 2023-11-18T20:42:45.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.172+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa3701ebee0 con 0x7fa370158690 2023-11-18T20:42:45.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.172+0000 7fa366ffd640 1 -- 172.21.15.60:0/173200231 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa368019020 con 0x7fa370158690 2023-11-18T20:42:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-11-18T20:42:45.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa35406af10 msgr2=0x7fa35406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa35406af10 0x7fa35406d3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa36c036000 tx=0x7fa36c03b040 comp rx=0 tx=0).stop 2023-11-18T20:42:45.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 msgr2=0x7fa37015a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa37015a590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa36800be50 tx=0x7fa36800bf40 comp rx=0 tx=0).stop 2023-11-18T20:42:45.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 -- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb79c06ae40 msgr2=0x7fb79c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 shutdown_connections 2023-11-18T20:42:45.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa35406af10 0x7fa35406d3d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa370158690 0x7fa37015a590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3700b1150 0x7fa37015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 --2- 172.21.15.60:0/173200231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3700b0cf0 0x7fa37013df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 >> 172.21.15.60:0/173200231 conn(0x7fa37006a4d0 msgr2=0x7fa3700cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb79c06ae40 0x7fb79c06d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb7b802cee0 tx=0x7fb7b806d040 comp rx=0 tx=0).stop 2023-11-18T20:42:45.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 shutdown_connections 2023-11-18T20:42:45.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 -- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7b00e3090 msgr2=0x7fb7b01812e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fa3783d5640 1 -- 172.21.15.60:0/173200231 wait complete. 2023-11-18T20:42:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7b00e3090 0x7fb7b01812e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7a802b9b0 tx=0x7fb7a8004730 comp rx=0 tx=0).stop 2023-11-18T20:42:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 -- 172.21.15.60:0/1316022358 shutdown_connections 2023-11-18T20:42:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb79c06ae40 0x7fb79c06d300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7b00e3090 0x7fb7b01812e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb7b00542d0 0x7fb7b000a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 --2- 172.21.15.60:0/1316022358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7b003e810 0x7fb7b0009d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 -- 172.21.15.60:0/1316022358 >> 172.21.15.60:0/1316022358 conn(0x7fb7b00559b0 msgr2=0x7fb7b0007340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 -- 172.21.15.60:0/1316022358 shutdown_connections 2023-11-18T20:42:45.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.175+0000 7fb7bed73640 1 -- 172.21.15.60:0/1316022358 wait complete. 2023-11-18T20:42:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.177+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f870c005500 con 0x7f872405b930 2023-11-18T20:42:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.177+0000 7f8717fff640 1 -- 172.21.15.60:0/1293981709 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8720035020 con 0x7f872405b930 2023-11-18T20:42:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:55834574854 2023-11-18T20:42:45.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.181+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f871006af00 msgr2=0x7f871006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.181+0000 7f87309fb640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f871006af00 0x7f871006d3c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8718006fd0 tx=0x7f8718008040 comp rx=0 tx=0).stop 2023-11-18T20:42:45.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.181+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f872405b930 msgr2=0x7f8724180450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.181+0000 7f87309fb640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f872405b930 0x7f8724180450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8720000c00 tx=0x7f87200046c0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2023-11-18T20:42:45.184 DEBUG:teuthology.parallel:result is None 2023-11-18T20:42:45.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.182+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 shutdown_connections 2023-11-18T20:42:45.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.182+0000 7f87309fb640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f871006af00 0x7f871006d3c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.182+0000 7f87309fb640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87240e77d0 0x7f8724034900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.182+0000 7f87309fb640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240e73f0 0x7f87240e1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.182+0000 7f87309fb640 1 --2- 172.21.15.60:0/1293981709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f872405b930 0x7f8724180450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.182+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 >> 172.21.15.60:0/1293981709 conn(0x7f8724059d30 msgr2=0x7f8724057b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.182+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 shutdown_connections 2023-11-18T20:42:45.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.182+0000 7f87309fb640 1 -- 172.21.15.60:0/1293981709 wait complete. 2023-11-18T20:42:45.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-11-18T20:42:45.189 DEBUG:teuthology.parallel:result is None 2023-11-18T20:42:45.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-11-18T20:42:45.193 DEBUG:teuthology.parallel:result is None 2023-11-18T20:42:45.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-11-18T20:42:45.194 DEBUG:teuthology.parallel:result is None 2023-11-18T20:42:45.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-11-18T20:42:45.195 DEBUG:teuthology.parallel:result is None 2023-11-18T20:42:45.197 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-11-18T20:42:45.197 DEBUG:teuthology.parallel:result is None 2023-11-18T20:42:45.197 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-18T20:42:45.197 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:42:45.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.343+0000 7ff29d58a640 1 Processor -- start 2023-11-18T20:42:45.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.343+0000 7ff29d58a640 1 -- start start 2023-11-18T20:42:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.344+0000 7ff29d58a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff29815b130 0x7ff2980d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.344+0000 7ff29d58a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff29815f760 0x7ff2980cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.344+0000 7ff29d58a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2980cf560 con 0x7ff2980cf150 2023-11-18T20:42:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.344+0000 7ff29d58a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29815fd20 con 0x7ff29815b130 2023-11-18T20:42:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.344+0000 7ff29d58a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2980d0730 con 0x7ff29815f760 2023-11-18T20:42:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.344+0000 7ff296ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff29815b130 0x7ff2980d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.344+0000 7ff296ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff29815b130 0x7ff2980d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41408/0 (socket says 172.21.15.60:41408) 2023-11-18T20:42:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.344+0000 7ff296ffd640 1 -- 172.21.15.60:0/56847139 learned_addr learned my addr 172.21.15.60:0/56847139 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.344+0000 7ff2967fc640 1 --2- 172.21.15.60:0/56847139 >> v2:172.21.15.100:3300/0 conn(0x7ff29815f760 0x7ff2980cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.345+0000 7ff296ffd640 1 -- 172.21.15.60:0/56847139 >> v2:172.21.15.100:3300/0 conn(0x7ff29815f760 msgr2=0x7ff2980cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.345+0000 7ff296ffd640 1 --2- 172.21.15.60:0/56847139 >> v2:172.21.15.100:3300/0 conn(0x7ff29815f760 0x7ff2980cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.345+0000 7ff296ffd640 1 -- 172.21.15.60:0/56847139 >> v1:172.21.15.100:6789/0 conn(0x7ff2980cf150 legacy=0x7ff29806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.345+0000 7ff296ffd640 1 -- 172.21.15.60:0/56847139 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980b3f10 con 0x7ff29815b130 2023-11-18T20:42:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.345+0000 7ff296ffd640 1 --2- 172.21.15.60:0/56847139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29815b130 0x7ff2980d01f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff2980b71b0 tx=0x7ff28802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b85047e45d173a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:45.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.345+0000 7ff295ffb640 1 -- 172.21.15.60:0/56847139 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28802ca80 con 0x7ff29815b130 2023-11-18T20:42:45.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.345+0000 7ff295ffb640 1 -- 172.21.15.60:0/56847139 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28802cbe0 con 0x7ff29815b130 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.346+0000 7ff295ffb640 1 -- 172.21.15.60:0/56847139 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880343f0 con 0x7ff29815b130 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.346+0000 7ff29d58a640 1 -- 172.21.15.60:0/56847139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29815b130 msgr2=0x7ff2980d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.346+0000 7ff29d58a640 1 --2- 172.21.15.60:0/56847139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29815b130 0x7ff2980d01f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff2980b71b0 tx=0x7ff28802bee0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.346+0000 7ff29d58a640 1 -- 172.21.15.60:0/56847139 shutdown_connections 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.346+0000 7ff29d58a640 1 --2- 172.21.15.60:0/56847139 >> v2:172.21.15.100:3300/0 conn(0x7ff29815f760 0x7ff2980cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.346+0000 7ff29d58a640 1 --2- 172.21.15.60:0/56847139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff29815b130 0x7ff2980d01f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.346+0000 7ff29d58a640 1 -- 172.21.15.60:0/56847139 >> 172.21.15.60:0/56847139 conn(0x7ff2980b2c10 msgr2=0x7ff2980b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.346+0000 7ff29d58a640 1 -- 172.21.15.60:0/56847139 shutdown_connections 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.346+0000 7ff29d58a640 1 -- 172.21.15.60:0/56847139 wait complete. 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.347+0000 7ff29d58a640 1 Processor -- start 2023-11-18T20:42:45.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.347+0000 7ff29d58a640 1 -- start start 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff29d58a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2980cf150 0x7ff2981559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff29d58a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff29815b130 0x7ff2980c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff29d58a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff29815f760 0x7ff2980b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff29d58a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff298155f20 con 0x7ff2980cf150 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff29d58a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980c7800 con 0x7ff29815f760 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff29d58a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29806b5e0 con 0x7ff29815b130 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff296ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2980cf150 0x7ff2981559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff296ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2980cf150 0x7ff2981559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41418/0 (socket says 172.21.15.60:41418) 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff296ffd640 1 -- 172.21.15.60:0/432688601 learned_addr learned my addr 172.21.15.60:0/432688601 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff2977fe640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff29815f760 0x7ff2980b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff2967fc640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff29815b130 0x7ff2980c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff296ffd640 1 -- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff29815b130 msgr2=0x7ff2980c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff296ffd640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff29815b130 0x7ff2980c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff296ffd640 1 -- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff29815f760 msgr2=0x7ff2980b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff296ffd640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff29815f760 0x7ff2980b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff296ffd640 1 -- 172.21.15.60:0/432688601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff288009b30 con 0x7ff2980cf150 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.348+0000 7ff2977fe640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff29815f760 0x7ff2980b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.349+0000 7ff296ffd640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2980cf150 0x7ff2981559e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff288000f80 tx=0x7ff288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.349+0000 7ff27f7fe640 1 -- 172.21.15.60:0/432688601 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288009510 con 0x7ff2980cf150 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.349+0000 7ff27f7fe640 1 -- 172.21.15.60:0/432688601 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28803e450 con 0x7ff2980cf150 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.349+0000 7ff29d58a640 1 -- 172.21.15.60:0/432688601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29806b8c0 con 0x7ff2980cf150 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.349+0000 7ff27f7fe640 1 -- 172.21.15.60:0/432688601 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28803e800 con 0x7ff2980cf150 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.349+0000 7ff29d58a640 1 -- 172.21.15.60:0/432688601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29806be80 con 0x7ff2980cf150 2023-11-18T20:42:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.350+0000 7ff29d58a640 1 -- 172.21.15.60:0/432688601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff298067fc0 con 0x7ff2980cf150 2023-11-18T20:42:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.350+0000 7ff27f7fe640 1 -- 172.21.15.60:0/432688601 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff28802cd50 con 0x7ff2980cf150 2023-11-18T20:42:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.351+0000 7ff27f7fe640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff28406ae40 0x7ff28406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.351+0000 7ff27f7fe640 1 -- 172.21.15.60:0/432688601 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2880ac1b0 con 0x7ff2980cf150 2023-11-18T20:42:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.351+0000 7ff2967fc640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff28406ae40 0x7ff28406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.351+0000 7ff2967fc640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff28406ae40 0x7ff28406d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff28c006fd0 tx=0x7ff28c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:45.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.355+0000 7ff27f7fe640 1 -- 172.21.15.60:0/432688601 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff2880772f0 con 0x7ff2980cf150 2023-11-18T20:42:45.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.531+0000 7ff29d58a640 1 -- 172.21.15.60:0/432688601 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2980cb690 con 0x7ff28406ae40 2023-11-18T20:42:45.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.538+0000 7ff27f7fe640 1 -- 172.21.15.60:0/432688601 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264547 (secure 0 0 0) 0x7ff28c00f830 con 0x7ff28406ae40 2023-11-18T20:42:45.542 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:42:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 -- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff28406ae40 msgr2=0x7ff28406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff28406ae40 0x7ff28406d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff28c006fd0 tx=0x7ff28c008040 comp rx=0 tx=0).stop 2023-11-18T20:42:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 -- 172.21.15.60:0/432688601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2980cf150 msgr2=0x7ff2981559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2980cf150 0x7ff2981559e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff288000f80 tx=0x7ff288004a40 comp rx=0 tx=0).stop 2023-11-18T20:42:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 -- 172.21.15.60:0/432688601 shutdown_connections 2023-11-18T20:42:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff28406ae40 0x7ff28406d300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff29815f760 0x7ff2980b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff29815b130 0x7ff2980c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 --2- 172.21.15.60:0/432688601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2980cf150 0x7ff2981559e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 -- 172.21.15.60:0/432688601 >> 172.21.15.60:0/432688601 conn(0x7ff2980b2c10 msgr2=0x7ff29815ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 -- 172.21.15.60:0/432688601 shutdown_connections 2023-11-18T20:42:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.541+0000 7ff29d58a640 1 -- 172.21.15.60:0/432688601 wait complete. 2023-11-18T20:42:45.553 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-18T20:42:45.437674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2916,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":2985984,"data_stored":1789585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002155"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084419+0000","last_change":"2023-11-18T20:42:38.103913+0000","last_active":"2023-11-18T20:42:39.084419+0000","last_peered":"2023-11-18T20:42:39.084419+0000","last_clean":"2023-11-18T20:42:39.084419+0000","last_became_active":"2023-11-18T20:42:38.103117+0000","last_became_peered":"2023-11-18T20:42:38.103117+0000","last_unstale":"2023-11-18T20:42:39.084419+0000","last_undegraded":"2023-11-18T20:42:39.084419+0000","last_fullsized":"2023-11-18T20:42:39.084419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:48:49.998802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085771+0000","last_change":"2023-11-18T20:42:38.089225+0000","last_active":"2023-11-18T20:42:39.085771+0000","last_peered":"2023-11-18T20:42:39.085771+0000","last_clean":"2023-11-18T20:42:39.085771+0000","last_became_active":"2023-11-18T20:42:38.088874+0000","last_became_peered":"2023-11-18T20:42:38.088874+0000","last_unstale":"2023-11-18T20:42:39.085771+0000","last_undegraded":"2023-11-18T20:42:39.085771+0000","last_fullsized":"2023-11-18T20:42:39.085771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:12:16.083479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085159+0000","last_change":"2023-11-18T20:42:38.090166+0000","last_active":"2023-11-18T20:42:39.085159+0000","last_peered":"2023-11-18T20:42:39.085159+0000","last_clean":"2023-11-18T20:42:39.085159+0000","last_became_active":"2023-11-18T20:42:38.089879+0000","last_became_peered":"2023-11-18T20:42:38.089879+0000","last_unstale":"2023-11-18T20:42:39.085159+0000","last_undegraded":"2023-11-18T20:42:39.085159+0000","last_fullsized":"2023-11-18T20:42:39.085159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:02.525692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086389+0000","last_change":"2023-11-18T20:42:38.105690+0000","last_active":"2023-11-18T20:42:39.086389+0000","last_peered":"2023-11-18T20:42:39.086389+0000","last_clean":"2023-11-18T20:42:39.086389+0000","last_became_active":"2023-11-18T20:42:38.105419+0000","last_became_peered":"2023-11-18T20:42:38.105419+0000","last_unstale":"2023-11-18T20:42:39.086389+0000","last_undegraded":"2023-11-18T20:42:39.086389+0000","last_fullsized":"2023-11-18T20:42:39.086389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:35:56.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084162+0000","last_change":"2023-11-18T20:42:38.106388+0000","last_active":"2023-11-18T20:42:39.084162+0000","last_peered":"2023-11-18T20:42:39.084162+0000","last_clean":"2023-11-18T20:42:39.084162+0000","last_became_active":"2023-11-18T20:42:38.105875+0000","last_became_peered":"2023-11-18T20:42:38.105875+0000","last_unstale":"2023-11-18T20:42:39.084162+0000","last_undegraded":"2023-11-18T20:42:39.084162+0000","last_fullsized":"2023-11-18T20:42:39.084162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:35:58.132615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086539+0000","last_change":"2023-11-18T20:42:38.104968+0000","last_active":"2023-11-18T20:42:39.086539+0000","last_peered":"2023-11-18T20:42:39.086539+0000","last_clean":"2023-11-18T20:42:39.086539+0000","last_became_active":"2023-11-18T20:42:38.104671+0000","last_became_peered":"2023-11-18T20:42:38.104671+0000","last_unstale":"2023-11-18T20:42:39.086539+0000","last_undegraded":"2023-11-18T20:42:39.086539+0000","last_fullsized":"2023-11-18T20:42:39.086539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:51:04.970201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083359+0000","last_change":"2023-11-18T20:42:38.101566+0000","last_active":"2023-11-18T20:42:39.083359+0000","last_peered":"2023-11-18T20:42:39.083359+0000","last_clean":"2023-11-18T20:42:39.083359+0000","last_became_active":"2023-11-18T20:42:38.101316+0000","last_became_peered":"2023-11-18T20:42:38.101316+0000","last_unstale":"2023-11-18T20:42:39.083359+0000","last_undegraded":"2023-11-18T20:42:39.083359+0000","last_fullsized":"2023-11-18T20:42:39.083359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:28.378102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084443+0000","last_change":"2023-11-18T20:42:38.102722+0000","last_active":"2023-11-18T20:42:39.084443+0000","last_peered":"2023-11-18T20:42:39.084443+0000","last_clean":"2023-11-18T20:42:39.084443+0000","last_became_active":"2023-11-18T20:42:38.102348+0000","last_became_peered":"2023-11-18T20:42:38.102348+0000","last_unstale":"2023-11-18T20:42:39.084443+0000","last_undegraded":"2023-11-18T20:42:39.084443+0000","last_fullsized":"2023-11-18T20:42:39.084443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:02.525692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084379+0000","last_change":"2023-11-18T20:42:38.107273+0000","last_active":"2023-11-18T20:42:39.084379+0000","last_peered":"2023-11-18T20:42:39.084379+0000","last_clean":"2023-11-18T20:42:39.084379+0000","last_became_active":"2023-11-18T20:42:38.107006+0000","last_became_peered":"2023-11-18T20:42:38.107006+0000","last_unstale":"2023-11-18T20:42:39.084379+0000","last_undegraded":"2023-11-18T20:42:39.084379+0000","last_fullsized":"2023-11-18T20:42:39.084379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:44:25.920568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086226+0000","last_change":"2023-11-18T20:42:38.105599+0000","last_active":"2023-11-18T20:42:39.086226+0000","last_peered":"2023-11-18T20:42:39.086226+0000","last_clean":"2023-11-18T20:42:39.086226+0000","last_became_active":"2023-11-18T20:42:38.105338+0000","last_became_peered":"2023-11-18T20:42:38.105338+0000","last_unstale":"2023-11-18T20:42:39.086226+0000","last_undegraded":"2023-11-18T20:42:39.086226+0000","last_fullsized":"2023-11-18T20:42:39.086226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086057+0000","last_change":"2023-11-18T20:42:38.100971+0000","last_active":"2023-11-18T20:42:39.086057+0000","last_peered":"2023-11-18T20:42:39.086057+0000","last_clean":"2023-11-18T20:42:39.086057+0000","last_became_active":"2023-11-18T20:42:38.100686+0000","last_became_peered":"2023-11-18T20:42:38.100686+0000","last_unstale":"2023-11-18T20:42:39.086057+0000","last_undegraded":"2023-11-18T20:42:39.086057+0000","last_fullsized":"2023-11-18T20:42:39.086057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083618+0000","last_change":"2023-11-18T20:42:38.098826+0000","last_active":"2023-11-18T20:42:39.083618+0000","last_peered":"2023-11-18T20:42:39.083618+0000","last_clean":"2023-11-18T20:42:39.083618+0000","last_became_active":"2023-11-18T20:42:38.098277+0000","last_became_peered":"2023-11-18T20:42:38.098277+0000","last_unstale":"2023-11-18T20:42:39.083618+0000","last_undegraded":"2023-11-18T20:42:39.083618+0000","last_fullsized":"2023-11-18T20:42:39.083618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:13.814862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085307+0000","last_change":"2023-11-18T20:42:38.098784+0000","last_active":"2023-11-18T20:42:39.085307+0000","last_peered":"2023-11-18T20:42:39.085307+0000","last_clean":"2023-11-18T20:42:39.085307+0000","last_became_active":"2023-11-18T20:42:38.098486+0000","last_became_peered":"2023-11-18T20:42:38.098486+0000","last_unstale":"2023-11-18T20:42:39.085307+0000","last_undegraded":"2023-11-18T20:42:39.085307+0000","last_fullsized":"2023-11-18T20:42:39.085307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084851+0000","last_change":"2023-11-18T20:42:38.092883+0000","last_active":"2023-11-18T20:42:39.084851+0000","last_peered":"2023-11-18T20:42:39.084851+0000","last_clean":"2023-11-18T20:42:39.084851+0000","last_became_active":"2023-11-18T20:42:38.092521+0000","last_became_peered":"2023-11-18T20:42:38.092521+0000","last_unstale":"2023-11-18T20:42:39.084851+0000","last_undegraded":"2023-11-18T20:42:39.084851+0000","last_fullsized":"2023-11-18T20:42:39.084851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:18.854895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084758+0000","last_change":"2023-11-18T20:42:38.090613+0000","last_active":"2023-11-18T20:42:39.084758+0000","last_peered":"2023-11-18T20:42:39.084758+0000","last_clean":"2023-11-18T20:42:39.084758+0000","last_became_active":"2023-11-18T20:42:38.090293+0000","last_became_peered":"2023-11-18T20:42:38.090293+0000","last_unstale":"2023-11-18T20:42:39.084758+0000","last_undegraded":"2023-11-18T20:42:39.084758+0000","last_fullsized":"2023-11-18T20:42:39.084758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086000+0000","last_change":"2023-11-18T20:42:38.098541+0000","last_active":"2023-11-18T20:42:39.086000+0000","last_peered":"2023-11-18T20:42:39.086000+0000","last_clean":"2023-11-18T20:42:39.086000+0000","last_became_active":"2023-11-18T20:42:38.098239+0000","last_became_peered":"2023-11-18T20:42:38.098239+0000","last_unstale":"2023-11-18T20:42:39.086000+0000","last_undegraded":"2023-11-18T20:42:39.086000+0000","last_fullsized":"2023-11-18T20:42:39.086000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086377+0000","last_change":"2023-11-18T20:42:38.107966+0000","last_active":"2023-11-18T20:42:39.086377+0000","last_peered":"2023-11-18T20:42:39.086377+0000","last_clean":"2023-11-18T20:42:39.086377+0000","last_became_active":"2023-11-18T20:42:38.107169+0000","last_became_peered":"2023-11-18T20:42:38.107169+0000","last_unstale":"2023-11-18T20:42:39.086377+0000","last_undegraded":"2023-11-18T20:42:39.086377+0000","last_fullsized":"2023-11-18T20:42:39.086377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:12:16.083479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082349+0000","last_change":"2023-11-18T20:42:38.095583+0000","last_active":"2023-11-18T20:42:39.082349+0000","last_peered":"2023-11-18T20:42:39.082349+0000","last_clean":"2023-11-18T20:42:39.082349+0000","last_became_active":"2023-11-18T20:42:38.095274+0000","last_became_peered":"2023-11-18T20:42:38.095274+0000","last_unstale":"2023-11-18T20:42:39.082349+0000","last_undegraded":"2023-11-18T20:42:39.082349+0000","last_fullsized":"2023-11-18T20:42:39.082349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:28:13.121246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083819+0000","last_change":"2023-11-18T20:42:38.107739+0000","last_active":"2023-11-18T20:42:39.083819+0000","last_peered":"2023-11-18T20:42:39.083819+0000","last_clean":"2023-11-18T20:42:39.083819+0000","last_became_active":"2023-11-18T20:42:38.106917+0000","last_became_peered":"2023-11-18T20:42:38.106917+0000","last_unstale":"2023-11-18T20:42:39.083819+0000","last_undegraded":"2023-11-18T20:42:39.083819+0000","last_fullsized":"2023-11-18T20:42:39.083819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:48:04.295517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086374+0000","last_change":"2023-11-18T20:42:38.103858+0000","last_active":"2023-11-18T20:42:39.086374+0000","last_peered":"2023-11-18T20:42:39.086374+0000","last_clean":"2023-11-18T20:42:39.086374+0000","last_became_active":"2023-11-18T20:42:38.103576+0000","last_became_peered":"2023-11-18T20:42:38.103576+0000","last_unstale":"2023-11-18T20:42:39.086374+0000","last_undegraded":"2023-11-18T20:42:39.086374+0000","last_fullsized":"2023-11-18T20:42:39.086374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085692+0000","last_change":"2023-11-18T20:42:38.100901+0000","last_active":"2023-11-18T20:42:39.085692+0000","last_peered":"2023-11-18T20:42:39.085692+0000","last_clean":"2023-11-18T20:42:39.085692+0000","last_became_active":"2023-11-18T20:42:38.100356+0000","last_became_peered":"2023-11-18T20:42:38.100356+0000","last_unstale":"2023-11-18T20:42:39.085692+0000","last_undegraded":"2023-11-18T20:42:39.085692+0000","last_fullsized":"2023-11-18T20:42:39.085692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:26.508116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086544+0000","last_change":"2023-11-18T20:42:38.102174+0000","last_active":"2023-11-18T20:42:39.086544+0000","last_peered":"2023-11-18T20:42:39.086544+0000","last_clean":"2023-11-18T20:42:39.086544+0000","last_became_active":"2023-11-18T20:42:38.100136+0000","last_became_peered":"2023-11-18T20:42:38.100136+0000","last_unstale":"2023-11-18T20:42:39.086544+0000","last_undegraded":"2023-11-18T20:42:39.086544+0000","last_fullsized":"2023-11-18T20:42:39.086544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:41:14.693957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084520+0000","last_change":"2023-11-18T20:42:38.099808+0000","last_active":"2023-11-18T20:42:39.084520+0000","last_peered":"2023-11-18T20:42:39.084520+0000","last_clean":"2023-11-18T20:42:39.084520+0000","last_became_active":"2023-11-18T20:42:38.099512+0000","last_became_peered":"2023-11-18T20:42:38.099512+0000","last_unstale":"2023-11-18T20:42:39.084520+0000","last_undegraded":"2023-11-18T20:42:39.084520+0000","last_fullsized":"2023-11-18T20:42:39.084520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:19.165207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086125+0000","last_change":"2023-11-18T20:42:38.100064+0000","last_active":"2023-11-18T20:42:39.086125+0000","last_peered":"2023-11-18T20:42:39.086125+0000","last_clean":"2023-11-18T20:42:39.086125+0000","last_became_active":"2023-11-18T20:42:38.099778+0000","last_became_peered":"2023-11-18T20:42:38.099778+0000","last_unstale":"2023-11-18T20:42:39.086125+0000","last_undegraded":"2023-11-18T20:42:39.086125+0000","last_fullsized":"2023-11-18T20:42:39.086125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084743+0000","last_change":"2023-11-18T20:42:38.101526+0000","last_active":"2023-11-18T20:42:39.084743+0000","last_peered":"2023-11-18T20:42:39.084743+0000","last_clean":"2023-11-18T20:42:39.084743+0000","last_became_active":"2023-11-18T20:42:38.099171+0000","last_became_peered":"2023-11-18T20:42:38.099171+0000","last_unstale":"2023-11-18T20:42:39.084743+0000","last_undegraded":"2023-11-18T20:42:39.084743+0000","last_fullsized":"2023-11-18T20:42:39.084743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:26.508116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086361+0000","last_change":"2023-11-18T20:42:38.100225+0000","last_active":"2023-11-18T20:42:39.086361+0000","last_peered":"2023-11-18T20:42:39.086361+0000","last_clean":"2023-11-18T20:42:39.086361+0000","last_became_active":"2023-11-18T20:42:38.099966+0000","last_became_peered":"2023-11-18T20:42:38.099966+0000","last_unstale":"2023-11-18T20:42:39.086361+0000","last_undegraded":"2023-11-18T20:42:39.086361+0000","last_fullsized":"2023-11-18T20:42:39.086361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085054+0000","last_change":"2023-11-18T20:42:38.094313+0000","last_active":"2023-11-18T20:42:39.085054+0000","last_peered":"2023-11-18T20:42:39.085054+0000","last_clean":"2023-11-18T20:42:39.085054+0000","last_became_active":"2023-11-18T20:42:38.094005+0000","last_became_peered":"2023-11-18T20:42:38.094005+0000","last_unstale":"2023-11-18T20:42:39.085054+0000","last_undegraded":"2023-11-18T20:42:39.085054+0000","last_fullsized":"2023-11-18T20:42:39.085054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085575+0000","last_change":"2023-11-18T20:42:38.100511+0000","last_active":"2023-11-18T20:42:39.085575+0000","last_peered":"2023-11-18T20:42:39.085575+0000","last_clean":"2023-11-18T20:42:39.085575+0000","last_became_active":"2023-11-18T20:42:38.100119+0000","last_became_peered":"2023-11-18T20:42:38.100119+0000","last_unstale":"2023-11-18T20:42:39.085575+0000","last_undegraded":"2023-11-18T20:42:39.085575+0000","last_fullsized":"2023-11-18T20:42:39.085575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:12:16.083479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085818+0000","last_change":"2023-11-18T20:42:38.098009+0000","last_active":"2023-11-18T20:42:39.085818+0000","last_peered":"2023-11-18T20:42:39.085818+0000","last_clean":"2023-11-18T20:42:39.085818+0000","last_became_active":"2023-11-18T20:42:38.097728+0000","last_became_peered":"2023-11-18T20:42:38.097728+0000","last_unstale":"2023-11-18T20:42:39.085818+0000","last_undegraded":"2023-11-18T20:42:39.085818+0000","last_fullsized":"2023-11-18T20:42:39.085818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086654+0000","last_change":"2023-11-18T20:42:38.099195+0000","last_active":"2023-11-18T20:42:39.086654+0000","last_peered":"2023-11-18T20:42:39.086654+0000","last_clean":"2023-11-18T20:42:39.086654+0000","last_became_active":"2023-11-18T20:42:38.098918+0000","last_became_peered":"2023-11-18T20:42:38.098918+0000","last_unstale":"2023-11-18T20:42:39.086654+0000","last_undegraded":"2023-11-18T20:42:39.086654+0000","last_fullsized":"2023-11-18T20:42:39.086654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:48:04.295517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082460+0000","last_change":"2023-11-18T20:42:38.095631+0000","last_active":"2023-11-18T20:42:39.082460+0000","last_peered":"2023-11-18T20:42:39.082460+0000","last_clean":"2023-11-18T20:42:39.082460+0000","last_became_active":"2023-11-18T20:42:38.095319+0000","last_became_peered":"2023-11-18T20:42:38.095319+0000","last_unstale":"2023-11-18T20:42:39.082460+0000","last_undegraded":"2023-11-18T20:42:39.082460+0000","last_fullsized":"2023-11-18T20:42:39.082460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:13.814862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085792+0000","last_change":"2023-11-18T20:42:38.103076+0000","last_active":"2023-11-18T20:42:39.085792+0000","last_peered":"2023-11-18T20:42:39.085792+0000","last_clean":"2023-11-18T20:42:39.085792+0000","last_became_active":"2023-11-18T20:42:38.102829+0000","last_became_peered":"2023-11-18T20:42:38.102829+0000","last_unstale":"2023-11-18T20:42:39.085792+0000","last_undegraded":"2023-11-18T20:42:39.085792+0000","last_fullsized":"2023-11-18T20:42:39.085792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:14.531419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083227+0000","last_change":"2023-11-18T20:42:38.089642+0000","last_active":"2023-11-18T20:42:39.083227+0000","last_peered":"2023-11-18T20:42:39.083227+0000","last_clean":"2023-11-18T20:42:39.083227+0000","last_became_active":"2023-11-18T20:42:38.089367+0000","last_became_peered":"2023-11-18T20:42:38.089367+0000","last_unstale":"2023-11-18T20:42:39.083227+0000","last_undegraded":"2023-11-18T20:42:39.083227+0000","last_fullsized":"2023-11-18T20:42:39.083227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:28:13.121246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085545+0000","last_change":"2023-11-18T20:42:37.111681+0000","last_active":"2023-11-18T20:42:39.085545+0000","last_peered":"2023-11-18T20:42:39.085545+0000","last_clean":"2023-11-18T20:42:39.085545+0000","last_became_active":"2023-11-18T20:42:37.111280+0000","last_became_peered":"2023-11-18T20:42:37.111280+0000","last_unstale":"2023-11-18T20:42:39.085545+0000","last_undegraded":"2023-11-18T20:42:39.085545+0000","last_fullsized":"2023-11-18T20:42:39.085545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:31.712397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085932+0000","last_change":"2023-11-18T20:42:38.097798+0000","last_active":"2023-11-18T20:42:39.085932+0000","last_peered":"2023-11-18T20:42:39.085932+0000","last_clean":"2023-11-18T20:42:39.085932+0000","last_became_active":"2023-11-18T20:42:38.096529+0000","last_became_peered":"2023-11-18T20:42:38.096529+0000","last_unstale":"2023-11-18T20:42:39.085932+0000","last_undegraded":"2023-11-18T20:42:39.085932+0000","last_fullsized":"2023-11-18T20:42:39.085932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086235+0000","last_change":"2023-11-18T20:42:37.106910+0000","last_active":"2023-11-18T20:42:39.086235+0000","last_peered":"2023-11-18T20:42:39.086235+0000","last_clean":"2023-11-18T20:42:39.086235+0000","last_became_active":"2023-11-18T20:42:37.106625+0000","last_became_peered":"2023-11-18T20:42:37.106625+0000","last_unstale":"2023-11-18T20:42:39.086235+0000","last_undegraded":"2023-11-18T20:42:39.086235+0000","last_fullsized":"2023-11-18T20:42:39.086235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:43.316226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084601+0000","last_change":"2023-11-18T20:42:37.109273+0000","last_active":"2023-11-18T20:42:39.084601+0000","last_peered":"2023-11-18T20:42:39.084601+0000","last_clean":"2023-11-18T20:42:39.084601+0000","last_became_active":"2023-11-18T20:42:37.108948+0000","last_became_peered":"2023-11-18T20:42:37.108948+0000","last_unstale":"2023-11-18T20:42:39.084601+0000","last_undegraded":"2023-11-18T20:42:39.084601+0000","last_fullsized":"2023-11-18T20:42:39.084601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:12.810253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084564+0000","last_change":"2023-11-18T20:42:38.101286+0000","last_active":"2023-11-18T20:42:39.084564+0000","last_peered":"2023-11-18T20:42:39.084564+0000","last_clean":"2023-11-18T20:42:39.084564+0000","last_became_active":"2023-11-18T20:42:38.100974+0000","last_became_peered":"2023-11-18T20:42:38.100974+0000","last_unstale":"2023-11-18T20:42:39.084564+0000","last_undegraded":"2023-11-18T20:42:39.084564+0000","last_fullsized":"2023-11-18T20:42:39.084564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:14.531419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086311+0000","last_change":"2023-11-18T20:42:37.105690+0000","last_active":"2023-11-18T20:42:39.086311+0000","last_peered":"2023-11-18T20:42:39.086311+0000","last_clean":"2023-11-18T20:42:39.086311+0000","last_became_active":"2023-11-18T20:42:37.105410+0000","last_became_peered":"2023-11-18T20:42:37.105410+0000","last_unstale":"2023-11-18T20:42:39.086311+0000","last_undegraded":"2023-11-18T20:42:39.086311+0000","last_fullsized":"2023-11-18T20:42:39.086311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:14:29.503121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084618+0000","last_change":"2023-11-18T20:42:38.096986+0000","last_active":"2023-11-18T20:42:39.084618+0000","last_peered":"2023-11-18T20:42:39.084618+0000","last_clean":"2023-11-18T20:42:39.084618+0000","last_became_active":"2023-11-18T20:42:38.095964+0000","last_became_peered":"2023-11-18T20:42:38.095964+0000","last_unstale":"2023-11-18T20:42:39.084618+0000","last_undegraded":"2023-11-18T20:42:39.084618+0000","last_fullsized":"2023-11-18T20:42:39.084618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086366+0000","last_change":"2023-11-18T20:42:37.111419+0000","last_active":"2023-11-18T20:42:39.086366+0000","last_peered":"2023-11-18T20:42:39.086366+0000","last_clean":"2023-11-18T20:42:39.086366+0000","last_became_active":"2023-11-18T20:42:37.110877+0000","last_became_peered":"2023-11-18T20:42:37.110877+0000","last_unstale":"2023-11-18T20:42:39.086366+0000","last_undegraded":"2023-11-18T20:42:39.086366+0000","last_fullsized":"2023-11-18T20:42:39.086366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086126+0000","last_change":"2023-11-18T20:42:38.108594+0000","last_active":"2023-11-18T20:42:39.086126+0000","last_peered":"2023-11-18T20:42:39.086126+0000","last_clean":"2023-11-18T20:42:39.086126+0000","last_became_active":"2023-11-18T20:42:38.108325+0000","last_became_peered":"2023-11-18T20:42:38.108325+0000","last_unstale":"2023-11-18T20:42:39.086126+0000","last_undegraded":"2023-11-18T20:42:39.086126+0000","last_fullsized":"2023-11-18T20:42:39.086126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086218+0000","last_change":"2023-11-18T20:42:37.106865+0000","last_active":"2023-11-18T20:42:39.086218+0000","last_peered":"2023-11-18T20:42:39.086218+0000","last_clean":"2023-11-18T20:42:39.086218+0000","last_became_active":"2023-11-18T20:42:37.106561+0000","last_became_peered":"2023-11-18T20:42:37.106561+0000","last_unstale":"2023-11-18T20:42:39.086218+0000","last_undegraded":"2023-11-18T20:42:39.086218+0000","last_fullsized":"2023-11-18T20:42:39.086218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:25.503507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085723+0000","last_change":"2023-11-18T20:42:38.101545+0000","last_active":"2023-11-18T20:42:39.085723+0000","last_peered":"2023-11-18T20:42:39.085723+0000","last_clean":"2023-11-18T20:42:39.085723+0000","last_became_active":"2023-11-18T20:42:38.101247+0000","last_became_peered":"2023-11-18T20:42:38.101247+0000","last_unstale":"2023-11-18T20:42:39.085723+0000","last_undegraded":"2023-11-18T20:42:39.085723+0000","last_fullsized":"2023-11-18T20:42:39.085723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085146+0000","last_change":"2023-11-18T20:42:37.109345+0000","last_active":"2023-11-18T20:42:39.085146+0000","last_peered":"2023-11-18T20:42:39.085146+0000","last_clean":"2023-11-18T20:42:39.085146+0000","last_became_active":"2023-11-18T20:42:37.109081+0000","last_became_peered":"2023-11-18T20:42:37.109081+0000","last_unstale":"2023-11-18T20:42:39.085146+0000","last_undegraded":"2023-11-18T20:42:39.085146+0000","last_fullsized":"2023-11-18T20:42:39.085146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083745+0000","last_change":"2023-11-18T20:42:38.100125+0000","last_active":"2023-11-18T20:42:39.083745+0000","last_peered":"2023-11-18T20:42:39.083745+0000","last_clean":"2023-11-18T20:42:39.083745+0000","last_became_active":"2023-11-18T20:42:38.099646+0000","last_became_peered":"2023-11-18T20:42:38.099646+0000","last_unstale":"2023-11-18T20:42:39.083745+0000","last_undegraded":"2023-11-18T20:42:39.083745+0000","last_fullsized":"2023-11-18T20:42:39.083745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:14:30.507730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086459+0000","last_change":"2023-11-18T20:42:37.106986+0000","last_active":"2023-11-18T20:42:39.086459+0000","last_peered":"2023-11-18T20:42:39.086459+0000","last_clean":"2023-11-18T20:42:39.086459+0000","last_became_active":"2023-11-18T20:42:37.106707+0000","last_became_peered":"2023-11-18T20:42:37.106707+0000","last_unstale":"2023-11-18T20:42:39.086459+0000","last_undegraded":"2023-11-18T20:42:39.086459+0000","last_fullsized":"2023-11-18T20:42:39.086459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:31.712397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083280+0000","last_change":"2023-11-18T20:42:38.107769+0000","last_active":"2023-11-18T20:42:39.083280+0000","last_peered":"2023-11-18T20:42:39.083280+0000","last_clean":"2023-11-18T20:42:39.083280+0000","last_became_active":"2023-11-18T20:42:38.104253+0000","last_became_peered":"2023-11-18T20:42:38.104253+0000","last_unstale":"2023-11-18T20:42:39.083280+0000","last_undegraded":"2023-11-18T20:42:39.083280+0000","last_fullsized":"2023-11-18T20:42:39.083280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:45:28.196671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085912+0000","last_change":"2023-11-18T20:42:37.098519+0000","last_active":"2023-11-18T20:42:39.085912+0000","last_peered":"2023-11-18T20:42:39.085912+0000","last_clean":"2023-11-18T20:42:39.085912+0000","last_became_active":"2023-11-18T20:42:37.097972+0000","last_became_peered":"2023-11-18T20:42:37.097972+0000","last_unstale":"2023-11-18T20:42:39.085912+0000","last_undegraded":"2023-11-18T20:42:39.085912+0000","last_fullsized":"2023-11-18T20:42:39.085912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:12.810253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084892+0000","last_change":"2023-11-18T20:42:38.100733+0000","last_active":"2023-11-18T20:42:39.084892+0000","last_peered":"2023-11-18T20:42:39.084892+0000","last_clean":"2023-11-18T20:42:39.084892+0000","last_became_active":"2023-11-18T20:42:38.100390+0000","last_became_peered":"2023-11-18T20:42:38.100390+0000","last_unstale":"2023-11-18T20:42:39.084892+0000","last_undegraded":"2023-11-18T20:42:39.084892+0000","last_fullsized":"2023-11-18T20:42:39.084892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:32.717006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086635+0000","last_change":"2023-11-18T20:42:37.110943+0000","last_active":"2023-11-18T20:42:39.086635+0000","last_peered":"2023-11-18T20:42:39.086635+0000","last_clean":"2023-11-18T20:42:39.086635+0000","last_became_active":"2023-11-18T20:42:37.110657+0000","last_became_peered":"2023-11-18T20:42:37.110657+0000","last_unstale":"2023-11-18T20:42:39.086635+0000","last_undegraded":"2023-11-18T20:42:39.086635+0000","last_fullsized":"2023-11-18T20:42:39.086635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084234+0000","last_change":"2023-11-18T20:42:38.102821+0000","last_active":"2023-11-18T20:42:39.084234+0000","last_peered":"2023-11-18T20:42:39.084234+0000","last_clean":"2023-11-18T20:42:39.084234+0000","last_became_active":"2023-11-18T20:42:38.102491+0000","last_became_peered":"2023-11-18T20:42:38.102491+0000","last_unstale":"2023-11-18T20:42:39.084234+0000","last_undegraded":"2023-11-18T20:42:39.084234+0000","last_fullsized":"2023-11-18T20:42:39.084234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:17:48.228298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085793+0000","last_change":"2023-11-18T20:42:37.107187+0000","last_active":"2023-11-18T20:42:39.085793+0000","last_peered":"2023-11-18T20:42:39.085793+0000","last_clean":"2023-11-18T20:42:39.085793+0000","last_became_active":"2023-11-18T20:42:37.106904+0000","last_became_peered":"2023-11-18T20:42:37.106904+0000","last_unstale":"2023-11-18T20:42:39.085793+0000","last_undegraded":"2023-11-18T20:42:39.085793+0000","last_fullsized":"2023-11-18T20:42:39.085793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:17.850286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085803+0000","last_change":"2023-11-18T20:42:38.102486+0000","last_active":"2023-11-18T20:42:39.085803+0000","last_peered":"2023-11-18T20:42:39.085803+0000","last_clean":"2023-11-18T20:42:39.085803+0000","last_became_active":"2023-11-18T20:42:38.102239+0000","last_became_peered":"2023-11-18T20:42:38.102239+0000","last_unstale":"2023-11-18T20:42:39.085803+0000","last_undegraded":"2023-11-18T20:42:39.085803+0000","last_fullsized":"2023-11-18T20:42:39.085803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:26.508116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082180+0000","last_change":"2023-11-18T20:42:37.105948+0000","last_active":"2023-11-18T20:42:39.082180+0000","last_peered":"2023-11-18T20:42:39.082180+0000","last_clean":"2023-11-18T20:42:39.082180+0000","last_became_active":"2023-11-18T20:42:37.105280+0000","last_became_peered":"2023-11-18T20:42:37.105280+0000","last_unstale":"2023-11-18T20:42:39.082180+0000","last_undegraded":"2023-11-18T20:42:39.082180+0000","last_fullsized":"2023-11-18T20:42:39.082180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:55:27.947068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083496+0000","last_change":"2023-11-18T20:42:38.107573+0000","last_active":"2023-11-18T20:42:39.083496+0000","last_peered":"2023-11-18T20:42:39.083496+0000","last_clean":"2023-11-18T20:42:39.083496+0000","last_became_active":"2023-11-18T20:42:38.103361+0000","last_became_peered":"2023-11-18T20:42:38.103361+0000","last_unstale":"2023-11-18T20:42:39.083496+0000","last_undegraded":"2023-11-18T20:42:39.083496+0000","last_fullsized":"2023-11-18T20:42:39.083496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:04:55.965852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083454+0000","last_change":"2023-11-18T20:42:37.097968+0000","last_active":"2023-11-18T20:42:39.083454+0000","last_peered":"2023-11-18T20:42:39.083454+0000","last_clean":"2023-11-18T20:42:39.083454+0000","last_became_active":"2023-11-18T20:42:37.097659+0000","last_became_peered":"2023-11-18T20:42:37.097659+0000","last_unstale":"2023-11-18T20:42:39.083454+0000","last_undegraded":"2023-11-18T20:42:39.083454+0000","last_fullsized":"2023-11-18T20:42:39.083454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:08:51.955879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083672+0000","last_change":"2023-11-18T20:42:38.107457+0000","last_active":"2023-11-18T20:42:39.083672+0000","last_peered":"2023-11-18T20:42:39.083672+0000","last_clean":"2023-11-18T20:42:39.083672+0000","last_became_active":"2023-11-18T20:42:38.103260+0000","last_became_peered":"2023-11-18T20:42:38.103260+0000","last_unstale":"2023-11-18T20:42:39.083672+0000","last_undegraded":"2023-11-18T20:42:39.083672+0000","last_fullsized":"2023-11-18T20:42:39.083672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:41:14.693957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082379+0000","last_change":"2023-11-18T20:42:37.109692+0000","last_active":"2023-11-18T20:42:39.082379+0000","last_peered":"2023-11-18T20:42:39.082379+0000","last_clean":"2023-11-18T20:42:39.082379+0000","last_became_active":"2023-11-18T20:42:37.109328+0000","last_became_peered":"2023-11-18T20:42:37.109328+0000","last_unstale":"2023-11-18T20:42:39.082379+0000","last_undegraded":"2023-11-18T20:42:39.082379+0000","last_fullsized":"2023-11-18T20:42:39.082379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:08:51.955879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084464+0000","last_change":"2023-11-18T20:42:38.107442+0000","last_active":"2023-11-18T20:42:39.084464+0000","last_peered":"2023-11-18T20:42:39.084464+0000","last_clean":"2023-11-18T20:42:39.084464+0000","last_became_active":"2023-11-18T20:42:38.106939+0000","last_became_peered":"2023-11-18T20:42:38.106939+0000","last_unstale":"2023-11-18T20:42:39.084464+0000","last_undegraded":"2023-11-18T20:42:39.084464+0000","last_fullsized":"2023-11-18T20:42:39.084464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:52:14.406378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085180+0000","last_change":"2023-11-18T20:42:37.107734+0000","last_active":"2023-11-18T20:42:39.085180+0000","last_peered":"2023-11-18T20:42:39.085180+0000","last_clean":"2023-11-18T20:42:39.085180+0000","last_became_active":"2023-11-18T20:42:37.107466+0000","last_became_peered":"2023-11-18T20:42:37.107466+0000","last_unstale":"2023-11-18T20:42:39.085180+0000","last_undegraded":"2023-11-18T20:42:39.085180+0000","last_fullsized":"2023-11-18T20:42:39.085180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:13.526810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086537+0000","last_change":"2023-11-18T20:42:38.105663+0000","last_active":"2023-11-18T20:42:39.086537+0000","last_peered":"2023-11-18T20:42:39.086537+0000","last_clean":"2023-11-18T20:42:39.086537+0000","last_became_active":"2023-11-18T20:42:38.105269+0000","last_became_peered":"2023-11-18T20:42:38.105269+0000","last_unstale":"2023-11-18T20:42:39.086537+0000","last_undegraded":"2023-11-18T20:42:39.086537+0000","last_fullsized":"2023-11-18T20:42:39.086537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:04:55.965852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082362+0000","last_change":"2023-11-18T20:42:37.109021+0000","last_active":"2023-11-18T20:42:39.082362+0000","last_peered":"2023-11-18T20:42:39.082362+0000","last_clean":"2023-11-18T20:42:39.082362+0000","last_became_active":"2023-11-18T20:42:37.108256+0000","last_became_peered":"2023-11-18T20:42:37.108256+0000","last_unstale":"2023-11-18T20:42:39.082362+0000","last_undegraded":"2023-11-18T20:42:39.082362+0000","last_fullsized":"2023-11-18T20:42:39.082362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:17:02.258981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083639+0000","last_change":"2023-11-18T20:42:38.107956+0000","last_active":"2023-11-18T20:42:39.083639+0000","last_peered":"2023-11-18T20:42:39.083639+0000","last_clean":"2023-11-18T20:42:39.083639+0000","last_became_active":"2023-11-18T20:42:38.107488+0000","last_became_peered":"2023-11-18T20:42:38.107488+0000","last_unstale":"2023-11-18T20:42:39.083639+0000","last_undegraded":"2023-11-18T20:42:39.083639+0000","last_fullsized":"2023-11-18T20:42:39.083639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:51:04.970201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-18T20:42:38.097227+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:42:38.097227+0000","last_peered":"2023-11-18T20:42:38.097227+0000","last_clean":"2023-11-18T20:42:38.097227+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:42:38.097227+0000","last_undegraded":"2023-11-18T20:42:38.097227+0000","last_fullsized":"2023-11-18T20:42:38.097227+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086001+0000","last_change":"2023-11-18T20:42:37.110512+0000","last_active":"2023-11-18T20:42:39.086001+0000","last_peered":"2023-11-18T20:42:39.086001+0000","last_clean":"2023-11-18T20:42:39.086001+0000","last_became_active":"2023-11-18T20:42:37.110213+0000","last_became_peered":"2023-11-18T20:42:37.110213+0000","last_unstale":"2023-11-18T20:42:39.086001+0000","last_undegraded":"2023-11-18T20:42:39.086001+0000","last_fullsized":"2023-11-18T20:42:39.086001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:43.316226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084751+0000","last_change":"2023-11-18T20:42:38.105196+0000","last_active":"2023-11-18T20:42:39.084751+0000","last_peered":"2023-11-18T20:42:39.084751+0000","last_clean":"2023-11-18T20:42:39.084751+0000","last_became_active":"2023-11-18T20:42:38.104886+0000","last_became_peered":"2023-11-18T20:42:38.104886+0000","last_unstale":"2023-11-18T20:42:39.084751+0000","last_undegraded":"2023-11-18T20:42:39.084751+0000","last_fullsized":"2023-11-18T20:42:39.084751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084031+0000","last_change":"2023-11-18T20:42:37.108345+0000","last_active":"2023-11-18T20:42:39.084031+0000","last_peered":"2023-11-18T20:42:39.084031+0000","last_clean":"2023-11-18T20:42:39.084031+0000","last_became_active":"2023-11-18T20:42:37.107995+0000","last_became_peered":"2023-11-18T20:42:37.107995+0000","last_unstale":"2023-11-18T20:42:39.084031+0000","last_undegraded":"2023-11-18T20:42:39.084031+0000","last_fullsized":"2023-11-18T20:42:39.084031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:15.427234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084749+0000","last_change":"2023-11-18T20:42:38.088232+0000","last_active":"2023-11-18T20:42:39.084749+0000","last_peered":"2023-11-18T20:42:39.084749+0000","last_clean":"2023-11-18T20:42:39.084749+0000","last_became_active":"2023-11-18T20:42:38.087631+0000","last_became_peered":"2023-11-18T20:42:38.087631+0000","last_unstale":"2023-11-18T20:42:39.084749+0000","last_undegraded":"2023-11-18T20:42:39.084749+0000","last_fullsized":"2023-11-18T20:42:39.084749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:44.320835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082457+0000","last_change":"2023-11-18T20:42:37.107910+0000","last_active":"2023-11-18T20:42:39.082457+0000","last_peered":"2023-11-18T20:42:39.082457+0000","last_clean":"2023-11-18T20:42:39.082457+0000","last_became_active":"2023-11-18T20:42:37.106792+0000","last_became_peered":"2023-11-18T20:42:37.106792+0000","last_unstale":"2023-11-18T20:42:39.082457+0000","last_undegraded":"2023-11-18T20:42:39.082457+0000","last_fullsized":"2023-11-18T20:42:39.082457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:03:11.287757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084900+0000","last_change":"2023-11-18T20:42:38.090117+0000","last_active":"2023-11-18T20:42:39.084900+0000","last_peered":"2023-11-18T20:42:39.084900+0000","last_clean":"2023-11-18T20:42:39.084900+0000","last_became_active":"2023-11-18T20:42:38.089821+0000","last_became_peered":"2023-11-18T20:42:38.089821+0000","last_unstale":"2023-11-18T20:42:39.084900+0000","last_undegraded":"2023-11-18T20:42:39.084900+0000","last_fullsized":"2023-11-18T20:42:39.084900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086690+0000","last_change":"2023-11-18T20:42:37.111162+0000","last_active":"2023-11-18T20:42:39.086690+0000","last_peered":"2023-11-18T20:42:39.086690+0000","last_clean":"2023-11-18T20:42:39.086690+0000","last_became_active":"2023-11-18T20:42:37.110692+0000","last_became_peered":"2023-11-18T20:42:37.110692+0000","last_unstale":"2023-11-18T20:42:39.086690+0000","last_undegraded":"2023-11-18T20:42:39.086690+0000","last_fullsized":"2023-11-18T20:42:39.086690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084091+0000","last_change":"2023-11-18T20:42:38.091366+0000","last_active":"2023-11-18T20:42:39.084091+0000","last_peered":"2023-11-18T20:42:39.084091+0000","last_clean":"2023-11-18T20:42:39.084091+0000","last_became_active":"2023-11-18T20:42:38.089726+0000","last_became_peered":"2023-11-18T20:42:38.089726+0000","last_unstale":"2023-11-18T20:42:39.084091+0000","last_undegraded":"2023-11-18T20:42:39.084091+0000","last_fullsized":"2023-11-18T20:42:39.084091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:44.320835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085074+0000","last_change":"2023-11-18T20:42:37.103840+0000","last_active":"2023-11-18T20:42:39.085074+0000","last_peered":"2023-11-18T20:42:39.085074+0000","last_clean":"2023-11-18T20:42:39.085074+0000","last_became_active":"2023-11-18T20:42:37.103516+0000","last_became_peered":"2023-11-18T20:42:37.103516+0000","last_unstale":"2023-11-18T20:42:39.085074+0000","last_undegraded":"2023-11-18T20:42:39.085074+0000","last_fullsized":"2023-11-18T20:42:39.085074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086520+0000","last_change":"2023-11-18T20:42:38.100140+0000","last_active":"2023-11-18T20:42:39.086520+0000","last_peered":"2023-11-18T20:42:39.086520+0000","last_clean":"2023-11-18T20:42:39.086520+0000","last_became_active":"2023-11-18T20:42:38.099880+0000","last_became_peered":"2023-11-18T20:42:38.099880+0000","last_unstale":"2023-11-18T20:42:39.086520+0000","last_undegraded":"2023-11-18T20:42:39.086520+0000","last_fullsized":"2023-11-18T20:42:39.086520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:28.378102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083840+0000","last_change":"2023-11-18T20:42:37.103907+0000","last_active":"2023-11-18T20:42:39.083840+0000","last_peered":"2023-11-18T20:42:39.083840+0000","last_clean":"2023-11-18T20:42:39.083840+0000","last_became_active":"2023-11-18T20:42:37.103642+0000","last_became_peered":"2023-11-18T20:42:37.103642+0000","last_unstale":"2023-11-18T20:42:39.083840+0000","last_undegraded":"2023-11-18T20:42:39.083840+0000","last_fullsized":"2023-11-18T20:42:39.083840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:25.503507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084990+0000","last_change":"2023-11-18T20:42:38.090609+0000","last_active":"2023-11-18T20:42:39.084990+0000","last_peered":"2023-11-18T20:42:39.084990+0000","last_clean":"2023-11-18T20:42:39.084990+0000","last_became_active":"2023-11-18T20:42:38.090275+0000","last_became_peered":"2023-11-18T20:42:38.090275+0000","last_unstale":"2023-11-18T20:42:39.084990+0000","last_undegraded":"2023-11-18T20:42:39.084990+0000","last_fullsized":"2023-11-18T20:42:39.084990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.450170+0000","last_change":"2023-11-18T20:42:37.110209+0000","last_active":"2023-11-18T20:42:39.450170+0000","last_peered":"2023-11-18T20:42:39.450170+0000","last_clean":"2023-11-18T20:42:39.450170+0000","last_became_active":"2023-11-18T20:42:37.109906+0000","last_became_peered":"2023-11-18T20:42:37.109906+0000","last_unstale":"2023-11-18T20:42:39.450170+0000","last_undegraded":"2023-11-18T20:42:39.450170+0000","last_fullsized":"2023-11-18T20:42:39.450170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:17.850286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084866+0000","last_change":"2023-11-18T20:42:38.104224+0000","last_active":"2023-11-18T20:42:39.084866+0000","last_peered":"2023-11-18T20:42:39.084866+0000","last_clean":"2023-11-18T20:42:39.084866+0000","last_became_active":"2023-11-18T20:42:38.103950+0000","last_became_peered":"2023-11-18T20:42:38.103950+0000","last_unstale":"2023-11-18T20:42:39.084866+0000","last_undegraded":"2023-11-18T20:42:39.084866+0000","last_fullsized":"2023-11-18T20:42:39.084866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086459+0000","last_change":"2023-11-18T20:42:37.102910+0000","last_active":"2023-11-18T20:42:39.086459+0000","last_peered":"2023-11-18T20:42:39.086459+0000","last_clean":"2023-11-18T20:42:39.086459+0000","last_became_active":"2023-11-18T20:42:37.102612+0000","last_became_peered":"2023-11-18T20:42:37.102612+0000","last_unstale":"2023-11-18T20:42:39.086459+0000","last_undegraded":"2023-11-18T20:42:39.086459+0000","last_fullsized":"2023-11-18T20:42:39.086459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085442+0000","last_change":"2023-11-18T20:42:38.094021+0000","last_active":"2023-11-18T20:42:39.085442+0000","last_peered":"2023-11-18T20:42:39.085442+0000","last_clean":"2023-11-18T20:42:39.085442+0000","last_became_active":"2023-11-18T20:42:38.093125+0000","last_became_peered":"2023-11-18T20:42:38.093125+0000","last_unstale":"2023-11-18T20:42:39.085442+0000","last_undegraded":"2023-11-18T20:42:39.085442+0000","last_fullsized":"2023-11-18T20:42:39.085442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:19.165207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083934+0000","last_change":"2023-11-18T20:42:37.102964+0000","last_active":"2023-11-18T20:42:39.083934+0000","last_peered":"2023-11-18T20:42:39.083934+0000","last_clean":"2023-11-18T20:42:39.083934+0000","last_became_active":"2023-11-18T20:42:37.102696+0000","last_became_peered":"2023-11-18T20:42:37.102696+0000","last_unstale":"2023-11-18T20:42:39.083934+0000","last_undegraded":"2023-11-18T20:42:39.083934+0000","last_fullsized":"2023-11-18T20:42:39.083934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:51.367829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084007+0000","last_change":"2023-11-18T20:42:38.089921+0000","last_active":"2023-11-18T20:42:39.084007+0000","last_peered":"2023-11-18T20:42:39.084007+0000","last_clean":"2023-11-18T20:42:39.084007+0000","last_became_active":"2023-11-18T20:42:38.089451+0000","last_became_peered":"2023-11-18T20:42:38.089451+0000","last_unstale":"2023-11-18T20:42:39.084007+0000","last_undegraded":"2023-11-18T20:42:39.084007+0000","last_fullsized":"2023-11-18T20:42:39.084007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:03:26.636590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084840+0000","last_change":"2023-11-18T20:42:37.110785+0000","last_active":"2023-11-18T20:42:39.084840+0000","last_peered":"2023-11-18T20:42:39.084840+0000","last_clean":"2023-11-18T20:42:39.084840+0000","last_became_active":"2023-11-18T20:42:37.110523+0000","last_became_peered":"2023-11-18T20:42:37.110523+0000","last_unstale":"2023-11-18T20:42:39.084840+0000","last_undegraded":"2023-11-18T20:42:39.084840+0000","last_fullsized":"2023-11-18T20:42:39.084840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:43.316226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085653+0000","last_change":"2023-11-18T20:42:38.097248+0000","last_active":"2023-11-18T20:42:39.085653+0000","last_peered":"2023-11-18T20:42:39.085653+0000","last_clean":"2023-11-18T20:42:39.085653+0000","last_became_active":"2023-11-18T20:42:38.096931+0000","last_became_peered":"2023-11-18T20:42:38.096931+0000","last_unstale":"2023-11-18T20:42:39.085653+0000","last_undegraded":"2023-11-18T20:42:39.085653+0000","last_fullsized":"2023-11-18T20:42:39.085653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:35:56.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085556+0000","last_change":"2023-11-18T20:42:37.106332+0000","last_active":"2023-11-18T20:42:39.085556+0000","last_peered":"2023-11-18T20:42:39.085556+0000","last_clean":"2023-11-18T20:42:39.085556+0000","last_became_active":"2023-11-18T20:42:37.106048+0000","last_became_peered":"2023-11-18T20:42:37.106048+0000","last_unstale":"2023-11-18T20:42:39.085556+0000","last_undegraded":"2023-11-18T20:42:39.085556+0000","last_fullsized":"2023-11-18T20:42:39.085556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:31.712397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085454+0000","last_change":"2023-11-18T20:42:38.100884+0000","last_active":"2023-11-18T20:42:39.085454+0000","last_peered":"2023-11-18T20:42:39.085454+0000","last_clean":"2023-11-18T20:42:39.085454+0000","last_became_active":"2023-11-18T20:42:38.100591+0000","last_became_peered":"2023-11-18T20:42:38.100591+0000","last_unstale":"2023-11-18T20:42:39.085454+0000","last_undegraded":"2023-11-18T20:42:39.085454+0000","last_fullsized":"2023-11-18T20:42:39.085454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084262+0000","last_change":"2023-11-18T20:42:37.099362+0000","last_active":"2023-11-18T20:42:39.084262+0000","last_peered":"2023-11-18T20:42:39.084262+0000","last_clean":"2023-11-18T20:42:39.084262+0000","last_became_active":"2023-11-18T20:42:37.099107+0000","last_became_peered":"2023-11-18T20:42:37.099107+0000","last_unstale":"2023-11-18T20:42:39.084262+0000","last_undegraded":"2023-11-18T20:42:39.084262+0000","last_fullsized":"2023-11-18T20:42:39.084262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086177+0000","last_change":"2023-11-18T20:42:38.101488+0000","last_active":"2023-11-18T20:42:39.086177+0000","last_peered":"2023-11-18T20:42:39.086177+0000","last_clean":"2023-11-18T20:42:39.086177+0000","last_became_active":"2023-11-18T20:42:38.101205+0000","last_became_peered":"2023-11-18T20:42:38.101205+0000","last_unstale":"2023-11-18T20:42:39.086177+0000","last_undegraded":"2023-11-18T20:42:39.086177+0000","last_fullsized":"2023-11-18T20:42:39.086177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086311+0000","last_change":"2023-11-18T20:42:38.106746+0000","last_active":"2023-11-18T20:42:39.086311+0000","last_peered":"2023-11-18T20:42:39.086311+0000","last_clean":"2023-11-18T20:42:39.086311+0000","last_became_active":"2023-11-18T20:42:38.101774+0000","last_became_peered":"2023-11-18T20:42:38.101774+0000","last_unstale":"2023-11-18T20:42:39.086311+0000","last_undegraded":"2023-11-18T20:42:39.086311+0000","last_fullsized":"2023-11-18T20:42:39.086311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083126+0000","last_change":"2023-11-18T20:42:37.110955+0000","last_active":"2023-11-18T20:42:39.083126+0000","last_peered":"2023-11-18T20:42:39.083126+0000","last_clean":"2023-11-18T20:42:39.083126+0000","last_became_active":"2023-11-18T20:42:37.110681+0000","last_became_peered":"2023-11-18T20:42:37.110681+0000","last_unstale":"2023-11-18T20:42:39.083126+0000","last_undegraded":"2023-11-18T20:42:39.083126+0000","last_fullsized":"2023-11-18T20:42:39.083126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:28:12.116637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086444+0000","last_change":"2023-11-18T20:42:38.105146+0000","last_active":"2023-11-18T20:42:39.086444+0000","last_peered":"2023-11-18T20:42:39.086444+0000","last_clean":"2023-11-18T20:42:39.086444+0000","last_became_active":"2023-11-18T20:42:38.101793+0000","last_became_peered":"2023-11-18T20:42:38.101793+0000","last_unstale":"2023-11-18T20:42:39.086444+0000","last_undegraded":"2023-11-18T20:42:39.086444+0000","last_fullsized":"2023-11-18T20:42:39.086444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:45:28.196671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083739+0000","last_change":"2023-11-18T20:42:37.102836+0000","last_active":"2023-11-18T20:42:39.083739+0000","last_peered":"2023-11-18T20:42:39.083739+0000","last_clean":"2023-11-18T20:42:39.083739+0000","last_became_active":"2023-11-18T20:42:37.102550+0000","last_became_peered":"2023-11-18T20:42:37.102550+0000","last_unstale":"2023-11-18T20:42:39.083739+0000","last_undegraded":"2023-11-18T20:42:39.083739+0000","last_fullsized":"2023-11-18T20:42:39.083739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085598+0000","last_change":"2023-11-18T20:42:38.102190+0000","last_active":"2023-11-18T20:42:39.085598+0000","last_peered":"2023-11-18T20:42:39.085598+0000","last_clean":"2023-11-18T20:42:39.085598+0000","last_became_active":"2023-11-18T20:42:38.101883+0000","last_became_peered":"2023-11-18T20:42:38.101883+0000","last_unstale":"2023-11-18T20:42:39.085598+0000","last_undegraded":"2023-11-18T20:42:39.085598+0000","last_fullsized":"2023-11-18T20:42:39.085598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:14.531419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085117+0000","last_change":"2023-11-18T20:42:37.108374+0000","last_active":"2023-11-18T20:42:39.085117+0000","last_peered":"2023-11-18T20:42:39.085117+0000","last_clean":"2023-11-18T20:42:39.085117+0000","last_became_active":"2023-11-18T20:42:37.108079+0000","last_became_peered":"2023-11-18T20:42:37.108079+0000","last_unstale":"2023-11-18T20:42:39.085117+0000","last_undegraded":"2023-11-18T20:42:39.085117+0000","last_fullsized":"2023-11-18T20:42:39.085117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:03:25.631982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085115+0000","last_change":"2023-11-18T20:42:38.095628+0000","last_active":"2023-11-18T20:42:39.085115+0000","last_peered":"2023-11-18T20:42:39.085115+0000","last_clean":"2023-11-18T20:42:39.085115+0000","last_became_active":"2023-11-18T20:42:38.095315+0000","last_became_peered":"2023-11-18T20:42:38.095315+0000","last_unstale":"2023-11-18T20:42:39.085115+0000","last_undegraded":"2023-11-18T20:42:39.085115+0000","last_fullsized":"2023-11-18T20:42:39.085115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085811+0000","last_change":"2023-11-18T20:42:37.101566+0000","last_active":"2023-11-18T20:42:39.085811+0000","last_peered":"2023-11-18T20:42:39.085811+0000","last_clean":"2023-11-18T20:42:39.085811+0000","last_became_active":"2023-11-18T20:42:37.101286+0000","last_became_peered":"2023-11-18T20:42:37.101286+0000","last_unstale":"2023-11-18T20:42:39.085811+0000","last_undegraded":"2023-11-18T20:42:39.085811+0000","last_fullsized":"2023-11-18T20:42:39.085811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:14:29.503121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":819200,"data_stored":665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:42:45.557 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:42:45.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.698+0000 7fcdc117c640 1 Processor -- start 2023-11-18T20:42:45.700 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.699+0000 7fcdc117c640 1 -- start start 2023-11-18T20:42:45.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.699+0000 7fcdc117c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdbc15c2f0 0x7fcdbc06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.699+0000 7fcdc117c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdbc06f040 0x7fcdbc140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.699+0000 7fcdc117c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdbc15ba40 con 0x7fcdbc140d40 2023-11-18T20:42:45.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.699+0000 7fcdc117c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdbc15c6d0 con 0x7fcdbc06f040 2023-11-18T20:42:45.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.699+0000 7fcdc117c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc13e530 con 0x7fcdbc15c2f0 2023-11-18T20:42:45.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.699+0000 7fcdba59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdbc06f040 0x7fcdbc140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.700+0000 7fcdba59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcdbc06f040 0x7fcdbc140800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41432/0 (socket says 172.21.15.60:41432) 2023-11-18T20:42:45.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.700+0000 7fcdba59c640 1 -- 172.21.15.60:0/4143044980 learned_addr learned my addr 172.21.15.60:0/4143044980 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:45.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.700+0000 7fcdbad9d640 1 --2- 172.21.15.60:0/4143044980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdbc15c2f0 0x7fcdbc06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.700+0000 7fcdba59c640 1 -- 172.21.15.60:0/4143044980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdbc15c2f0 msgr2=0x7fcdbc06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:45.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.700+0000 7fcdba59c640 1 --2- 172.21.15.60:0/4143044980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdbc15c2f0 0x7fcdbc06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.703 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.700+0000 7fcdba59c640 1 -- 172.21.15.60:0/4143044980 >> v1:172.21.15.100:6789/0 conn(0x7fcdbc140d40 legacy=0x7fcdbc15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.700+0000 7fcdba59c640 1 -- 172.21.15.60:0/4143044980 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc1411d0 con 0x7fcdbc06f040 2023-11-18T20:42:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdba59c640 1 --2- 172.21.15.60:0/4143044980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc06f040 0x7fcdbc140800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcda40098d0 tx=0x7fcda402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed45207cf5c9f3f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdb9d9b640 1 -- 172.21.15.60:0/4143044980 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4030610 con 0x7fcdbc06f040 2023-11-18T20:42:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdb9d9b640 1 -- 172.21.15.60:0/4143044980 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda4030770 con 0x7fcdbc06f040 2023-11-18T20:42:45.704 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdb9d9b640 1 -- 172.21.15.60:0/4143044980 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4030b00 con 0x7fcdbc06f040 2023-11-18T20:42:45.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdc117c640 1 -- 172.21.15.60:0/4143044980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc06f040 msgr2=0x7fcdbc140800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdc117c640 1 --2- 172.21.15.60:0/4143044980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc06f040 0x7fcdbc140800 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcda40098d0 tx=0x7fcda402b960 comp rx=0 tx=0).stop 2023-11-18T20:42:45.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdc117c640 1 -- 172.21.15.60:0/4143044980 shutdown_connections 2023-11-18T20:42:45.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdc117c640 1 --2- 172.21.15.60:0/4143044980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc06f040 0x7fcdbc140800 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdc117c640 1 --2- 172.21.15.60:0/4143044980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdbc15c2f0 0x7fcdbc06eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdc117c640 1 -- 172.21.15.60:0/4143044980 >> 172.21.15.60:0/4143044980 conn(0x7fcdbc06a6b0 msgr2=0x7fcdbc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdc117c640 1 -- 172.21.15.60:0/4143044980 shutdown_connections 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.701+0000 7fcdc117c640 1 -- 172.21.15.60:0/4143044980 wait complete. 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.703+0000 7fcdc117c640 1 Processor -- start 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.703+0000 7fcdc117c640 1 -- start start 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.703+0000 7fcdc117c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdbc06f040 0x7fcdbc1508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.703+0000 7fcdc117c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc140d40 0x7fcdbc153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.703+0000 7fcdc117c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdbc15c2f0 0x7fcdbc0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.703+0000 7fcdc117c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc150df0 con 0x7fcdbc140d40 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.703+0000 7fcdc117c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc154170 con 0x7fcdbc06f040 2023-11-18T20:42:45.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.703+0000 7fcdc117c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc154450 con 0x7fcdbc15c2f0 2023-11-18T20:42:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdba59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc140d40 0x7fcdbc153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdba59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc140d40 0x7fcdbc153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41446/0 (socket says 172.21.15.60:41446) 2023-11-18T20:42:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdba59c640 1 -- 172.21.15.60:0/1438291023 learned_addr learned my addr 172.21.15.60:0/1438291023 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdbad9d640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdbc06f040 0x7fcdbc1508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdbb59e640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdbc15c2f0 0x7fcdbc0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdba59c640 1 -- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdbc15c2f0 msgr2=0x7fcdbc0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdba59c640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdbc15c2f0 0x7fcdbc0b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdba59c640 1 -- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdbc06f040 msgr2=0x7fcdbc1508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdba59c640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdbc06f040 0x7fcdbc1508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdba59c640 1 -- 172.21.15.60:0/1438291023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda40094d0 con 0x7fcdbc140d40 2023-11-18T20:42:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdba59c640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc140d40 0x7fcdbc153690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcda40098d0 tx=0x7fcda4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdb37fe640 1 -- 172.21.15.60:0/1438291023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4047070 con 0x7fcdbc140d40 2023-11-18T20:42:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdb37fe640 1 -- 172.21.15.60:0/1438291023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda4030e30 con 0x7fcdbc140d40 2023-11-18T20:42:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdb37fe640 1 -- 172.21.15.60:0/1438291023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4039cd0 con 0x7fcdbc140d40 2023-11-18T20:42:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdc117c640 1 -- 172.21.15.60:0/1438291023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc06f4a0 con 0x7fcdbc140d40 2023-11-18T20:42:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.704+0000 7fcdc117c640 1 -- 172.21.15.60:0/1438291023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc154a10 con 0x7fcdbc140d40 2023-11-18T20:42:45.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.705+0000 7fcdc117c640 1 -- 172.21.15.60:0/1438291023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdbc069400 con 0x7fcdbc140d40 2023-11-18T20:42:45.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.706+0000 7fcdb37fe640 1 -- 172.21.15.60:0/1438291023 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcda404c410 con 0x7fcdbc140d40 2023-11-18T20:42:45.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.706+0000 7fcdb37fe640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcd9c06ae40 0x7fcd9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:45.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.706+0000 7fcdb37fe640 1 -- 172.21.15.60:0/1438291023 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcda403e070 con 0x7fcdbc140d40 2023-11-18T20:42:45.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.706+0000 7fcdbad9d640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcd9c06ae40 0x7fcd9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:45.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.707+0000 7fcdbad9d640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcd9c06ae40 0x7fcd9c06d300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40079a0 tx=0x7fcdb4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:45.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.710+0000 7fcdb37fe640 1 -- 172.21.15.60:0/1438291023 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcda4076c30 con 0x7fcdbc140d40 2023-11-18T20:42:45.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.882+0000 7fcdc117c640 1 -- 172.21.15.60:0/1438291023 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcdbc13e850 con 0x7fcd9c06ae40 2023-11-18T20:42:45.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.888+0000 7fcdb37fe640 1 -- 172.21.15.60:0/1438291023 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264547 (secure 0 0 0) 0x7fcdb40108d0 con 0x7fcd9c06ae40 2023-11-18T20:42:45.893 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:45.894 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:42:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 -- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcd9c06ae40 msgr2=0x7fcd9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcd9c06ae40 0x7fcd9c06d300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40079a0 tx=0x7fcdb4008040 comp rx=0 tx=0).stop 2023-11-18T20:42:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 -- 172.21.15.60:0/1438291023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc140d40 msgr2=0x7fcdbc153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc140d40 0x7fcdbc153690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcda40098d0 tx=0x7fcda4004730 comp rx=0 tx=0).stop 2023-11-18T20:42:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 -- 172.21.15.60:0/1438291023 shutdown_connections 2023-11-18T20:42:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcd9c06ae40 0x7fcd9c06d300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcdbc15c2f0 0x7fcdbc0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcdbc140d40 0x7fcdbc153690 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 --2- 172.21.15.60:0/1438291023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdbc06f040 0x7fcdbc1508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 -- 172.21.15.60:0/1438291023 >> 172.21.15.60:0/1438291023 conn(0x7fcdbc06a6b0 msgr2=0x7fcdbc15db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 -- 172.21.15.60:0/1438291023 shutdown_connections 2023-11-18T20:42:45.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:45.891+0000 7fcdc117c640 1 -- 172.21.15.60:0/1438291023 wait complete. 2023-11-18T20:42:45.903 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-18T20:42:45.437674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2916,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":2985984,"data_stored":1789585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002155"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084419+0000","last_change":"2023-11-18T20:42:38.103913+0000","last_active":"2023-11-18T20:42:39.084419+0000","last_peered":"2023-11-18T20:42:39.084419+0000","last_clean":"2023-11-18T20:42:39.084419+0000","last_became_active":"2023-11-18T20:42:38.103117+0000","last_became_peered":"2023-11-18T20:42:38.103117+0000","last_unstale":"2023-11-18T20:42:39.084419+0000","last_undegraded":"2023-11-18T20:42:39.084419+0000","last_fullsized":"2023-11-18T20:42:39.084419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:48:49.998802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085771+0000","last_change":"2023-11-18T20:42:38.089225+0000","last_active":"2023-11-18T20:42:39.085771+0000","last_peered":"2023-11-18T20:42:39.085771+0000","last_clean":"2023-11-18T20:42:39.085771+0000","last_became_active":"2023-11-18T20:42:38.088874+0000","last_became_peered":"2023-11-18T20:42:38.088874+0000","last_unstale":"2023-11-18T20:42:39.085771+0000","last_undegraded":"2023-11-18T20:42:39.085771+0000","last_fullsized":"2023-11-18T20:42:39.085771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:12:16.083479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085159+0000","last_change":"2023-11-18T20:42:38.090166+0000","last_active":"2023-11-18T20:42:39.085159+0000","last_peered":"2023-11-18T20:42:39.085159+0000","last_clean":"2023-11-18T20:42:39.085159+0000","last_became_active":"2023-11-18T20:42:38.089879+0000","last_became_peered":"2023-11-18T20:42:38.089879+0000","last_unstale":"2023-11-18T20:42:39.085159+0000","last_undegraded":"2023-11-18T20:42:39.085159+0000","last_fullsized":"2023-11-18T20:42:39.085159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:02.525692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086389+0000","last_change":"2023-11-18T20:42:38.105690+0000","last_active":"2023-11-18T20:42:39.086389+0000","last_peered":"2023-11-18T20:42:39.086389+0000","last_clean":"2023-11-18T20:42:39.086389+0000","last_became_active":"2023-11-18T20:42:38.105419+0000","last_became_peered":"2023-11-18T20:42:38.105419+0000","last_unstale":"2023-11-18T20:42:39.086389+0000","last_undegraded":"2023-11-18T20:42:39.086389+0000","last_fullsized":"2023-11-18T20:42:39.086389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:35:56.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084162+0000","last_change":"2023-11-18T20:42:38.106388+0000","last_active":"2023-11-18T20:42:39.084162+0000","last_peered":"2023-11-18T20:42:39.084162+0000","last_clean":"2023-11-18T20:42:39.084162+0000","last_became_active":"2023-11-18T20:42:38.105875+0000","last_became_peered":"2023-11-18T20:42:38.105875+0000","last_unstale":"2023-11-18T20:42:39.084162+0000","last_undegraded":"2023-11-18T20:42:39.084162+0000","last_fullsized":"2023-11-18T20:42:39.084162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:35:58.132615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086539+0000","last_change":"2023-11-18T20:42:38.104968+0000","last_active":"2023-11-18T20:42:39.086539+0000","last_peered":"2023-11-18T20:42:39.086539+0000","last_clean":"2023-11-18T20:42:39.086539+0000","last_became_active":"2023-11-18T20:42:38.104671+0000","last_became_peered":"2023-11-18T20:42:38.104671+0000","last_unstale":"2023-11-18T20:42:39.086539+0000","last_undegraded":"2023-11-18T20:42:39.086539+0000","last_fullsized":"2023-11-18T20:42:39.086539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:51:04.970201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083359+0000","last_change":"2023-11-18T20:42:38.101566+0000","last_active":"2023-11-18T20:42:39.083359+0000","last_peered":"2023-11-18T20:42:39.083359+0000","last_clean":"2023-11-18T20:42:39.083359+0000","last_became_active":"2023-11-18T20:42:38.101316+0000","last_became_peered":"2023-11-18T20:42:38.101316+0000","last_unstale":"2023-11-18T20:42:39.083359+0000","last_undegraded":"2023-11-18T20:42:39.083359+0000","last_fullsized":"2023-11-18T20:42:39.083359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:28.378102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084443+0000","last_change":"2023-11-18T20:42:38.102722+0000","last_active":"2023-11-18T20:42:39.084443+0000","last_peered":"2023-11-18T20:42:39.084443+0000","last_clean":"2023-11-18T20:42:39.084443+0000","last_became_active":"2023-11-18T20:42:38.102348+0000","last_became_peered":"2023-11-18T20:42:38.102348+0000","last_unstale":"2023-11-18T20:42:39.084443+0000","last_undegraded":"2023-11-18T20:42:39.084443+0000","last_fullsized":"2023-11-18T20:42:39.084443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:02.525692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084379+0000","last_change":"2023-11-18T20:42:38.107273+0000","last_active":"2023-11-18T20:42:39.084379+0000","last_peered":"2023-11-18T20:42:39.084379+0000","last_clean":"2023-11-18T20:42:39.084379+0000","last_became_active":"2023-11-18T20:42:38.107006+0000","last_became_peered":"2023-11-18T20:42:38.107006+0000","last_unstale":"2023-11-18T20:42:39.084379+0000","last_undegraded":"2023-11-18T20:42:39.084379+0000","last_fullsized":"2023-11-18T20:42:39.084379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:44:25.920568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086226+0000","last_change":"2023-11-18T20:42:38.105599+0000","last_active":"2023-11-18T20:42:39.086226+0000","last_peered":"2023-11-18T20:42:39.086226+0000","last_clean":"2023-11-18T20:42:39.086226+0000","last_became_active":"2023-11-18T20:42:38.105338+0000","last_became_peered":"2023-11-18T20:42:38.105338+0000","last_unstale":"2023-11-18T20:42:39.086226+0000","last_undegraded":"2023-11-18T20:42:39.086226+0000","last_fullsized":"2023-11-18T20:42:39.086226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086057+0000","last_change":"2023-11-18T20:42:38.100971+0000","last_active":"2023-11-18T20:42:39.086057+0000","last_peered":"2023-11-18T20:42:39.086057+0000","last_clean":"2023-11-18T20:42:39.086057+0000","last_became_active":"2023-11-18T20:42:38.100686+0000","last_became_peered":"2023-11-18T20:42:38.100686+0000","last_unstale":"2023-11-18T20:42:39.086057+0000","last_undegraded":"2023-11-18T20:42:39.086057+0000","last_fullsized":"2023-11-18T20:42:39.086057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083618+0000","last_change":"2023-11-18T20:42:38.098826+0000","last_active":"2023-11-18T20:42:39.083618+0000","last_peered":"2023-11-18T20:42:39.083618+0000","last_clean":"2023-11-18T20:42:39.083618+0000","last_became_active":"2023-11-18T20:42:38.098277+0000","last_became_peered":"2023-11-18T20:42:38.098277+0000","last_unstale":"2023-11-18T20:42:39.083618+0000","last_undegraded":"2023-11-18T20:42:39.083618+0000","last_fullsized":"2023-11-18T20:42:39.083618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:13.814862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085307+0000","last_change":"2023-11-18T20:42:38.098784+0000","last_active":"2023-11-18T20:42:39.085307+0000","last_peered":"2023-11-18T20:42:39.085307+0000","last_clean":"2023-11-18T20:42:39.085307+0000","last_became_active":"2023-11-18T20:42:38.098486+0000","last_became_peered":"2023-11-18T20:42:38.098486+0000","last_unstale":"2023-11-18T20:42:39.085307+0000","last_undegraded":"2023-11-18T20:42:39.085307+0000","last_fullsized":"2023-11-18T20:42:39.085307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084851+0000","last_change":"2023-11-18T20:42:38.092883+0000","last_active":"2023-11-18T20:42:39.084851+0000","last_peered":"2023-11-18T20:42:39.084851+0000","last_clean":"2023-11-18T20:42:39.084851+0000","last_became_active":"2023-11-18T20:42:38.092521+0000","last_became_peered":"2023-11-18T20:42:38.092521+0000","last_unstale":"2023-11-18T20:42:39.084851+0000","last_undegraded":"2023-11-18T20:42:39.084851+0000","last_fullsized":"2023-11-18T20:42:39.084851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:18.854895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084758+0000","last_change":"2023-11-18T20:42:38.090613+0000","last_active":"2023-11-18T20:42:39.084758+0000","last_peered":"2023-11-18T20:42:39.084758+0000","last_clean":"2023-11-18T20:42:39.084758+0000","last_became_active":"2023-11-18T20:42:38.090293+0000","last_became_peered":"2023-11-18T20:42:38.090293+0000","last_unstale":"2023-11-18T20:42:39.084758+0000","last_undegraded":"2023-11-18T20:42:39.084758+0000","last_fullsized":"2023-11-18T20:42:39.084758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086000+0000","last_change":"2023-11-18T20:42:38.098541+0000","last_active":"2023-11-18T20:42:39.086000+0000","last_peered":"2023-11-18T20:42:39.086000+0000","last_clean":"2023-11-18T20:42:39.086000+0000","last_became_active":"2023-11-18T20:42:38.098239+0000","last_became_peered":"2023-11-18T20:42:38.098239+0000","last_unstale":"2023-11-18T20:42:39.086000+0000","last_undegraded":"2023-11-18T20:42:39.086000+0000","last_fullsized":"2023-11-18T20:42:39.086000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086377+0000","last_change":"2023-11-18T20:42:38.107966+0000","last_active":"2023-11-18T20:42:39.086377+0000","last_peered":"2023-11-18T20:42:39.086377+0000","last_clean":"2023-11-18T20:42:39.086377+0000","last_became_active":"2023-11-18T20:42:38.107169+0000","last_became_peered":"2023-11-18T20:42:38.107169+0000","last_unstale":"2023-11-18T20:42:39.086377+0000","last_undegraded":"2023-11-18T20:42:39.086377+0000","last_fullsized":"2023-11-18T20:42:39.086377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:12:16.083479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082349+0000","last_change":"2023-11-18T20:42:38.095583+0000","last_active":"2023-11-18T20:42:39.082349+0000","last_peered":"2023-11-18T20:42:39.082349+0000","last_clean":"2023-11-18T20:42:39.082349+0000","last_became_active":"2023-11-18T20:42:38.095274+0000","last_became_peered":"2023-11-18T20:42:38.095274+0000","last_unstale":"2023-11-18T20:42:39.082349+0000","last_undegraded":"2023-11-18T20:42:39.082349+0000","last_fullsized":"2023-11-18T20:42:39.082349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:28:13.121246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083819+0000","last_change":"2023-11-18T20:42:38.107739+0000","last_active":"2023-11-18T20:42:39.083819+0000","last_peered":"2023-11-18T20:42:39.083819+0000","last_clean":"2023-11-18T20:42:39.083819+0000","last_became_active":"2023-11-18T20:42:38.106917+0000","last_became_peered":"2023-11-18T20:42:38.106917+0000","last_unstale":"2023-11-18T20:42:39.083819+0000","last_undegraded":"2023-11-18T20:42:39.083819+0000","last_fullsized":"2023-11-18T20:42:39.083819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:48:04.295517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086374+0000","last_change":"2023-11-18T20:42:38.103858+0000","last_active":"2023-11-18T20:42:39.086374+0000","last_peered":"2023-11-18T20:42:39.086374+0000","last_clean":"2023-11-18T20:42:39.086374+0000","last_became_active":"2023-11-18T20:42:38.103576+0000","last_became_peered":"2023-11-18T20:42:38.103576+0000","last_unstale":"2023-11-18T20:42:39.086374+0000","last_undegraded":"2023-11-18T20:42:39.086374+0000","last_fullsized":"2023-11-18T20:42:39.086374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085692+0000","last_change":"2023-11-18T20:42:38.100901+0000","last_active":"2023-11-18T20:42:39.085692+0000","last_peered":"2023-11-18T20:42:39.085692+0000","last_clean":"2023-11-18T20:42:39.085692+0000","last_became_active":"2023-11-18T20:42:38.100356+0000","last_became_peered":"2023-11-18T20:42:38.100356+0000","last_unstale":"2023-11-18T20:42:39.085692+0000","last_undegraded":"2023-11-18T20:42:39.085692+0000","last_fullsized":"2023-11-18T20:42:39.085692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:26.508116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086544+0000","last_change":"2023-11-18T20:42:38.102174+0000","last_active":"2023-11-18T20:42:39.086544+0000","last_peered":"2023-11-18T20:42:39.086544+0000","last_clean":"2023-11-18T20:42:39.086544+0000","last_became_active":"2023-11-18T20:42:38.100136+0000","last_became_peered":"2023-11-18T20:42:38.100136+0000","last_unstale":"2023-11-18T20:42:39.086544+0000","last_undegraded":"2023-11-18T20:42:39.086544+0000","last_fullsized":"2023-11-18T20:42:39.086544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:41:14.693957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084520+0000","last_change":"2023-11-18T20:42:38.099808+0000","last_active":"2023-11-18T20:42:39.084520+0000","last_peered":"2023-11-18T20:42:39.084520+0000","last_clean":"2023-11-18T20:42:39.084520+0000","last_became_active":"2023-11-18T20:42:38.099512+0000","last_became_peered":"2023-11-18T20:42:38.099512+0000","last_unstale":"2023-11-18T20:42:39.084520+0000","last_undegraded":"2023-11-18T20:42:39.084520+0000","last_fullsized":"2023-11-18T20:42:39.084520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:19.165207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086125+0000","last_change":"2023-11-18T20:42:38.100064+0000","last_active":"2023-11-18T20:42:39.086125+0000","last_peered":"2023-11-18T20:42:39.086125+0000","last_clean":"2023-11-18T20:42:39.086125+0000","last_became_active":"2023-11-18T20:42:38.099778+0000","last_became_peered":"2023-11-18T20:42:38.099778+0000","last_unstale":"2023-11-18T20:42:39.086125+0000","last_undegraded":"2023-11-18T20:42:39.086125+0000","last_fullsized":"2023-11-18T20:42:39.086125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084743+0000","last_change":"2023-11-18T20:42:38.101526+0000","last_active":"2023-11-18T20:42:39.084743+0000","last_peered":"2023-11-18T20:42:39.084743+0000","last_clean":"2023-11-18T20:42:39.084743+0000","last_became_active":"2023-11-18T20:42:38.099171+0000","last_became_peered":"2023-11-18T20:42:38.099171+0000","last_unstale":"2023-11-18T20:42:39.084743+0000","last_undegraded":"2023-11-18T20:42:39.084743+0000","last_fullsized":"2023-11-18T20:42:39.084743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:26.508116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086361+0000","last_change":"2023-11-18T20:42:38.100225+0000","last_active":"2023-11-18T20:42:39.086361+0000","last_peered":"2023-11-18T20:42:39.086361+0000","last_clean":"2023-11-18T20:42:39.086361+0000","last_became_active":"2023-11-18T20:42:38.099966+0000","last_became_peered":"2023-11-18T20:42:38.099966+0000","last_unstale":"2023-11-18T20:42:39.086361+0000","last_undegraded":"2023-11-18T20:42:39.086361+0000","last_fullsized":"2023-11-18T20:42:39.086361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085054+0000","last_change":"2023-11-18T20:42:38.094313+0000","last_active":"2023-11-18T20:42:39.085054+0000","last_peered":"2023-11-18T20:42:39.085054+0000","last_clean":"2023-11-18T20:42:39.085054+0000","last_became_active":"2023-11-18T20:42:38.094005+0000","last_became_peered":"2023-11-18T20:42:38.094005+0000","last_unstale":"2023-11-18T20:42:39.085054+0000","last_undegraded":"2023-11-18T20:42:39.085054+0000","last_fullsized":"2023-11-18T20:42:39.085054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085575+0000","last_change":"2023-11-18T20:42:38.100511+0000","last_active":"2023-11-18T20:42:39.085575+0000","last_peered":"2023-11-18T20:42:39.085575+0000","last_clean":"2023-11-18T20:42:39.085575+0000","last_became_active":"2023-11-18T20:42:38.100119+0000","last_became_peered":"2023-11-18T20:42:38.100119+0000","last_unstale":"2023-11-18T20:42:39.085575+0000","last_undegraded":"2023-11-18T20:42:39.085575+0000","last_fullsized":"2023-11-18T20:42:39.085575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:12:16.083479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085818+0000","last_change":"2023-11-18T20:42:38.098009+0000","last_active":"2023-11-18T20:42:39.085818+0000","last_peered":"2023-11-18T20:42:39.085818+0000","last_clean":"2023-11-18T20:42:39.085818+0000","last_became_active":"2023-11-18T20:42:38.097728+0000","last_became_peered":"2023-11-18T20:42:38.097728+0000","last_unstale":"2023-11-18T20:42:39.085818+0000","last_undegraded":"2023-11-18T20:42:39.085818+0000","last_fullsized":"2023-11-18T20:42:39.085818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086654+0000","last_change":"2023-11-18T20:42:38.099195+0000","last_active":"2023-11-18T20:42:39.086654+0000","last_peered":"2023-11-18T20:42:39.086654+0000","last_clean":"2023-11-18T20:42:39.086654+0000","last_became_active":"2023-11-18T20:42:38.098918+0000","last_became_peered":"2023-11-18T20:42:38.098918+0000","last_unstale":"2023-11-18T20:42:39.086654+0000","last_undegraded":"2023-11-18T20:42:39.086654+0000","last_fullsized":"2023-11-18T20:42:39.086654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:48:04.295517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082460+0000","last_change":"2023-11-18T20:42:38.095631+0000","last_active":"2023-11-18T20:42:39.082460+0000","last_peered":"2023-11-18T20:42:39.082460+0000","last_clean":"2023-11-18T20:42:39.082460+0000","last_became_active":"2023-11-18T20:42:38.095319+0000","last_became_peered":"2023-11-18T20:42:38.095319+0000","last_unstale":"2023-11-18T20:42:39.082460+0000","last_undegraded":"2023-11-18T20:42:39.082460+0000","last_fullsized":"2023-11-18T20:42:39.082460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:13.814862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085792+0000","last_change":"2023-11-18T20:42:38.103076+0000","last_active":"2023-11-18T20:42:39.085792+0000","last_peered":"2023-11-18T20:42:39.085792+0000","last_clean":"2023-11-18T20:42:39.085792+0000","last_became_active":"2023-11-18T20:42:38.102829+0000","last_became_peered":"2023-11-18T20:42:38.102829+0000","last_unstale":"2023-11-18T20:42:39.085792+0000","last_undegraded":"2023-11-18T20:42:39.085792+0000","last_fullsized":"2023-11-18T20:42:39.085792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:14.531419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083227+0000","last_change":"2023-11-18T20:42:38.089642+0000","last_active":"2023-11-18T20:42:39.083227+0000","last_peered":"2023-11-18T20:42:39.083227+0000","last_clean":"2023-11-18T20:42:39.083227+0000","last_became_active":"2023-11-18T20:42:38.089367+0000","last_became_peered":"2023-11-18T20:42:38.089367+0000","last_unstale":"2023-11-18T20:42:39.083227+0000","last_undegraded":"2023-11-18T20:42:39.083227+0000","last_fullsized":"2023-11-18T20:42:39.083227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:28:13.121246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085545+0000","last_change":"2023-11-18T20:42:37.111681+0000","last_active":"2023-11-18T20:42:39.085545+0000","last_peered":"2023-11-18T20:42:39.085545+0000","last_clean":"2023-11-18T20:42:39.085545+0000","last_became_active":"2023-11-18T20:42:37.111280+0000","last_became_peered":"2023-11-18T20:42:37.111280+0000","last_unstale":"2023-11-18T20:42:39.085545+0000","last_undegraded":"2023-11-18T20:42:39.085545+0000","last_fullsized":"2023-11-18T20:42:39.085545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:31.712397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085932+0000","last_change":"2023-11-18T20:42:38.097798+0000","last_active":"2023-11-18T20:42:39.085932+0000","last_peered":"2023-11-18T20:42:39.085932+0000","last_clean":"2023-11-18T20:42:39.085932+0000","last_became_active":"2023-11-18T20:42:38.096529+0000","last_became_peered":"2023-11-18T20:42:38.096529+0000","last_unstale":"2023-11-18T20:42:39.085932+0000","last_undegraded":"2023-11-18T20:42:39.085932+0000","last_fullsized":"2023-11-18T20:42:39.085932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086235+0000","last_change":"2023-11-18T20:42:37.106910+0000","last_active":"2023-11-18T20:42:39.086235+0000","last_peered":"2023-11-18T20:42:39.086235+0000","last_clean":"2023-11-18T20:42:39.086235+0000","last_became_active":"2023-11-18T20:42:37.106625+0000","last_became_peered":"2023-11-18T20:42:37.106625+0000","last_unstale":"2023-11-18T20:42:39.086235+0000","last_undegraded":"2023-11-18T20:42:39.086235+0000","last_fullsized":"2023-11-18T20:42:39.086235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:43.316226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084601+0000","last_change":"2023-11-18T20:42:37.109273+0000","last_active":"2023-11-18T20:42:39.084601+0000","last_peered":"2023-11-18T20:42:39.084601+0000","last_clean":"2023-11-18T20:42:39.084601+0000","last_became_active":"2023-11-18T20:42:37.108948+0000","last_became_peered":"2023-11-18T20:42:37.108948+0000","last_unstale":"2023-11-18T20:42:39.084601+0000","last_undegraded":"2023-11-18T20:42:39.084601+0000","last_fullsized":"2023-11-18T20:42:39.084601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:12.810253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084564+0000","last_change":"2023-11-18T20:42:38.101286+0000","last_active":"2023-11-18T20:42:39.084564+0000","last_peered":"2023-11-18T20:42:39.084564+0000","last_clean":"2023-11-18T20:42:39.084564+0000","last_became_active":"2023-11-18T20:42:38.100974+0000","last_became_peered":"2023-11-18T20:42:38.100974+0000","last_unstale":"2023-11-18T20:42:39.084564+0000","last_undegraded":"2023-11-18T20:42:39.084564+0000","last_fullsized":"2023-11-18T20:42:39.084564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:14.531419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086311+0000","last_change":"2023-11-18T20:42:37.105690+0000","last_active":"2023-11-18T20:42:39.086311+0000","last_peered":"2023-11-18T20:42:39.086311+0000","last_clean":"2023-11-18T20:42:39.086311+0000","last_became_active":"2023-11-18T20:42:37.105410+0000","last_became_peered":"2023-11-18T20:42:37.105410+0000","last_unstale":"2023-11-18T20:42:39.086311+0000","last_undegraded":"2023-11-18T20:42:39.086311+0000","last_fullsized":"2023-11-18T20:42:39.086311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:14:29.503121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084618+0000","last_change":"2023-11-18T20:42:38.096986+0000","last_active":"2023-11-18T20:42:39.084618+0000","last_peered":"2023-11-18T20:42:39.084618+0000","last_clean":"2023-11-18T20:42:39.084618+0000","last_became_active":"2023-11-18T20:42:38.095964+0000","last_became_peered":"2023-11-18T20:42:38.095964+0000","last_unstale":"2023-11-18T20:42:39.084618+0000","last_undegraded":"2023-11-18T20:42:39.084618+0000","last_fullsized":"2023-11-18T20:42:39.084618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086366+0000","last_change":"2023-11-18T20:42:37.111419+0000","last_active":"2023-11-18T20:42:39.086366+0000","last_peered":"2023-11-18T20:42:39.086366+0000","last_clean":"2023-11-18T20:42:39.086366+0000","last_became_active":"2023-11-18T20:42:37.110877+0000","last_became_peered":"2023-11-18T20:42:37.110877+0000","last_unstale":"2023-11-18T20:42:39.086366+0000","last_undegraded":"2023-11-18T20:42:39.086366+0000","last_fullsized":"2023-11-18T20:42:39.086366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086126+0000","last_change":"2023-11-18T20:42:38.108594+0000","last_active":"2023-11-18T20:42:39.086126+0000","last_peered":"2023-11-18T20:42:39.086126+0000","last_clean":"2023-11-18T20:42:39.086126+0000","last_became_active":"2023-11-18T20:42:38.108325+0000","last_became_peered":"2023-11-18T20:42:38.108325+0000","last_unstale":"2023-11-18T20:42:39.086126+0000","last_undegraded":"2023-11-18T20:42:39.086126+0000","last_fullsized":"2023-11-18T20:42:39.086126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086218+0000","last_change":"2023-11-18T20:42:37.106865+0000","last_active":"2023-11-18T20:42:39.086218+0000","last_peered":"2023-11-18T20:42:39.086218+0000","last_clean":"2023-11-18T20:42:39.086218+0000","last_became_active":"2023-11-18T20:42:37.106561+0000","last_became_peered":"2023-11-18T20:42:37.106561+0000","last_unstale":"2023-11-18T20:42:39.086218+0000","last_undegraded":"2023-11-18T20:42:39.086218+0000","last_fullsized":"2023-11-18T20:42:39.086218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:25.503507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085723+0000","last_change":"2023-11-18T20:42:38.101545+0000","last_active":"2023-11-18T20:42:39.085723+0000","last_peered":"2023-11-18T20:42:39.085723+0000","last_clean":"2023-11-18T20:42:39.085723+0000","last_became_active":"2023-11-18T20:42:38.101247+0000","last_became_peered":"2023-11-18T20:42:38.101247+0000","last_unstale":"2023-11-18T20:42:39.085723+0000","last_undegraded":"2023-11-18T20:42:39.085723+0000","last_fullsized":"2023-11-18T20:42:39.085723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085146+0000","last_change":"2023-11-18T20:42:37.109345+0000","last_active":"2023-11-18T20:42:39.085146+0000","last_peered":"2023-11-18T20:42:39.085146+0000","last_clean":"2023-11-18T20:42:39.085146+0000","last_became_active":"2023-11-18T20:42:37.109081+0000","last_became_peered":"2023-11-18T20:42:37.109081+0000","last_unstale":"2023-11-18T20:42:39.085146+0000","last_undegraded":"2023-11-18T20:42:39.085146+0000","last_fullsized":"2023-11-18T20:42:39.085146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083745+0000","last_change":"2023-11-18T20:42:38.100125+0000","last_active":"2023-11-18T20:42:39.083745+0000","last_peered":"2023-11-18T20:42:39.083745+0000","last_clean":"2023-11-18T20:42:39.083745+0000","last_became_active":"2023-11-18T20:42:38.099646+0000","last_became_peered":"2023-11-18T20:42:38.099646+0000","last_unstale":"2023-11-18T20:42:39.083745+0000","last_undegraded":"2023-11-18T20:42:39.083745+0000","last_fullsized":"2023-11-18T20:42:39.083745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:14:30.507730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086459+0000","last_change":"2023-11-18T20:42:37.106986+0000","last_active":"2023-11-18T20:42:39.086459+0000","last_peered":"2023-11-18T20:42:39.086459+0000","last_clean":"2023-11-18T20:42:39.086459+0000","last_became_active":"2023-11-18T20:42:37.106707+0000","last_became_peered":"2023-11-18T20:42:37.106707+0000","last_unstale":"2023-11-18T20:42:39.086459+0000","last_undegraded":"2023-11-18T20:42:39.086459+0000","last_fullsized":"2023-11-18T20:42:39.086459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:31.712397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083280+0000","last_change":"2023-11-18T20:42:38.107769+0000","last_active":"2023-11-18T20:42:39.083280+0000","last_peered":"2023-11-18T20:42:39.083280+0000","last_clean":"2023-11-18T20:42:39.083280+0000","last_became_active":"2023-11-18T20:42:38.104253+0000","last_became_peered":"2023-11-18T20:42:38.104253+0000","last_unstale":"2023-11-18T20:42:39.083280+0000","last_undegraded":"2023-11-18T20:42:39.083280+0000","last_fullsized":"2023-11-18T20:42:39.083280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:45:28.196671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085912+0000","last_change":"2023-11-18T20:42:37.098519+0000","last_active":"2023-11-18T20:42:39.085912+0000","last_peered":"2023-11-18T20:42:39.085912+0000","last_clean":"2023-11-18T20:42:39.085912+0000","last_became_active":"2023-11-18T20:42:37.097972+0000","last_became_peered":"2023-11-18T20:42:37.097972+0000","last_unstale":"2023-11-18T20:42:39.085912+0000","last_undegraded":"2023-11-18T20:42:39.085912+0000","last_fullsized":"2023-11-18T20:42:39.085912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:56:12.810253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084892+0000","last_change":"2023-11-18T20:42:38.100733+0000","last_active":"2023-11-18T20:42:39.084892+0000","last_peered":"2023-11-18T20:42:39.084892+0000","last_clean":"2023-11-18T20:42:39.084892+0000","last_became_active":"2023-11-18T20:42:38.100390+0000","last_became_peered":"2023-11-18T20:42:38.100390+0000","last_unstale":"2023-11-18T20:42:39.084892+0000","last_undegraded":"2023-11-18T20:42:39.084892+0000","last_fullsized":"2023-11-18T20:42:39.084892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:32.717006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086635+0000","last_change":"2023-11-18T20:42:37.110943+0000","last_active":"2023-11-18T20:42:39.086635+0000","last_peered":"2023-11-18T20:42:39.086635+0000","last_clean":"2023-11-18T20:42:39.086635+0000","last_became_active":"2023-11-18T20:42:37.110657+0000","last_became_peered":"2023-11-18T20:42:37.110657+0000","last_unstale":"2023-11-18T20:42:39.086635+0000","last_undegraded":"2023-11-18T20:42:39.086635+0000","last_fullsized":"2023-11-18T20:42:39.086635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084234+0000","last_change":"2023-11-18T20:42:38.102821+0000","last_active":"2023-11-18T20:42:39.084234+0000","last_peered":"2023-11-18T20:42:39.084234+0000","last_clean":"2023-11-18T20:42:39.084234+0000","last_became_active":"2023-11-18T20:42:38.102491+0000","last_became_peered":"2023-11-18T20:42:38.102491+0000","last_unstale":"2023-11-18T20:42:39.084234+0000","last_undegraded":"2023-11-18T20:42:39.084234+0000","last_fullsized":"2023-11-18T20:42:39.084234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:17:48.228298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085793+0000","last_change":"2023-11-18T20:42:37.107187+0000","last_active":"2023-11-18T20:42:39.085793+0000","last_peered":"2023-11-18T20:42:39.085793+0000","last_clean":"2023-11-18T20:42:39.085793+0000","last_became_active":"2023-11-18T20:42:37.106904+0000","last_became_peered":"2023-11-18T20:42:37.106904+0000","last_unstale":"2023-11-18T20:42:39.085793+0000","last_undegraded":"2023-11-18T20:42:39.085793+0000","last_fullsized":"2023-11-18T20:42:39.085793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:17.850286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085803+0000","last_change":"2023-11-18T20:42:38.102486+0000","last_active":"2023-11-18T20:42:39.085803+0000","last_peered":"2023-11-18T20:42:39.085803+0000","last_clean":"2023-11-18T20:42:39.085803+0000","last_became_active":"2023-11-18T20:42:38.102239+0000","last_became_peered":"2023-11-18T20:42:38.102239+0000","last_unstale":"2023-11-18T20:42:39.085803+0000","last_undegraded":"2023-11-18T20:42:39.085803+0000","last_fullsized":"2023-11-18T20:42:39.085803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:26.508116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082180+0000","last_change":"2023-11-18T20:42:37.105948+0000","last_active":"2023-11-18T20:42:39.082180+0000","last_peered":"2023-11-18T20:42:39.082180+0000","last_clean":"2023-11-18T20:42:39.082180+0000","last_became_active":"2023-11-18T20:42:37.105280+0000","last_became_peered":"2023-11-18T20:42:37.105280+0000","last_unstale":"2023-11-18T20:42:39.082180+0000","last_undegraded":"2023-11-18T20:42:39.082180+0000","last_fullsized":"2023-11-18T20:42:39.082180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:55:27.947068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083496+0000","last_change":"2023-11-18T20:42:38.107573+0000","last_active":"2023-11-18T20:42:39.083496+0000","last_peered":"2023-11-18T20:42:39.083496+0000","last_clean":"2023-11-18T20:42:39.083496+0000","last_became_active":"2023-11-18T20:42:38.103361+0000","last_became_peered":"2023-11-18T20:42:38.103361+0000","last_unstale":"2023-11-18T20:42:39.083496+0000","last_undegraded":"2023-11-18T20:42:39.083496+0000","last_fullsized":"2023-11-18T20:42:39.083496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:04:55.965852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083454+0000","last_change":"2023-11-18T20:42:37.097968+0000","last_active":"2023-11-18T20:42:39.083454+0000","last_peered":"2023-11-18T20:42:39.083454+0000","last_clean":"2023-11-18T20:42:39.083454+0000","last_became_active":"2023-11-18T20:42:37.097659+0000","last_became_peered":"2023-11-18T20:42:37.097659+0000","last_unstale":"2023-11-18T20:42:39.083454+0000","last_undegraded":"2023-11-18T20:42:39.083454+0000","last_fullsized":"2023-11-18T20:42:39.083454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:08:51.955879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083672+0000","last_change":"2023-11-18T20:42:38.107457+0000","last_active":"2023-11-18T20:42:39.083672+0000","last_peered":"2023-11-18T20:42:39.083672+0000","last_clean":"2023-11-18T20:42:39.083672+0000","last_became_active":"2023-11-18T20:42:38.103260+0000","last_became_peered":"2023-11-18T20:42:38.103260+0000","last_unstale":"2023-11-18T20:42:39.083672+0000","last_undegraded":"2023-11-18T20:42:39.083672+0000","last_fullsized":"2023-11-18T20:42:39.083672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:41:14.693957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082379+0000","last_change":"2023-11-18T20:42:37.109692+0000","last_active":"2023-11-18T20:42:39.082379+0000","last_peered":"2023-11-18T20:42:39.082379+0000","last_clean":"2023-11-18T20:42:39.082379+0000","last_became_active":"2023-11-18T20:42:37.109328+0000","last_became_peered":"2023-11-18T20:42:37.109328+0000","last_unstale":"2023-11-18T20:42:39.082379+0000","last_undegraded":"2023-11-18T20:42:39.082379+0000","last_fullsized":"2023-11-18T20:42:39.082379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:08:51.955879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084464+0000","last_change":"2023-11-18T20:42:38.107442+0000","last_active":"2023-11-18T20:42:39.084464+0000","last_peered":"2023-11-18T20:42:39.084464+0000","last_clean":"2023-11-18T20:42:39.084464+0000","last_became_active":"2023-11-18T20:42:38.106939+0000","last_became_peered":"2023-11-18T20:42:38.106939+0000","last_unstale":"2023-11-18T20:42:39.084464+0000","last_undegraded":"2023-11-18T20:42:39.084464+0000","last_fullsized":"2023-11-18T20:42:39.084464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:52:14.406378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085180+0000","last_change":"2023-11-18T20:42:37.107734+0000","last_active":"2023-11-18T20:42:39.085180+0000","last_peered":"2023-11-18T20:42:39.085180+0000","last_clean":"2023-11-18T20:42:39.085180+0000","last_became_active":"2023-11-18T20:42:37.107466+0000","last_became_peered":"2023-11-18T20:42:37.107466+0000","last_unstale":"2023-11-18T20:42:39.085180+0000","last_undegraded":"2023-11-18T20:42:39.085180+0000","last_fullsized":"2023-11-18T20:42:39.085180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:13.526810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086537+0000","last_change":"2023-11-18T20:42:38.105663+0000","last_active":"2023-11-18T20:42:39.086537+0000","last_peered":"2023-11-18T20:42:39.086537+0000","last_clean":"2023-11-18T20:42:39.086537+0000","last_became_active":"2023-11-18T20:42:38.105269+0000","last_became_peered":"2023-11-18T20:42:38.105269+0000","last_unstale":"2023-11-18T20:42:39.086537+0000","last_undegraded":"2023-11-18T20:42:39.086537+0000","last_fullsized":"2023-11-18T20:42:39.086537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:04:55.965852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082362+0000","last_change":"2023-11-18T20:42:37.109021+0000","last_active":"2023-11-18T20:42:39.082362+0000","last_peered":"2023-11-18T20:42:39.082362+0000","last_clean":"2023-11-18T20:42:39.082362+0000","last_became_active":"2023-11-18T20:42:37.108256+0000","last_became_peered":"2023-11-18T20:42:37.108256+0000","last_unstale":"2023-11-18T20:42:39.082362+0000","last_undegraded":"2023-11-18T20:42:39.082362+0000","last_fullsized":"2023-11-18T20:42:39.082362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:17:02.258981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083639+0000","last_change":"2023-11-18T20:42:38.107956+0000","last_active":"2023-11-18T20:42:39.083639+0000","last_peered":"2023-11-18T20:42:39.083639+0000","last_clean":"2023-11-18T20:42:39.083639+0000","last_became_active":"2023-11-18T20:42:38.107488+0000","last_became_peered":"2023-11-18T20:42:38.107488+0000","last_unstale":"2023-11-18T20:42:39.083639+0000","last_undegraded":"2023-11-18T20:42:39.083639+0000","last_fullsized":"2023-11-18T20:42:39.083639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:51:04.970201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-18T20:42:38.097227+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:42:38.097227+0000","last_peered":"2023-11-18T20:42:38.097227+0000","last_clean":"2023-11-18T20:42:38.097227+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:42:38.097227+0000","last_undegraded":"2023-11-18T20:42:38.097227+0000","last_fullsized":"2023-11-18T20:42:38.097227+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086001+0000","last_change":"2023-11-18T20:42:37.110512+0000","last_active":"2023-11-18T20:42:39.086001+0000","last_peered":"2023-11-18T20:42:39.086001+0000","last_clean":"2023-11-18T20:42:39.086001+0000","last_became_active":"2023-11-18T20:42:37.110213+0000","last_became_peered":"2023-11-18T20:42:37.110213+0000","last_unstale":"2023-11-18T20:42:39.086001+0000","last_undegraded":"2023-11-18T20:42:39.086001+0000","last_fullsized":"2023-11-18T20:42:39.086001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:43.316226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084751+0000","last_change":"2023-11-18T20:42:38.105196+0000","last_active":"2023-11-18T20:42:39.084751+0000","last_peered":"2023-11-18T20:42:39.084751+0000","last_clean":"2023-11-18T20:42:39.084751+0000","last_became_active":"2023-11-18T20:42:38.104886+0000","last_became_peered":"2023-11-18T20:42:38.104886+0000","last_unstale":"2023-11-18T20:42:39.084751+0000","last_undegraded":"2023-11-18T20:42:39.084751+0000","last_fullsized":"2023-11-18T20:42:39.084751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:59:11.417541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084031+0000","last_change":"2023-11-18T20:42:37.108345+0000","last_active":"2023-11-18T20:42:39.084031+0000","last_peered":"2023-11-18T20:42:39.084031+0000","last_clean":"2023-11-18T20:42:39.084031+0000","last_became_active":"2023-11-18T20:42:37.107995+0000","last_became_peered":"2023-11-18T20:42:37.107995+0000","last_unstale":"2023-11-18T20:42:39.084031+0000","last_undegraded":"2023-11-18T20:42:39.084031+0000","last_fullsized":"2023-11-18T20:42:39.084031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:15.427234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084749+0000","last_change":"2023-11-18T20:42:38.088232+0000","last_active":"2023-11-18T20:42:39.084749+0000","last_peered":"2023-11-18T20:42:39.084749+0000","last_clean":"2023-11-18T20:42:39.084749+0000","last_became_active":"2023-11-18T20:42:38.087631+0000","last_became_peered":"2023-11-18T20:42:38.087631+0000","last_unstale":"2023-11-18T20:42:39.084749+0000","last_undegraded":"2023-11-18T20:42:39.084749+0000","last_fullsized":"2023-11-18T20:42:39.084749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:44.320835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.082457+0000","last_change":"2023-11-18T20:42:37.107910+0000","last_active":"2023-11-18T20:42:39.082457+0000","last_peered":"2023-11-18T20:42:39.082457+0000","last_clean":"2023-11-18T20:42:39.082457+0000","last_became_active":"2023-11-18T20:42:37.106792+0000","last_became_peered":"2023-11-18T20:42:37.106792+0000","last_unstale":"2023-11-18T20:42:39.082457+0000","last_undegraded":"2023-11-18T20:42:39.082457+0000","last_fullsized":"2023-11-18T20:42:39.082457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:03:11.287757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084900+0000","last_change":"2023-11-18T20:42:38.090117+0000","last_active":"2023-11-18T20:42:39.084900+0000","last_peered":"2023-11-18T20:42:39.084900+0000","last_clean":"2023-11-18T20:42:39.084900+0000","last_became_active":"2023-11-18T20:42:38.089821+0000","last_became_peered":"2023-11-18T20:42:38.089821+0000","last_unstale":"2023-11-18T20:42:39.084900+0000","last_undegraded":"2023-11-18T20:42:39.084900+0000","last_fullsized":"2023-11-18T20:42:39.084900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086690+0000","last_change":"2023-11-18T20:42:37.111162+0000","last_active":"2023-11-18T20:42:39.086690+0000","last_peered":"2023-11-18T20:42:39.086690+0000","last_clean":"2023-11-18T20:42:39.086690+0000","last_became_active":"2023-11-18T20:42:37.110692+0000","last_became_peered":"2023-11-18T20:42:37.110692+0000","last_unstale":"2023-11-18T20:42:39.086690+0000","last_undegraded":"2023-11-18T20:42:39.086690+0000","last_fullsized":"2023-11-18T20:42:39.086690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084091+0000","last_change":"2023-11-18T20:42:38.091366+0000","last_active":"2023-11-18T20:42:39.084091+0000","last_peered":"2023-11-18T20:42:39.084091+0000","last_clean":"2023-11-18T20:42:39.084091+0000","last_became_active":"2023-11-18T20:42:38.089726+0000","last_became_peered":"2023-11-18T20:42:38.089726+0000","last_unstale":"2023-11-18T20:42:39.084091+0000","last_undegraded":"2023-11-18T20:42:39.084091+0000","last_fullsized":"2023-11-18T20:42:39.084091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:44.320835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085074+0000","last_change":"2023-11-18T20:42:37.103840+0000","last_active":"2023-11-18T20:42:39.085074+0000","last_peered":"2023-11-18T20:42:39.085074+0000","last_clean":"2023-11-18T20:42:39.085074+0000","last_became_active":"2023-11-18T20:42:37.103516+0000","last_became_peered":"2023-11-18T20:42:37.103516+0000","last_unstale":"2023-11-18T20:42:39.085074+0000","last_undegraded":"2023-11-18T20:42:39.085074+0000","last_fullsized":"2023-11-18T20:42:39.085074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086520+0000","last_change":"2023-11-18T20:42:38.100140+0000","last_active":"2023-11-18T20:42:39.086520+0000","last_peered":"2023-11-18T20:42:39.086520+0000","last_clean":"2023-11-18T20:42:39.086520+0000","last_became_active":"2023-11-18T20:42:38.099880+0000","last_became_peered":"2023-11-18T20:42:38.099880+0000","last_unstale":"2023-11-18T20:42:39.086520+0000","last_undegraded":"2023-11-18T20:42:39.086520+0000","last_fullsized":"2023-11-18T20:42:39.086520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:28.378102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083840+0000","last_change":"2023-11-18T20:42:37.103907+0000","last_active":"2023-11-18T20:42:39.083840+0000","last_peered":"2023-11-18T20:42:39.083840+0000","last_clean":"2023-11-18T20:42:39.083840+0000","last_became_active":"2023-11-18T20:42:37.103642+0000","last_became_peered":"2023-11-18T20:42:37.103642+0000","last_unstale":"2023-11-18T20:42:39.083840+0000","last_undegraded":"2023-11-18T20:42:39.083840+0000","last_fullsized":"2023-11-18T20:42:39.083840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:19:25.503507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084990+0000","last_change":"2023-11-18T20:42:38.090609+0000","last_active":"2023-11-18T20:42:39.084990+0000","last_peered":"2023-11-18T20:42:39.084990+0000","last_clean":"2023-11-18T20:42:39.084990+0000","last_became_active":"2023-11-18T20:42:38.090275+0000","last_became_peered":"2023-11-18T20:42:38.090275+0000","last_unstale":"2023-11-18T20:42:39.084990+0000","last_undegraded":"2023-11-18T20:42:39.084990+0000","last_fullsized":"2023-11-18T20:42:39.084990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:02.637261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.450170+0000","last_change":"2023-11-18T20:42:37.110209+0000","last_active":"2023-11-18T20:42:39.450170+0000","last_peered":"2023-11-18T20:42:39.450170+0000","last_clean":"2023-11-18T20:42:39.450170+0000","last_became_active":"2023-11-18T20:42:37.109906+0000","last_became_peered":"2023-11-18T20:42:37.109906+0000","last_unstale":"2023-11-18T20:42:39.450170+0000","last_undegraded":"2023-11-18T20:42:39.450170+0000","last_fullsized":"2023-11-18T20:42:39.450170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:17.850286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084866+0000","last_change":"2023-11-18T20:42:38.104224+0000","last_active":"2023-11-18T20:42:39.084866+0000","last_peered":"2023-11-18T20:42:39.084866+0000","last_clean":"2023-11-18T20:42:39.084866+0000","last_became_active":"2023-11-18T20:42:38.103950+0000","last_became_peered":"2023-11-18T20:42:38.103950+0000","last_unstale":"2023-11-18T20:42:39.084866+0000","last_undegraded":"2023-11-18T20:42:39.084866+0000","last_fullsized":"2023-11-18T20:42:39.084866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086459+0000","last_change":"2023-11-18T20:42:37.102910+0000","last_active":"2023-11-18T20:42:39.086459+0000","last_peered":"2023-11-18T20:42:39.086459+0000","last_clean":"2023-11-18T20:42:39.086459+0000","last_became_active":"2023-11-18T20:42:37.102612+0000","last_became_peered":"2023-11-18T20:42:37.102612+0000","last_unstale":"2023-11-18T20:42:39.086459+0000","last_undegraded":"2023-11-18T20:42:39.086459+0000","last_fullsized":"2023-11-18T20:42:39.086459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085442+0000","last_change":"2023-11-18T20:42:38.094021+0000","last_active":"2023-11-18T20:42:39.085442+0000","last_peered":"2023-11-18T20:42:39.085442+0000","last_clean":"2023-11-18T20:42:39.085442+0000","last_became_active":"2023-11-18T20:42:38.093125+0000","last_became_peered":"2023-11-18T20:42:38.093125+0000","last_unstale":"2023-11-18T20:42:39.085442+0000","last_undegraded":"2023-11-18T20:42:39.085442+0000","last_fullsized":"2023-11-18T20:42:39.085442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:19.165207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083934+0000","last_change":"2023-11-18T20:42:37.102964+0000","last_active":"2023-11-18T20:42:39.083934+0000","last_peered":"2023-11-18T20:42:39.083934+0000","last_clean":"2023-11-18T20:42:39.083934+0000","last_became_active":"2023-11-18T20:42:37.102696+0000","last_became_peered":"2023-11-18T20:42:37.102696+0000","last_unstale":"2023-11-18T20:42:39.083934+0000","last_undegraded":"2023-11-18T20:42:39.083934+0000","last_fullsized":"2023-11-18T20:42:39.083934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:51.367829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084007+0000","last_change":"2023-11-18T20:42:38.089921+0000","last_active":"2023-11-18T20:42:39.084007+0000","last_peered":"2023-11-18T20:42:39.084007+0000","last_clean":"2023-11-18T20:42:39.084007+0000","last_became_active":"2023-11-18T20:42:38.089451+0000","last_became_peered":"2023-11-18T20:42:38.089451+0000","last_unstale":"2023-11-18T20:42:39.084007+0000","last_undegraded":"2023-11-18T20:42:39.084007+0000","last_fullsized":"2023-11-18T20:42:39.084007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:03:26.636590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084840+0000","last_change":"2023-11-18T20:42:37.110785+0000","last_active":"2023-11-18T20:42:39.084840+0000","last_peered":"2023-11-18T20:42:39.084840+0000","last_clean":"2023-11-18T20:42:39.084840+0000","last_became_active":"2023-11-18T20:42:37.110523+0000","last_became_peered":"2023-11-18T20:42:37.110523+0000","last_unstale":"2023-11-18T20:42:39.084840+0000","last_undegraded":"2023-11-18T20:42:39.084840+0000","last_fullsized":"2023-11-18T20:42:39.084840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:49:43.316226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085653+0000","last_change":"2023-11-18T20:42:38.097248+0000","last_active":"2023-11-18T20:42:39.085653+0000","last_peered":"2023-11-18T20:42:39.085653+0000","last_clean":"2023-11-18T20:42:39.085653+0000","last_became_active":"2023-11-18T20:42:38.096931+0000","last_became_peered":"2023-11-18T20:42:38.096931+0000","last_unstale":"2023-11-18T20:42:39.085653+0000","last_undegraded":"2023-11-18T20:42:39.085653+0000","last_fullsized":"2023-11-18T20:42:39.085653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:35:56.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085556+0000","last_change":"2023-11-18T20:42:37.106332+0000","last_active":"2023-11-18T20:42:39.085556+0000","last_peered":"2023-11-18T20:42:39.085556+0000","last_clean":"2023-11-18T20:42:39.085556+0000","last_became_active":"2023-11-18T20:42:37.106048+0000","last_became_peered":"2023-11-18T20:42:37.106048+0000","last_unstale":"2023-11-18T20:42:39.085556+0000","last_undegraded":"2023-11-18T20:42:39.085556+0000","last_fullsized":"2023-11-18T20:42:39.085556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:54:31.712397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085454+0000","last_change":"2023-11-18T20:42:38.100884+0000","last_active":"2023-11-18T20:42:39.085454+0000","last_peered":"2023-11-18T20:42:39.085454+0000","last_clean":"2023-11-18T20:42:39.085454+0000","last_became_active":"2023-11-18T20:42:38.100591+0000","last_became_peered":"2023-11-18T20:42:38.100591+0000","last_unstale":"2023-11-18T20:42:39.085454+0000","last_undegraded":"2023-11-18T20:42:39.085454+0000","last_fullsized":"2023-11-18T20:42:39.085454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.084262+0000","last_change":"2023-11-18T20:42:37.099362+0000","last_active":"2023-11-18T20:42:39.084262+0000","last_peered":"2023-11-18T20:42:39.084262+0000","last_clean":"2023-11-18T20:42:39.084262+0000","last_became_active":"2023-11-18T20:42:37.099107+0000","last_became_peered":"2023-11-18T20:42:37.099107+0000","last_unstale":"2023-11-18T20:42:39.084262+0000","last_undegraded":"2023-11-18T20:42:39.084262+0000","last_fullsized":"2023-11-18T20:42:39.084262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:01.521083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086177+0000","last_change":"2023-11-18T20:42:38.101488+0000","last_active":"2023-11-18T20:42:39.086177+0000","last_peered":"2023-11-18T20:42:39.086177+0000","last_clean":"2023-11-18T20:42:39.086177+0000","last_became_active":"2023-11-18T20:42:38.101205+0000","last_became_peered":"2023-11-18T20:42:38.101205+0000","last_unstale":"2023-11-18T20:42:39.086177+0000","last_undegraded":"2023-11-18T20:42:39.086177+0000","last_fullsized":"2023-11-18T20:42:39.086177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:44:52.372438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086311+0000","last_change":"2023-11-18T20:42:38.106746+0000","last_active":"2023-11-18T20:42:39.086311+0000","last_peered":"2023-11-18T20:42:39.086311+0000","last_clean":"2023-11-18T20:42:39.086311+0000","last_became_active":"2023-11-18T20:42:38.101774+0000","last_became_peered":"2023-11-18T20:42:38.101774+0000","last_unstale":"2023-11-18T20:42:39.086311+0000","last_undegraded":"2023-11-18T20:42:39.086311+0000","last_fullsized":"2023-11-18T20:42:39.086311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:45.408782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083126+0000","last_change":"2023-11-18T20:42:37.110955+0000","last_active":"2023-11-18T20:42:39.083126+0000","last_peered":"2023-11-18T20:42:39.083126+0000","last_clean":"2023-11-18T20:42:39.083126+0000","last_became_active":"2023-11-18T20:42:37.110681+0000","last_became_peered":"2023-11-18T20:42:37.110681+0000","last_unstale":"2023-11-18T20:42:39.083126+0000","last_undegraded":"2023-11-18T20:42:39.083126+0000","last_fullsized":"2023-11-18T20:42:39.083126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:28:12.116637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.086444+0000","last_change":"2023-11-18T20:42:38.105146+0000","last_active":"2023-11-18T20:42:39.086444+0000","last_peered":"2023-11-18T20:42:39.086444+0000","last_clean":"2023-11-18T20:42:39.086444+0000","last_became_active":"2023-11-18T20:42:38.101793+0000","last_became_peered":"2023-11-18T20:42:38.101793+0000","last_unstale":"2023-11-18T20:42:39.086444+0000","last_undegraded":"2023-11-18T20:42:39.086444+0000","last_fullsized":"2023-11-18T20:42:39.086444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:45:28.196671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.083739+0000","last_change":"2023-11-18T20:42:37.102836+0000","last_active":"2023-11-18T20:42:39.083739+0000","last_peered":"2023-11-18T20:42:39.083739+0000","last_clean":"2023-11-18T20:42:39.083739+0000","last_became_active":"2023-11-18T20:42:37.102550+0000","last_became_peered":"2023-11-18T20:42:37.102550+0000","last_unstale":"2023-11-18T20:42:39.083739+0000","last_undegraded":"2023-11-18T20:42:39.083739+0000","last_fullsized":"2023-11-18T20:42:39.083739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:03:18.160598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085598+0000","last_change":"2023-11-18T20:42:38.102190+0000","last_active":"2023-11-18T20:42:39.085598+0000","last_peered":"2023-11-18T20:42:39.085598+0000","last_clean":"2023-11-18T20:42:39.085598+0000","last_became_active":"2023-11-18T20:42:38.101883+0000","last_became_peered":"2023-11-18T20:42:38.101883+0000","last_unstale":"2023-11-18T20:42:39.085598+0000","last_undegraded":"2023-11-18T20:42:39.085598+0000","last_fullsized":"2023-11-18T20:42:39.085598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:14.531419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085117+0000","last_change":"2023-11-18T20:42:37.108374+0000","last_active":"2023-11-18T20:42:39.085117+0000","last_peered":"2023-11-18T20:42:39.085117+0000","last_clean":"2023-11-18T20:42:39.085117+0000","last_became_active":"2023-11-18T20:42:37.108079+0000","last_became_peered":"2023-11-18T20:42:37.108079+0000","last_unstale":"2023-11-18T20:42:39.085117+0000","last_undegraded":"2023-11-18T20:42:39.085117+0000","last_fullsized":"2023-11-18T20:42:39.085117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:03:25.631982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085115+0000","last_change":"2023-11-18T20:42:38.095628+0000","last_active":"2023-11-18T20:42:39.085115+0000","last_peered":"2023-11-18T20:42:39.085115+0000","last_clean":"2023-11-18T20:42:39.085115+0000","last_became_active":"2023-11-18T20:42:38.095315+0000","last_became_peered":"2023-11-18T20:42:38.095315+0000","last_unstale":"2023-11-18T20:42:39.085115+0000","last_undegraded":"2023-11-18T20:42:39.085115+0000","last_fullsized":"2023-11-18T20:42:39.085115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:37.085490+0000","last_clean_scrub_stamp":"2023-11-18T20:42:37.085490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:02:16.431843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-18T20:42:39.085811+0000","last_change":"2023-11-18T20:42:37.101566+0000","last_active":"2023-11-18T20:42:39.085811+0000","last_peered":"2023-11-18T20:42:39.085811+0000","last_clean":"2023-11-18T20:42:39.085811+0000","last_became_active":"2023-11-18T20:42:37.101286+0000","last_became_peered":"2023-11-18T20:42:37.101286+0000","last_unstale":"2023-11-18T20:42:39.085811+0000","last_undegraded":"2023-11-18T20:42:39.085811+0000","last_fullsized":"2023-11-18T20:42:39.085811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:36.080881+0000","last_clean_scrub_stamp":"2023-11-18T20:42:36.080881+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:14:29.503121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":819200,"data_stored":665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:42:45.907 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-18T20:42:45.908 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-18T20:42:45.908 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-18T20:42:45.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-18T20:42:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.048+0000 7fbd69789640 1 Processor -- start 2023-11-18T20:42:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.048+0000 7fbd69789640 1 -- start start 2023-11-18T20:42:46.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd69789640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 0x7fbd6415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd69789640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd6415e730 0x7fbd6415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd69789640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6415e2a0 con 0x7fbd64143490 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd69789640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6415b520 con 0x7fbd6415e730 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd69789640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd64143870 con 0x7fbd640b1400 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd637fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd6415e730 0x7fbd6415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd62ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbd64143490 0x7fbd640b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39228/0 (socket says 172.21.15.60:39228) 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd62ffd640 1 -- 172.21.15.60:0/2445627714 learned_addr learned my addr 172.21.15.60:0/2445627714 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd627fc640 1 --2- 172.21.15.60:0/2445627714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 0x7fbd6415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd637fe640 1 -- 172.21.15.60:0/2445627714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 msgr2=0x7fbd6415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd637fe640 1 --2- 172.21.15.60:0/2445627714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 0x7fbd6415a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd637fe640 1 -- 172.21.15.60:0/2445627714 >> v1:172.21.15.60:6789/0 conn(0x7fbd64143490 legacy=0x7fbd640b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd61ffb640 1 -- 172.21.15.60:0/2445627714 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672074147 0 0) 0x7fbd6415e2a0 con 0x7fbd64143490 2023-11-18T20:42:46.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd637fe640 1 -- 172.21.15.60:0/2445627714 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6415b240 con 0x7fbd6415e730 2023-11-18T20:42:46.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.049+0000 7fbd627fc640 1 --2- 172.21.15.60:0/2445627714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 0x7fbd6415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:46.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.050+0000 7fbd637fe640 1 --2- 172.21.15.60:0/2445627714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6415e730 0x7fbd6415aa20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009d90 tx=0x7fbd5402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcd9ae2ffb4c30a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:46.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.050+0000 7fbd61ffb640 1 -- 172.21.15.60:0/2445627714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5402cc50 con 0x7fbd6415e730 2023-11-18T20:42:46.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.050+0000 7fbd61ffb640 1 -- 172.21.15.60:0/2445627714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5402cdb0 con 0x7fbd6415e730 2023-11-18T20:42:46.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.050+0000 7fbd61ffb640 1 -- 172.21.15.60:0/2445627714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54034660 con 0x7fbd6415e730 2023-11-18T20:42:46.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.051+0000 7fbd69789640 1 -- 172.21.15.60:0/2445627714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6415e730 msgr2=0x7fbd6415aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.051+0000 7fbd69789640 1 --2- 172.21.15.60:0/2445627714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6415e730 0x7fbd6415aa20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009d90 tx=0x7fbd5402bea0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.051+0000 7fbd69789640 1 -- 172.21.15.60:0/2445627714 shutdown_connections 2023-11-18T20:42:46.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.051+0000 7fbd69789640 1 --2- 172.21.15.60:0/2445627714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6415e730 0x7fbd6415aa20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.051+0000 7fbd69789640 1 --2- 172.21.15.60:0/2445627714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 0x7fbd6415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.051+0000 7fbd69789640 1 -- 172.21.15.60:0/2445627714 >> 172.21.15.60:0/2445627714 conn(0x7fbd6406a4d0 msgr2=0x7fbd6406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:46.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.051+0000 7fbd69789640 1 -- 172.21.15.60:0/2445627714 shutdown_connections 2023-11-18T20:42:46.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.051+0000 7fbd69789640 1 -- 172.21.15.60:0/2445627714 wait complete. 2023-11-18T20:42:46.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.052+0000 7fbd69789640 1 Processor -- start 2023-11-18T20:42:46.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.052+0000 7fbd69789640 1 -- start start 2023-11-18T20:42:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd69789640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 0x7fbd64140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd69789640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd64143490 0x7fbd64071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd69789640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6415e730 0x7fbd6413c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd69789640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641406c0 con 0x7fbd64143490 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd69789640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6413d110 con 0x7fbd6415e730 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd69789640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6413ce30 con 0x7fbd640b1400 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd627fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd64143490 0x7fbd64071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd627fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd64143490 0x7fbd64071680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41464/0 (socket says 172.21.15.60:41464) 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd627fc640 1 -- 172.21.15.60:0/3209229692 learned_addr learned my addr 172.21.15.60:0/3209229692 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd637fe640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6415e730 0x7fbd6413c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd62ffd640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 0x7fbd64140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd627fc640 1 -- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 msgr2=0x7fbd64140ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd627fc640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 0x7fbd64140ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd627fc640 1 -- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6415e730 msgr2=0x7fbd6413c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd627fc640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6415e730 0x7fbd6413c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd627fc640 1 -- 172.21.15.60:0/3209229692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd54009a70 con 0x7fbd64143490 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd637fe640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6415e730 0x7fbd6413c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd62ffd640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 0x7fbd64140ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.053+0000 7fbd627fc640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd64143490 0x7fbd64071680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c7f0 tx=0x7fbd4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.054+0000 7fbd5b7fe640 1 -- 172.21.15.60:0/3209229692 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c007a30 con 0x7fbd64143490 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.054+0000 7fbd5b7fe640 1 -- 172.21.15.60:0/3209229692 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c007b90 con 0x7fbd64143490 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.054+0000 7fbd69789640 1 -- 172.21.15.60:0/3209229692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd64140000 con 0x7fbd64143490 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.054+0000 7fbd5b7fe640 1 -- 172.21.15.60:0/3209229692 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c00ce40 con 0x7fbd64143490 2023-11-18T20:42:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.054+0000 7fbd69789640 1 -- 172.21.15.60:0/3209229692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd641426e0 con 0x7fbd64143490 2023-11-18T20:42:46.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.055+0000 7fbd69789640 1 -- 172.21.15.60:0/3209229692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd64140800 con 0x7fbd64143490 2023-11-18T20:42:46.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.055+0000 7fbd5b7fe640 1 -- 172.21.15.60:0/3209229692 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd4c0113f0 con 0x7fbd64143490 2023-11-18T20:42:46.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.055+0000 7fbd5b7fe640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd4406ae40 0x7fbd4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.056+0000 7fbd5b7fe640 1 -- 172.21.15.60:0/3209229692 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd4c0897b0 con 0x7fbd64143490 2023-11-18T20:42:46.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.056+0000 7fbd62ffd640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd4406ae40 0x7fbd4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.056+0000 7fbd62ffd640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd4406ae40 0x7fbd4406d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c000f60 tx=0x7fbd5c007c10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.060+0000 7fbd5b7fe640 1 -- 172.21.15.60:0/3209229692 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd64140800 con 0x7fbd64143490 2023-11-18T20:42:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.275+0000 7fbd69789640 1 -- 172.21.15.60:0/3209229692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbd64140800 con 0x7fbd64143490 2023-11-18T20:42:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.275+0000 7fbd5b7fe640 1 -- 172.21.15.60:0/3209229692 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbd64140800 con 0x7fbd64143490 2023-11-18T20:42:46.277 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:46.277 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-18T20:42:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 -- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd4406ae40 msgr2=0x7fbd4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd4406ae40 0x7fbd4406d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c000f60 tx=0x7fbd5c007c10 comp rx=0 tx=0).stop 2023-11-18T20:42:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 -- 172.21.15.60:0/3209229692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd64143490 msgr2=0x7fbd64071680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd64143490 0x7fbd64071680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c7f0 tx=0x7fbd4c00cbf0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 -- 172.21.15.60:0/3209229692 shutdown_connections 2023-11-18T20:42:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd4406ae40 0x7fbd4406d300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6415e730 0x7fbd6413c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd64143490 0x7fbd64071680 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 --2- 172.21.15.60:0/3209229692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd640b1400 0x7fbd64140ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 -- 172.21.15.60:0/3209229692 >> 172.21.15.60:0/3209229692 conn(0x7fbd6406a4d0 msgr2=0x7fbd640af080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 -- 172.21.15.60:0/3209229692 shutdown_connections 2023-11-18T20:42:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.277+0000 7fbd69789640 1 -- 172.21.15.60:0/3209229692 wait complete. 2023-11-18T20:42:46.289 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-18T20:42:46.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:46.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.395+0000 7f7c05bfb640 1 Processor -- start 2023-11-18T20:42:46.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.395+0000 7f7c05bfb640 1 -- start start 2023-11-18T20:42:46.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7c05bfb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 0x7f7c000b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7c05bfb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7c000d2040 0x7f7c000b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7c05bfb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7c000cda40 0x7f7c000cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7c05bfb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c000d1bb0 con 0x7f7c000cda40 2023-11-18T20:42:46.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7c05bfb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0015d6b0 con 0x7f7c000d2040 2023-11-18T20:42:46.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7c05bfb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000ce700 con 0x7f7c0015d2d0 2023-11-18T20:42:46.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bfffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7c000cda40 0x7f7c000cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bfffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7c000cda40 0x7f7c000cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41474/0 (socket says 172.21.15.60:41474) 2023-11-18T20:42:46.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bfffff640 1 -- 172.21.15.60:0/1812329568 learned_addr learned my addr 172.21.15.60:0/1812329568 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:46.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bff7fe640 1 --2- 172.21.15.60:0/1812329568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 0x7f7c000b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bfeffd640 1 --2- 172.21.15.60:0/1812329568 >> v2:172.21.15.100:3300/0 conn(0x7f7c000d2040 0x7f7c000b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bfffff640 1 -- 172.21.15.60:0/1812329568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 msgr2=0x7f7c000b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:46.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bfffff640 1 --2- 172.21.15.60:0/1812329568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 0x7f7c000b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bfffff640 1 -- 172.21.15.60:0/1812329568 >> v2:172.21.15.100:3300/0 conn(0x7f7c000d2040 msgr2=0x7f7c000b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:46.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bfffff640 1 --2- 172.21.15.60:0/1812329568 >> v2:172.21.15.100:3300/0 conn(0x7f7c000d2040 0x7f7c000b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bfffff640 1 -- 172.21.15.60:0/1812329568 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c000d1280 con 0x7f7c000cda40 2023-11-18T20:42:46.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.396+0000 7f7bfeffd640 1 --2- 172.21.15.60:0/1812329568 >> v2:172.21.15.100:3300/0 conn(0x7f7c000d2040 0x7f7c000b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:46.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.397+0000 7f7bfffff640 1 --2- 172.21.15.60:0/1812329568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 0x7f7c000cded0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7c0006c600 tx=0x7f7bf802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff6e4a41460b4ae server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:46.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.397+0000 7f7bfe7fc640 1 -- 172.21.15.60:0/1812329568 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf802ca70 con 0x7f7c000cda40 2023-11-18T20:42:46.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.397+0000 7f7bfe7fc640 1 -- 172.21.15.60:0/1812329568 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf802cbd0 con 0x7f7c000cda40 2023-11-18T20:42:46.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.397+0000 7f7bfe7fc640 1 -- 172.21.15.60:0/1812329568 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80343f0 con 0x7f7c000cda40 2023-11-18T20:42:46.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.397+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1812329568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 msgr2=0x7f7c000cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.397+0000 7f7c05bfb640 1 --2- 172.21.15.60:0/1812329568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 0x7f7c000cded0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7c0006c600 tx=0x7f7bf802bed0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.398+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1812329568 shutdown_connections 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.398+0000 7f7c05bfb640 1 --2- 172.21.15.60:0/1812329568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 0x7f7c000cded0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.398+0000 7f7c05bfb640 1 --2- 172.21.15.60:0/1812329568 >> v2:172.21.15.100:3300/0 conn(0x7f7c000d2040 0x7f7c000b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.398+0000 7f7c05bfb640 1 --2- 172.21.15.60:0/1812329568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 0x7f7c000b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.398+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1812329568 >> 172.21.15.60:0/1812329568 conn(0x7f7c00072050 msgr2=0x7f7c00072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.398+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1812329568 shutdown_connections 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.398+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1812329568 wait complete. 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.399+0000 7f7c05bfb640 1 Processor -- start 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.399+0000 7f7c05bfb640 1 -- start start 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7c05bfb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 0x7f7c000ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7c05bfb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000d2040 0x7f7c00152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7c05bfb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 0x7f7c000cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7c05bfb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000cd330 con 0x7f7c000cda40 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7c05bfb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000c8750 con 0x7f7c000d2040 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7c05bfb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c000ca890 con 0x7f7c0015d2d0 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 0x7f7c000ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 0x7f7c000ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41480/0 (socket says 172.21.15.60:41480) 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bff7fe640 1 -- 172.21.15.60:0/1901402247 learned_addr learned my addr 172.21.15.60:0/1901402247 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bfffff640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 0x7f7c000cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bfeffd640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000d2040 0x7f7c00152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bff7fe640 1 -- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 msgr2=0x7f7c000cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bff7fe640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 0x7f7c000cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bff7fe640 1 -- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000d2040 msgr2=0x7f7c00152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bff7fe640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000d2040 0x7f7c00152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bff7fe640 1 -- 172.21.15.60:0/1901402247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf8009b30 con 0x7f7c000cda40 2023-11-18T20:42:46.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bfffff640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 0x7f7c000cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:46.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7bff7fe640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 0x7f7c000ca2b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000e870 tx=0x7f7bf000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:46.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7be7fff640 1 -- 172.21.15.60:0/1901402247 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00094f0 con 0x7f7c000cda40 2023-11-18T20:42:46.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7be7fff640 1 -- 172.21.15.60:0/1901402247 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf000c940 con 0x7f7c000cda40 2023-11-18T20:42:46.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.400+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1901402247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c000c8a10 con 0x7f7c000cda40 2023-11-18T20:42:46.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.401+0000 7f7be7fff640 1 -- 172.21.15.60:0/1901402247 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf000ccf0 con 0x7f7c000cda40 2023-11-18T20:42:46.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.401+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1901402247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c000c8fd0 con 0x7f7c000cda40 2023-11-18T20:42:46.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.402+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1901402247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c00069400 con 0x7f7c000cda40 2023-11-18T20:42:46.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.402+0000 7f7be7fff640 1 -- 172.21.15.60:0/1901402247 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bf0013470 con 0x7f7c000cda40 2023-11-18T20:42:46.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.402+0000 7f7be7fff640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7bec06ae40 0x7f7bec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.402+0000 7f7be7fff640 1 -- 172.21.15.60:0/1901402247 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bf00897c0 con 0x7f7c000cda40 2023-11-18T20:42:46.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.403+0000 7f7bfeffd640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7bec06ae40 0x7f7bec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.403+0000 7f7bfeffd640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7bec06ae40 0x7f7bec06d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4006fd0 tx=0x7f7bf4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:46.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.407+0000 7f7be7fff640 1 -- 172.21.15.60:0/1901402247 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bf00547e0 con 0x7f7c000cda40 2023-11-18T20:42:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.584+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1901402247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c00067720 con 0x7f7c000cda40 2023-11-18T20:42:46.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.585+0000 7f7be7fff640 1 -- 172.21.15.60:0/1901402247 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f7bf0021090 con 0x7f7c000cda40 2023-11-18T20:42:46.586 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:46.586 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7bec06ae40 msgr2=0x7f7bec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7bec06ae40 0x7f7bec06d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4006fd0 tx=0x7f7bf4008040 comp rx=0 tx=0).stop 2023-11-18T20:42:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1901402247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 msgr2=0x7f7c000ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 0x7f7c000ca2b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000e870 tx=0x7f7bf000ed30 comp rx=0 tx=0).stop 2023-11-18T20:42:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1901402247 shutdown_connections 2023-11-18T20:42:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7bec06ae40 0x7f7bec06d300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c0015d2d0 0x7f7c000cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c000d2040 0x7f7c00152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 --2- 172.21.15.60:0/1901402247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c000cda40 0x7f7c000ca2b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1901402247 >> 172.21.15.60:0/1901402247 conn(0x7f7c00072050 msgr2=0x7f7c000b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1901402247 shutdown_connections 2023-11-18T20:42:46.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.587+0000 7f7c05bfb640 1 -- 172.21.15.60:0/1901402247 wait complete. 2023-11-18T20:42:46.599 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:42:46.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:42:46.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.707+0000 7fa29770f640 1 Processor -- start 2023-11-18T20:42:46.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.707+0000 7fa29770f640 1 -- start start 2023-11-18T20:42:46.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa29770f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa290140310 0x7fa2901618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa29770f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2901409d0 0x7fa29006a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa29770f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2901428d0 0x7fa29006aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa29770f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa290161e20 con 0x7fa2901428d0 2023-11-18T20:42:46.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa29770f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2901406f0 con 0x7fa2901409d0 2023-11-18T20:42:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa29770f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa290143960 con 0x7fa290140310 2023-11-18T20:42:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa295cac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2901428d0 0x7fa29006aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa295cac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2901428d0 0x7fa29006aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41492/0 (socket says 172.21.15.60:41492) 2023-11-18T20:42:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa295cac640 1 -- 172.21.15.60:0/562957878 learned_addr learned my addr 172.21.15.60:0/562957878 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa2954ab640 1 --2- 172.21.15.60:0/562957878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa290140310 0x7fa2901618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa294caa640 1 --2- 172.21.15.60:0/562957878 >> v2:172.21.15.100:3300/0 conn(0x7fa2901409d0 0x7fa29006a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa295cac640 1 -- 172.21.15.60:0/562957878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa290140310 msgr2=0x7fa2901618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:46.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa295cac640 1 --2- 172.21.15.60:0/562957878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa290140310 0x7fa2901618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa295cac640 1 -- 172.21.15.60:0/562957878 >> v2:172.21.15.100:3300/0 conn(0x7fa2901409d0 msgr2=0x7fa29006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:46.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa295cac640 1 --2- 172.21.15.60:0/562957878 >> v2:172.21.15.100:3300/0 conn(0x7fa2901409d0 0x7fa29006a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.708+0000 7fa295cac640 1 -- 172.21.15.60:0/562957878 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa290142de0 con 0x7fa2901428d0 2023-11-18T20:42:46.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.709+0000 7fa295cac640 1 --2- 172.21.15.60:0/562957878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 0x7fa29006aa10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa29006e630 tx=0x7fa28802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2051987718c2967 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:46.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.709+0000 7fa287fff640 1 -- 172.21.15.60:0/562957878 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28802c9e0 con 0x7fa2901428d0 2023-11-18T20:42:46.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.709+0000 7fa287fff640 1 -- 172.21.15.60:0/562957878 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28802cb40 con 0x7fa2901428d0 2023-11-18T20:42:46.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.709+0000 7fa287fff640 1 -- 172.21.15.60:0/562957878 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2880343f0 con 0x7fa2901428d0 2023-11-18T20:42:46.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.710+0000 7fa29770f640 1 -- 172.21.15.60:0/562957878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 msgr2=0x7fa29006aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.710+0000 7fa29770f640 1 --2- 172.21.15.60:0/562957878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 0x7fa29006aa10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa29006e630 tx=0x7fa28802be40 comp rx=0 tx=0).stop 2023-11-18T20:42:46.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.710+0000 7fa29770f640 1 -- 172.21.15.60:0/562957878 shutdown_connections 2023-11-18T20:42:46.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.710+0000 7fa29770f640 1 --2- 172.21.15.60:0/562957878 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 0x7fa29006aa10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.710+0000 7fa29770f640 1 --2- 172.21.15.60:0/562957878 >> v2:172.21.15.100:3300/0 conn(0x7fa2901409d0 0x7fa29006a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.710+0000 7fa29770f640 1 --2- 172.21.15.60:0/562957878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa290140310 0x7fa2901618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.710+0000 7fa29770f640 1 -- 172.21.15.60:0/562957878 >> 172.21.15.60:0/562957878 conn(0x7fa2900b1e50 msgr2=0x7fa2900b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:46.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.710+0000 7fa29770f640 1 -- 172.21.15.60:0/562957878 shutdown_connections 2023-11-18T20:42:46.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.710+0000 7fa29770f640 1 -- 172.21.15.60:0/562957878 wait complete. 2023-11-18T20:42:46.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.711+0000 7fa29770f640 1 Processor -- start 2023-11-18T20:42:46.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.711+0000 7fa29770f640 1 -- start start 2023-11-18T20:42:46.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa29770f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa290140310 0x7fa290131390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa29770f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2901409d0 0x7fa2900714a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa29770f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 0x7fa2900719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa29770f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29012f860 con 0x7fa2901428d0 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa29770f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa290132310 con 0x7fa290140310 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa29770f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa290131d50 con 0x7fa2901409d0 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa295cac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 0x7fa2900719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa295cac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 0x7fa2900719e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41504/0 (socket says 172.21.15.60:41504) 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa295cac640 1 -- 172.21.15.60:0/3535986148 learned_addr learned my addr 172.21.15.60:0/3535986148 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa2954ab640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa290140310 0x7fa290131390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa294caa640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2901409d0 0x7fa2900714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa295cac640 1 -- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2901409d0 msgr2=0x7fa2900714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa295cac640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2901409d0 0x7fa2900714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa295cac640 1 -- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa290140310 msgr2=0x7fa290131390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa295cac640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa290140310 0x7fa290131390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa295cac640 1 -- 172.21.15.60:0/3535986148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa288009b30 con 0x7fa2901428d0 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa2954ab640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa290140310 0x7fa290131390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa294caa640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2901409d0 0x7fa2900714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa295cac640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 0x7fa2900719e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa288009f40 tx=0x7fa288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa285ffb640 1 -- 172.21.15.60:0/3535986148 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa288009510 con 0x7fa2901428d0 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa285ffb640 1 -- 172.21.15.60:0/3535986148 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28803e450 con 0x7fa2901428d0 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa29770f640 1 -- 172.21.15.60:0/3535986148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa290132030 con 0x7fa2901428d0 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.712+0000 7fa285ffb640 1 -- 172.21.15.60:0/3535986148 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28803e800 con 0x7fa2901428d0 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.713+0000 7fa29770f640 1 -- 172.21.15.60:0/3535986148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2901318d0 con 0x7fa2901428d0 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.714+0000 7fa29770f640 1 -- 172.21.15.60:0/3535986148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29012f9a0 con 0x7fa2901428d0 2023-11-18T20:42:46.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.714+0000 7fa285ffb640 1 -- 172.21.15.60:0/3535986148 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa28803e960 con 0x7fa2901428d0 2023-11-18T20:42:46.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.714+0000 7fa285ffb640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa27406ae40 0x7fa27406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:46.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.715+0000 7fa285ffb640 1 -- 172.21.15.60:0/3535986148 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2880ac1a0 con 0x7fa2901428d0 2023-11-18T20:42:46.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.715+0000 7fa2954ab640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa27406ae40 0x7fa27406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:46.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.715+0000 7fa2954ab640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa27406ae40 0x7fa27406d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0099a0 tx=0x7fa28c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:46.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.719+0000 7fa285ffb640 1 -- 172.21.15.60:0/3535986148 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa29012f9a0 con 0x7fa2901428d0 2023-11-18T20:42:46.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.892+0000 7fa29770f640 1 -- 172.21.15.60:0/3535986148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa29012f9a0 con 0x7fa2901428d0 2023-11-18T20:42:46.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.892+0000 7fa285ffb640 1 -- 172.21.15.60:0/3535986148 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa29012f9a0 con 0x7fa2901428d0 2023-11-18T20:42:46.894 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:42:46.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 -- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa27406ae40 msgr2=0x7fa27406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa27406ae40 0x7fa27406d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0099a0 tx=0x7fa28c008040 comp rx=0 tx=0).stop 2023-11-18T20:42:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 -- 172.21.15.60:0/3535986148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 msgr2=0x7fa2900719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 0x7fa2900719e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa288009f40 tx=0x7fa288004a40 comp rx=0 tx=0).stop 2023-11-18T20:42:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 -- 172.21.15.60:0/3535986148 shutdown_connections 2023-11-18T20:42:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa27406ae40 0x7fa27406d300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2901428d0 0x7fa2900719e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2901409d0 0x7fa2900714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 --2- 172.21.15.60:0/3535986148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa290140310 0x7fa290131390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:46.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 -- 172.21.15.60:0/3535986148 >> 172.21.15.60:0/3535986148 conn(0x7fa2900b1e50 msgr2=0x7fa29015dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:46.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 -- 172.21.15.60:0/3535986148 shutdown_connections 2023-11-18T20:42:46.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:46.894+0000 7fa29770f640 1 -- 172.21.15.60:0/3535986148 wait complete. 2023-11-18T20:42:46.907 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:42:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.015+0000 7f5f5437c640 1 Processor -- start 2023-11-18T20:42:47.016 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.015+0000 7f5f5437c640 1 -- start start 2023-11-18T20:42:47.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f5437c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f4c15d1d0 0x7f5f4c06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f5437c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f4c0ad640 0x7f5f4c06ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f5437c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c0adae0 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f5437c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c0addc0 con 0x7f5f4c15d1d0 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f5437c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c0ae0a0 con 0x7f5f4c0ad640 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f52118640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f4c15d1d0 0x7f5f4c06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f52118640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f4c15d1d0 0x7f5f4c06b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41516/0 (socket says 172.21.15.60:41516) 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f52118640 1 -- 172.21.15.60:0/909134157 learned_addr learned my addr 172.21.15.60:0/909134157 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f51917640 1 --2- 172.21.15.60:0/909134157 >> v2:172.21.15.100:3300/0 conn(0x7f5f4c0ad640 0x7f5f4c06ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f52118640 1 -- 172.21.15.60:0/909134157 >> v2:172.21.15.100:3300/0 conn(0x7f5f4c0ad640 msgr2=0x7f5f4c06ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f52118640 1 --2- 172.21.15.60:0/909134157 >> v2:172.21.15.100:3300/0 conn(0x7f5f4c0ad640 0x7f5f4c06ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f52118640 1 -- 172.21.15.60:0/909134157 >> v1:172.21.15.100:6789/0 conn(0x7f5f4c0aeb80 legacy=0x7f5f4c160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.016+0000 7f5f52118640 1 -- 172.21.15.60:0/909134157 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0ae380 con 0x7f5f4c15d1d0 2023-11-18T20:42:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.017+0000 7f5f52118640 1 --2- 172.21.15.60:0/909134157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c15d1d0 0x7f5f4c06b4e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0cc440 tx=0x7f5f4802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27887599419ff513 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.017+0000 7f5f51116640 1 -- 172.21.15.60:0/909134157 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4802cb20 con 0x7f5f4c15d1d0 2023-11-18T20:42:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.017+0000 7f5f51116640 1 -- 172.21.15.60:0/909134157 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4802cc80 con 0x7f5f4c15d1d0 2023-11-18T20:42:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.017+0000 7f5f51116640 1 -- 172.21.15.60:0/909134157 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480343f0 con 0x7f5f4c15d1d0 2023-11-18T20:42:47.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.018+0000 7f5f5437c640 1 -- 172.21.15.60:0/909134157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c15d1d0 msgr2=0x7f5f4c06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.018+0000 7f5f5437c640 1 --2- 172.21.15.60:0/909134157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c15d1d0 0x7f5f4c06b4e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0cc440 tx=0x7f5f4802bf80 comp rx=0 tx=0).stop 2023-11-18T20:42:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.018+0000 7f5f5437c640 1 -- 172.21.15.60:0/909134157 shutdown_connections 2023-11-18T20:42:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.018+0000 7f5f5437c640 1 --2- 172.21.15.60:0/909134157 >> v2:172.21.15.100:3300/0 conn(0x7f5f4c0ad640 0x7f5f4c06ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.018+0000 7f5f5437c640 1 --2- 172.21.15.60:0/909134157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c15d1d0 0x7f5f4c06b4e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.018+0000 7f5f5437c640 1 -- 172.21.15.60:0/909134157 >> 172.21.15.60:0/909134157 conn(0x7f5f4c070e70 msgr2=0x7f5f4c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.018+0000 7f5f5437c640 1 -- 172.21.15.60:0/909134157 shutdown_connections 2023-11-18T20:42:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.018+0000 7f5f5437c640 1 -- 172.21.15.60:0/909134157 wait complete. 2023-11-18T20:42:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.019+0000 7f5f5437c640 1 Processor -- start 2023-11-18T20:42:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.019+0000 7f5f5437c640 1 -- start start 2023-11-18T20:42:47.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f5437c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f4c0ad640 0x7f5f4c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f5437c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0aeb80 0x7f5f4c14fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f5437c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f4c15d1d0 0x7f5f4c0cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f5437c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c159c60 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f5437c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1585f0 con 0x7f5f4c0ad640 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f5437c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0cf340 con 0x7f5f4c15d1d0 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f51917640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0aeb80 0x7f5f4c14fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f51917640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0aeb80 0x7f5f4c14fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41524/0 (socket says 172.21.15.60:41524) 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f51917640 1 -- 172.21.15.60:0/2233426834 learned_addr learned my addr 172.21.15.60:0/2233426834 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f52919640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f4c15d1d0 0x7f5f4c0cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f52118640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f4c0ad640 0x7f5f4c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f51917640 1 -- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f4c15d1d0 msgr2=0x7f5f4c0cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f51917640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f4c15d1d0 0x7f5f4c0cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f51917640 1 -- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f4c0ad640 msgr2=0x7f5f4c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f51917640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f4c0ad640 0x7f5f4c15b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f51917640 1 -- 172.21.15.60:0/2233426834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48009b30 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f52919640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f4c15d1d0 0x7f5f4c0cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f51917640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0aeb80 0x7f5f4c14fdd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00e9e0 tx=0x7f5f3c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f42ffd640 1 -- 172.21.15.60:0/2233426834 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c009670 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f42ffd640 1 -- 172.21.15.60:0/2233426834 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c00c9d0 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f5437c640 1 -- 172.21.15.60:0/2233426834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c15beb0 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.020+0000 7f5f42ffd640 1 -- 172.21.15.60:0/2233426834 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00cd80 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.021+0000 7f5f5437c640 1 -- 172.21.15.60:0/2233426834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c159880 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.022+0000 7f5f5437c640 1 -- 172.21.15.60:0/2233426834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c0cc360 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.022+0000 7f5f42ffd640 1 -- 172.21.15.60:0/2233426834 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f3c013470 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.022+0000 7f5f42ffd640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f3006ae40 0x7f5f3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.022+0000 7f5f42ffd640 1 -- 172.21.15.60:0/2233426834 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f3c089870 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.022+0000 7f5f52118640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f3006ae40 0x7f5f3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.023+0000 7f5f52118640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f3006ae40 0x7f5f3006d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5f48035000 tx=0x7f5f48030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.026+0000 7f5f42ffd640 1 -- 172.21.15.60:0/2233426834 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f3c054920 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.203+0000 7f5f5437c640 1 -- 172.21.15.60:0/2233426834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f4c159da0 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.203+0000 7f5f42ffd640 1 -- 172.21.15.60:0/2233426834 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f3c017020 con 0x7f5f4c0aeb80 2023-11-18T20:42:47.205 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:42:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 -- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f3006ae40 msgr2=0x7f5f3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f3006ae40 0x7f5f3006d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5f48035000 tx=0x7f5f48030000 comp rx=0 tx=0).stop 2023-11-18T20:42:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 -- 172.21.15.60:0/2233426834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0aeb80 msgr2=0x7f5f4c14fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0aeb80 0x7f5f4c14fdd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00e9e0 tx=0x7f5f3c00eea0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 -- 172.21.15.60:0/2233426834 shutdown_connections 2023-11-18T20:42:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f3006ae40 0x7f5f3006d300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f4c15d1d0 0x7f5f4c0cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f4c0aeb80 0x7f5f4c14fdd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 --2- 172.21.15.60:0/2233426834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f4c0ad640 0x7f5f4c15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 -- 172.21.15.60:0/2233426834 >> 172.21.15.60:0/2233426834 conn(0x7f5f4c070e70 msgr2=0x7f5f4c0b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:47.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 -- 172.21.15.60:0/2233426834 shutdown_connections 2023-11-18T20:42:47.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.205+0000 7f5f5437c640 1 -- 172.21.15.60:0/2233426834 wait complete. 2023-11-18T20:42:47.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:42:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.325+0000 7f1454c76640 1 Processor -- start 2023-11-18T20:42:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.325+0000 7f1454c76640 1 -- start start 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f1454c76640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f145015c2f0 0x7f145006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f1454c76640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145015ba40 con 0x7f1450140f10 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f1454c76640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145006f4a0 con 0x7f145006f040 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f1454c76640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145015c6d0 con 0x7f145015c2f0 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144ed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1450140f10 0x7f145013f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39242/0 (socket says 172.21.15.60:39242) 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144ed9d640 1 -- 172.21.15.60:0/3360722078 learned_addr learned my addr 172.21.15.60:0/3360722078 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144e59c640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f145015c2f0 0x7f145006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888549537 0 0) 0x7f145015ba40 con 0x7f1450140f10 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f143c003520 con 0x7f1450140f10 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3854751877 0 0) 0x7f143c003520 con 0x7f1450140f10 2023-11-18T20:42:47.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f145015c2f0 msgr2=0x7f145006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f145015c2f0 0x7f145006eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 >> v1:172.21.15.100:6789/0 conn(0x7f145006f040 legacy=0x7f1450140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14501416e0 con 0x7f1450140f10 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f14480019f0 con 0x7f1450140f10 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 >> v1:172.21.15.60:6789/0 conn(0x7f1450140f10 legacy=0x7f145013f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f143c0039a0 0x7f143c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143c0047c0 0x7f143c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f145015c2f0 0x7f143c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c00ba30 con 0x7f143c0047c0 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c00bba0 con 0x7f143c0039a0 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.326+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c00bd60 con 0x7f145015c2f0 2023-11-18T20:42:47.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.327+0000 7f1445d9b640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143c0047c0 0x7f143c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.327+0000 7f144e59c640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f143c0039a0 0x7f143c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.327+0000 7f144ed9d640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f145015c2f0 0x7f143c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.327+0000 7f1445d9b640 1 -- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f145015c2f0 msgr2=0x7f143c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.327+0000 7f1445d9b640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f145015c2f0 0x7f143c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.327+0000 7f1445d9b640 1 -- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f143c0039a0 msgr2=0x7f143c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.327+0000 7f1445d9b640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f143c0039a0 0x7f143c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.327+0000 7f1445d9b640 1 -- 172.21.15.60:0/3360722078 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c003520 con 0x7f143c0047c0 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f1445d9b640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143c0047c0 0x7f143c004c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1438001720 tx=0x7f143c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ec8844ec1b1972 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380052d0 con 0x7f143c0047c0 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143802e050 con 0x7f143c0047c0 2023-11-18T20:42:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f144dd9b640 1 -- 172.21.15.60:0/3360722078 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14380035f0 con 0x7f143c0047c0 2023-11-18T20:42:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f1454c76640 1 -- 172.21.15.60:0/3360722078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143c0047c0 msgr2=0x7f143c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f1454c76640 1 --2- 172.21.15.60:0/3360722078 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f143c0047c0 0x7f143c004c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1438001720 tx=0x7f143c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f144ed9d640 1 -- 172.21.15.60:0/3360722078 reap_dead start 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f1454c76640 1 -- 172.21.15.60:0/3360722078 shutdown_connections 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f1454c76640 1 -- 172.21.15.60:0/3360722078 >> 172.21.15.60:0/3360722078 conn(0x7f145006a6b0 msgr2=0x7f145006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f1454c76640 1 -- 172.21.15.60:0/3360722078 shutdown_connections 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.328+0000 7f1454c76640 1 -- 172.21.15.60:0/3360722078 wait complete. 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.330+0000 7f1454c76640 1 Processor -- start 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.330+0000 7f1454c76640 1 -- start start 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.330+0000 7f1454c76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f143c0047c0 0x7f1450151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.330+0000 7f1454c76640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14501f3100 0x7f1450153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.330+0000 7f1454c76640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f145015a080 0x7f145015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f1454c76640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1450154d10 con 0x7f145015a080 2023-11-18T20:42:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f1454c76640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145015aa70 con 0x7f143c0047c0 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f1454c76640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1450154090 con 0x7f14501f3100 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f144ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f145015a080 0x7f145015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f144ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f145015a080 0x7f145015a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41532/0 (socket says 172.21.15.60:41532) 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f144ed9d640 1 -- 172.21.15.60:0/474747993 learned_addr learned my addr 172.21.15.60:0/474747993 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f144e59c640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f143c0047c0 0x7f1450151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f1445d9b640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14501f3100 0x7f1450153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f144ed9d640 1 -- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14501f3100 msgr2=0x7f1450153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f144ed9d640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14501f3100 0x7f1450153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f144ed9d640 1 -- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f143c0047c0 msgr2=0x7f1450151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f144ed9d640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f143c0047c0 0x7f1450151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f144ed9d640 1 -- 172.21.15.60:0/474747993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1450154370 con 0x7f145015a080 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f144ed9d640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f145015a080 0x7f145015a530 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f14500703f0 tx=0x7f1448007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f14477fe640 1 -- 172.21.15.60:0/474747993 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144800aad0 con 0x7f145015a080 2023-11-18T20:42:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f14477fe640 1 -- 172.21.15.60:0/474747993 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144800ac50 con 0x7f145015a080 2023-11-18T20:42:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f1454c76640 1 -- 172.21.15.60:0/474747993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1450154650 con 0x7f145015a080 2023-11-18T20:42:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f14477fe640 1 -- 172.21.15.60:0/474747993 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14480133f0 con 0x7f145015a080 2023-11-18T20:42:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.331+0000 7f1454c76640 1 -- 172.21.15.60:0/474747993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14501518b0 con 0x7f145015a080 2023-11-18T20:42:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.332+0000 7f1454c76640 1 -- 172.21.15.60:0/474747993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1450154e50 con 0x7f145015a080 2023-11-18T20:42:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.333+0000 7f14477fe640 1 -- 172.21.15.60:0/474747993 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1448013550 con 0x7f145015a080 2023-11-18T20:42:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.333+0000 7f14477fe640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f143006ae40 0x7f143006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.333+0000 7f14477fe640 1 -- 172.21.15.60:0/474747993 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1448089f00 con 0x7f145015a080 2023-11-18T20:42:47.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.334+0000 7f144e59c640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f143006ae40 0x7f143006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.334+0000 7f144e59c640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f143006ae40 0x7f143006d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1440009a80 tx=0x7f1440008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.338+0000 7f14477fe640 1 -- 172.21.15.60:0/474747993 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1450154e50 con 0x7f145015a080 2023-11-18T20:42:47.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.515+0000 7f1454c76640 1 -- 172.21.15.60:0/474747993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1450154e50 con 0x7f145015a080 2023-11-18T20:42:47.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.515+0000 7f14477fe640 1 -- 172.21.15.60:0/474747993 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1450154e50 con 0x7f145015a080 2023-11-18T20:42:47.518 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:42:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 -- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f143006ae40 msgr2=0x7f143006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f143006ae40 0x7f143006d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1440009a80 tx=0x7f1440008040 comp rx=0 tx=0).stop 2023-11-18T20:42:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 -- 172.21.15.60:0/474747993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f145015a080 msgr2=0x7f145015a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f145015a080 0x7f145015a530 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f14500703f0 tx=0x7f1448007b60 comp rx=0 tx=0).stop 2023-11-18T20:42:47.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 -- 172.21.15.60:0/474747993 shutdown_connections 2023-11-18T20:42:47.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f143006ae40 0x7f143006d300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f145015a080 0x7f145015a530 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14501f3100 0x7f1450153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 --2- 172.21.15.60:0/474747993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f143c0047c0 0x7f1450151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 -- 172.21.15.60:0/474747993 >> 172.21.15.60:0/474747993 conn(0x7f145006a6b0 msgr2=0x7f14500b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:47.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 -- 172.21.15.60:0/474747993 shutdown_connections 2023-11-18T20:42:47.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.517+0000 7f1454c76640 1 -- 172.21.15.60:0/474747993 wait complete. 2023-11-18T20:42:47.530 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:42:47.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.634+0000 7fe4e1a87640 1 Processor -- start 2023-11-18T20:42:47.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.635+0000 7fe4e1a87640 1 -- start start 2023-11-18T20:42:47.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.635+0000 7fe4e1a87640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4dc15d330 0x7fe4dc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.635+0000 7fe4e1a87640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4dc0d2d90 0x7fe4dc0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.635+0000 7fe4e1a87640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4dc0b2b10 con 0x7fe4dc0d2910 2023-11-18T20:42:47.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.635+0000 7fe4e1a87640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4dc0d1590 con 0x7fe4dc0d2d90 2023-11-18T20:42:47.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.635+0000 7fe4e1a87640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc0d1870 con 0x7fe4dc15d330 2023-11-18T20:42:47.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.635+0000 7fe4dbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4dc0d2d90 0x7fe4dc0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.635+0000 7fe4daffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe4dc0d2910 0x7fe4dc0d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39246/0 (socket says 172.21.15.60:39246) 2023-11-18T20:42:47.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.635+0000 7fe4daffd640 1 -- 172.21.15.60:0/514013410 learned_addr learned my addr 172.21.15.60:0/514013410 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:47.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.636+0000 7fe4db7fe640 1 --2- 172.21.15.60:0/514013410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4dc15d330 0x7fe4dc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.636+0000 7fe4dbfff640 1 -- 172.21.15.60:0/514013410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4dc15d330 msgr2=0x7fe4dc0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.636+0000 7fe4dbfff640 1 --2- 172.21.15.60:0/514013410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4dc15d330 0x7fe4dc0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.636+0000 7fe4dbfff640 1 -- 172.21.15.60:0/514013410 >> v1:172.21.15.60:6789/0 conn(0x7fe4dc0d2910 legacy=0x7fe4dc0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.636+0000 7fe4dbfff640 1 -- 172.21.15.60:0/514013410 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc0d1b50 con 0x7fe4dc0d2d90 2023-11-18T20:42:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.636+0000 7fe4da7fc640 1 -- 172.21.15.60:0/514013410 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533829501 0 0) 0x7fe4dc0b2b10 con 0x7fe4dc0d2910 2023-11-18T20:42:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4dbfff640 1 --2- 172.21.15.60:0/514013410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2d90 0x7fe4dc0b6390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009d90 tx=0x7fe4d402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d499ec443868c3de server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4da7fc640 1 -- 172.21.15.60:0/514013410 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d402cbc0 con 0x7fe4dc0d2d90 2023-11-18T20:42:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4da7fc640 1 -- 172.21.15.60:0/514013410 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d402cd20 con 0x7fe4dc0d2d90 2023-11-18T20:42:47.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4da7fc640 1 -- 172.21.15.60:0/514013410 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4034660 con 0x7fe4dc0d2d90 2023-11-18T20:42:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4e1a87640 1 -- 172.21.15.60:0/514013410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2d90 msgr2=0x7fe4dc0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4e1a87640 1 --2- 172.21.15.60:0/514013410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2d90 0x7fe4dc0b6390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009d90 tx=0x7fe4d402be10 comp rx=0 tx=0).stop 2023-11-18T20:42:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4e1a87640 1 -- 172.21.15.60:0/514013410 shutdown_connections 2023-11-18T20:42:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4e1a87640 1 --2- 172.21.15.60:0/514013410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2d90 0x7fe4dc0b6390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4e1a87640 1 --2- 172.21.15.60:0/514013410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4dc15d330 0x7fe4dc0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4e1a87640 1 -- 172.21.15.60:0/514013410 >> 172.21.15.60:0/514013410 conn(0x7fe4dc072020 msgr2=0x7fe4dc072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4e1a87640 1 -- 172.21.15.60:0/514013410 shutdown_connections 2023-11-18T20:42:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.637+0000 7fe4e1a87640 1 -- 172.21.15.60:0/514013410 wait complete. 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.638+0000 7fe4e1a87640 1 Processor -- start 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.639+0000 7fe4e1a87640 1 -- start start 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.639+0000 7fe4e1a87640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2910 0x7fe4dc0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.639+0000 7fe4e1a87640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4dc0d2d90 0x7fe4dc15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.639+0000 7fe4e1a87640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4dc15d330 0x7fe4dc159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.639+0000 7fe4e1a87640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc160730 con 0x7fe4dc0d2910 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.639+0000 7fe4e1a87640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc15e8d0 con 0x7fe4dc15d330 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.639+0000 7fe4e1a87640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc15ebb0 con 0x7fe4dc0d2d90 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.639+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2910 0x7fe4dc0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.639+0000 7fe4db7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2910 0x7fe4dc0cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41552/0 (socket says 172.21.15.60:41552) 2023-11-18T20:42:47.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.639+0000 7fe4db7fe640 1 -- 172.21.15.60:0/3264201770 learned_addr learned my addr 172.21.15.60:0/3264201770 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4daffd640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4dc0d2d90 0x7fe4dc15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4dbfff640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4dc15d330 0x7fe4dc159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4db7fe640 1 -- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4dc0d2d90 msgr2=0x7fe4dc15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4db7fe640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4dc0d2d90 0x7fe4dc15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4db7fe640 1 -- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4dc15d330 msgr2=0x7fe4dc159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4db7fe640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4dc15d330 0x7fe4dc159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4db7fe640 1 -- 172.21.15.60:0/3264201770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d4009a70 con 0x7fe4dc0d2910 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4db7fe640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2910 0x7fe4dc0cc470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00e9b0 tx=0x7fe4cc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4c3fff640 1 -- 172.21.15.60:0/3264201770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc009670 con 0x7fe4dc0d2910 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4c3fff640 1 -- 172.21.15.60:0/3264201770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc00c9a0 con 0x7fe4dc0d2910 2023-11-18T20:42:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4e1a87640 1 -- 172.21.15.60:0/3264201770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc15ee90 con 0x7fe4dc0d2910 2023-11-18T20:42:47.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4c3fff640 1 -- 172.21.15.60:0/3264201770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00cd50 con 0x7fe4dc0d2910 2023-11-18T20:42:47.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.640+0000 7fe4e1a87640 1 -- 172.21.15.60:0/3264201770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc15f450 con 0x7fe4dc0d2910 2023-11-18T20:42:47.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.641+0000 7fe4e1a87640 1 -- 172.21.15.60:0/3264201770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4dc15f610 con 0x7fe4dc0d2910 2023-11-18T20:42:47.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.641+0000 7fe4c3fff640 1 -- 172.21.15.60:0/3264201770 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe4cc013470 con 0x7fe4dc0d2910 2023-11-18T20:42:47.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.642+0000 7fe4c3fff640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe4c806ae40 0x7fe4c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.642+0000 7fe4c3fff640 1 -- 172.21.15.60:0/3264201770 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4cc089860 con 0x7fe4dc0d2910 2023-11-18T20:42:47.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.642+0000 7fe4daffd640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe4c806ae40 0x7fe4c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.643+0000 7fe4daffd640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe4c806ae40 0x7fe4c806d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0c9d90 tx=0x7fe4d0007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.646+0000 7fe4c3fff640 1 -- 172.21.15.60:0/3264201770 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4cc054910 con 0x7fe4dc0d2910 2023-11-18T20:42:47.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.859+0000 7fe4e1a87640 1 -- 172.21.15.60:0/3264201770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4dc069400 con 0x7fe4dc0d2910 2023-11-18T20:42:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.859+0000 7fe4c3fff640 1 -- 172.21.15.60:0/3264201770 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fe4cc017090 con 0x7fe4dc0d2910 2023-11-18T20:42:47.861 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:47.861 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1530496056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1530496056},{"type":"v1","addr":"172.21.15.60:6833","nonce":1530496056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3842389676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3842389676},{"type":"v1","addr":"172.21.15.60:6835","nonce":3842389676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4248907695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4248907695},{"type":"v1","addr":"172.21.15.100:6836","nonce":4248907695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:42:37.459556+0000","modified":"2023-11-18T20:42:39.458016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1851816776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1851816776},{"type":"v1","addr":"172.21.15.100:6837","nonce":1851816776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-18T20:42:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2023-11-18T20:42:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 -- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe4c806ae40 msgr2=0x7fe4c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe4c806ae40 0x7fe4c806d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0c9d90 tx=0x7fe4d0007c00 comp rx=0 tx=0).stop 2023-11-18T20:42:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 -- 172.21.15.60:0/3264201770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2910 msgr2=0x7fe4dc0cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2910 0x7fe4dc0cc470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00e9b0 tx=0x7fe4cc00ee70 comp rx=0 tx=0).stop 2023-11-18T20:42:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 -- 172.21.15.60:0/3264201770 shutdown_connections 2023-11-18T20:42:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe4c806ae40 0x7fe4c806d300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4dc15d330 0x7fe4dc159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4dc0d2d90 0x7fe4dc15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 --2- 172.21.15.60:0/3264201770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4dc0d2910 0x7fe4dc0cc470 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 -- 172.21.15.60:0/3264201770 >> 172.21.15.60:0/3264201770 conn(0x7fe4dc072020 msgr2=0x7fe4dc0b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:47.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 -- 172.21.15.60:0/3264201770 shutdown_connections 2023-11-18T20:42:47.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.861+0000 7fe4e1a87640 1 -- 172.21.15.60:0/3264201770 wait complete. 2023-11-18T20:42:47.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:47.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.978+0000 7f29649fb640 1 Processor -- start 2023-11-18T20:42:47.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.978+0000 7f29649fb640 1 -- start start 2023-11-18T20:42:47.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.979+0000 7f29649fb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f296015d330 0x7f29600b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.979+0000 7f29649fb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29600b2b10 con 0x7f29600d3900 2023-11-18T20:42:47.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.979+0000 7f29649fb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29600d1590 con 0x7f29600d2910 2023-11-18T20:42:47.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.979+0000 7f29649fb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29600d1870 con 0x7f296015d330 2023-11-18T20:42:47.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.979+0000 7f295ed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f29600d3900 0x7f29600b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39258/0 (socket says 172.21.15.60:39258) 2023-11-18T20:42:47.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.979+0000 7f295ed9d640 1 -- 172.21.15.60:0/570012454 learned_addr learned my addr 172.21.15.60:0/570012454 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295e59c640 1 --2- 172.21.15.60:0/570012454 >> v2:172.21.15.100:3300/0 conn(0x7f296015d330 0x7f29600b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477118382 0 0) 0x7f29600b2b10 con 0x7f29600d3900 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f294c003520 con 0x7f29600d3900 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2476469362 0 0) 0x7f294c003520 con 0x7f29600d3900 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 >> v2:172.21.15.100:3300/0 conn(0x7f296015d330 msgr2=0x7f29600b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 --2- 172.21.15.60:0/570012454 >> v2:172.21.15.100:3300/0 conn(0x7f296015d330 0x7f29600b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 >> v1:172.21.15.100:6789/0 conn(0x7f29600d2910 legacy=0x7f29600d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29600d1b50 con 0x7f29600d3900 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f29500019f0 con 0x7f29600d3900 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295e59c640 1 --2- 172.21.15.60:0/570012454 >> v2:172.21.15.100:3300/0 conn(0x7f296015d330 0x7f29600b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 >> v1:172.21.15.60:6789/0 conn(0x7f29600d3900 legacy=0x7f29600b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f294c003970 0x7f294c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f294c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f294c005180 0x7f294c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29600d1870 con 0x7f294c0047c0 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c00c120 con 0x7f294c005180 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c00c2e0 con 0x7f294c003970 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295dd9b640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f294c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295e59c640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f294c003970 0x7f294c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295ed9d640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f294c005180 0x7f294c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295dd9b640 1 -- 172.21.15.60:0/570012454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f294c003970 msgr2=0x7f294c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295dd9b640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f294c003970 0x7f294c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295dd9b640 1 -- 172.21.15.60:0/570012454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f294c005180 msgr2=0x7f294c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:47.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295dd9b640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f294c005180 0x7f294c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295ed9d640 1 -- 172.21.15.60:0/570012454 reap_dead start 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.980+0000 7f295dd9b640 1 -- 172.21.15.60:0/570012454 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c003520 con 0x7f294c0047c0 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.981+0000 7f295dd9b640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f294c004c40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2948001750 tx=0x7f294c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a489a8aa011c751b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.981+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480052d0 con 0x7f294c0047c0 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.981+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294802e050 con 0x7f294c0047c0 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.981+0000 7f295d59a640 1 -- 172.21.15.60:0/570012454 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948003830 con 0x7f294c0047c0 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.982+0000 7f29649fb640 1 -- 172.21.15.60:0/570012454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 msgr2=0x7f294c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.982+0000 7f29649fb640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f294c004c40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2948001750 tx=0x7f294c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.982+0000 7f29649fb640 1 -- 172.21.15.60:0/570012454 shutdown_connections 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.982+0000 7f29649fb640 1 --2- 172.21.15.60:0/570012454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f294c004c40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.982+0000 7f29649fb640 1 -- 172.21.15.60:0/570012454 >> 172.21.15.60:0/570012454 conn(0x7f2960072020 msgr2=0x7f2960072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:47.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.982+0000 7f29649fb640 1 -- 172.21.15.60:0/570012454 shutdown_connections 2023-11-18T20:42:47.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.982+0000 7f29649fb640 1 -- 172.21.15.60:0/570012454 wait complete. 2023-11-18T20:42:47.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.983+0000 7f29649fb640 1 Processor -- start 2023-11-18T20:42:47.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.983+0000 7f29649fb640 1 -- start start 2023-11-18T20:42:47.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f29649fb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f29601460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f29649fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2960156a90 0x7f2960156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f29649fb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2960158b50 0x7f2960158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f29649fb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2960159e80 con 0x7f294c0047c0 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f29649fb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29600c8170 con 0x7f2960156a90 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f29649fb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2960157450 con 0x7f2960158b50 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f295e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f29601460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f295e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f29601460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41572/0 (socket says 172.21.15.60:41572) 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f295e59c640 1 -- 172.21.15.60:0/1488282598 learned_addr learned my addr 172.21.15.60:0/1488282598 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f295e59c640 1 -- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2960158b50 msgr2=0x7f2960158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f295e59c640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2960158b50 0x7f2960158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f295e59c640 1 -- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2960156a90 msgr2=0x7f2960156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f295dd9b640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2960156a90 0x7f2960156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f295e59c640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2960156a90 0x7f2960156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:47.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f295e59c640 1 -- 172.21.15.60:0/1488282598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29601598c0 con 0x7f294c0047c0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.984+0000 7f295e59c640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f29601460a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2958011c50 tx=0x7f295800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.985+0000 7f2956ffd640 1 -- 172.21.15.60:0/1488282598 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295800fd20 con 0x7f294c0047c0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.985+0000 7f2956ffd640 1 -- 172.21.15.60:0/1488282598 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295800fe80 con 0x7f294c0047c0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.985+0000 7f29649fb640 1 -- 172.21.15.60:0/1488282598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2960159ba0 con 0x7f294c0047c0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.985+0000 7f2956ffd640 1 -- 172.21.15.60:0/1488282598 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958004270 con 0x7f294c0047c0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.985+0000 7f29649fb640 1 -- 172.21.15.60:0/1488282598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29601465e0 con 0x7f294c0047c0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.986+0000 7f29649fb640 1 -- 172.21.15.60:0/1488282598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29600c7bf0 con 0x7f294c0047c0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.986+0000 7f2956ffd640 1 -- 172.21.15.60:0/1488282598 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f29580043d0 con 0x7f294c0047c0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.987+0000 7f2956ffd640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f294006ae40 0x7f294006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.987+0000 7f2956ffd640 1 -- 172.21.15.60:0/1488282598 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2958089d20 con 0x7f294c0047c0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.987+0000 7f295dd9b640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f294006ae40 0x7f294006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:47.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.987+0000 7f295dd9b640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f294006ae40 0x7f294006d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2948033000 tx=0x7f2948009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:47.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:47.991+0000 7f2956ffd640 1 -- 172.21.15.60:0/1488282598 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29600c7bf0 con 0x7f294c0047c0 2023-11-18T20:42:48.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.170+0000 7f29649fb640 1 -- 172.21.15.60:0/1488282598 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29600c7bf0 con 0x7f294c0047c0 2023-11-18T20:42:48.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.171+0000 7f2956ffd640 1 -- 172.21.15.60:0/1488282598 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f29600c7bf0 con 0x7f294c0047c0 2023-11-18T20:42:48.173 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:48.173 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:48.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 -- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f294006ae40 msgr2=0x7f294006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f294006ae40 0x7f294006d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2948033000 tx=0x7f2948009030 comp rx=0 tx=0).stop 2023-11-18T20:42:48.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 -- 172.21.15.60:0/1488282598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 msgr2=0x7f29601460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f29601460a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2958011c50 tx=0x7f295800f610 comp rx=0 tx=0).stop 2023-11-18T20:42:48.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 -- 172.21.15.60:0/1488282598 shutdown_connections 2023-11-18T20:42:48.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f294006ae40 0x7f294006d300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2960158b50 0x7f2960158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2960156a90 0x7f2960156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 --2- 172.21.15.60:0/1488282598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f294c0047c0 0x7f29601460a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 -- 172.21.15.60:0/1488282598 >> 172.21.15.60:0/1488282598 conn(0x7f2960072020 msgr2=0x7f296015e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:48.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 -- 172.21.15.60:0/1488282598 shutdown_connections 2023-11-18T20:42:48.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.173+0000 7f29649fb640 1 -- 172.21.15.60:0/1488282598 wait complete. 2023-11-18T20:42:48.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:42:48.185 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:42:48.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.293+0000 7f47f5d0c640 1 Processor -- start 2023-11-18T20:42:48.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.293+0000 7f47f5d0c640 1 -- start start 2023-11-18T20:42:48.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47f5d0c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f015d2d0 0x7f47f00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47f5d0c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f47f0160f70 0x7f47f00cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47f5d0c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f01614e0 con 0x7f47f00ce480 2023-11-18T20:42:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47f5d0c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f01617c0 con 0x7f47f0160f70 2023-11-18T20:42:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47f5d0c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0161aa0 con 0x7f47f015d2d0 2023-11-18T20:42:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47effff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f47f00ce480 0x7f47f006b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39262/0 (socket says 172.21.15.60:39262) 2023-11-18T20:42:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47effff640 1 -- 172.21.15.60:0/1657092962 learned_addr learned my addr 172.21.15.60:0/1657092962 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853854146 0 0) 0x7f47f01614e0 con 0x7f47f00ce480 2023-11-18T20:42:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ef7fe640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f015d2d0 0x7f47f00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d8003520 con 0x7f47f00ce480 2023-11-18T20:42:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47eeffd640 1 --2- 172.21.15.60:0/1657092962 >> v2:172.21.15.100:3300/0 conn(0x7f47f0160f70 0x7f47f00cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1730407908 0 0) 0x7f47d8003520 con 0x7f47f00ce480 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f015d2d0 msgr2=0x7f47f00b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f015d2d0 0x7f47f00b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 >> v2:172.21.15.100:3300/0 conn(0x7f47f0160f70 msgr2=0x7f47f00cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 --2- 172.21.15.60:0/1657092962 >> v2:172.21.15.100:3300/0 conn(0x7f47f0160f70 0x7f47f00cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f0161d80 con 0x7f47f00ce480 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f47e80019f0 con 0x7f47f00ce480 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 >> v1:172.21.15.60:6789/0 conn(0x7f47f00ce480 legacy=0x7f47f006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d8003970 0x7f47d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f015d2d0 0x7f47d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.294+0000 7f47ee7fc640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d8004d00 0x7f47d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f01617c0 con 0x7f47d8004d00 2023-11-18T20:42:48.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0161aa0 con 0x7f47d8003970 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d800b980 con 0x7f47f015d2d0 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47effff640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d8004d00 0x7f47d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47eeffd640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f015d2d0 0x7f47d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47ef7fe640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d8003970 0x7f47d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47effff640 1 -- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f015d2d0 msgr2=0x7f47d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47effff640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f015d2d0 0x7f47d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47effff640 1 -- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d8003970 msgr2=0x7f47d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47effff640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47d8003970 0x7f47d8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.295+0000 7f47effff640 1 -- 172.21.15.60:0/1657092962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d8003520 con 0x7f47d8004d00 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.296+0000 7f47effff640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d8004d00 0x7f47d8005180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f47e800a8d0 tx=0x7f47d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7a8a8a0e53bd23 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.296+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8034630 con 0x7f47d8004d00 2023-11-18T20:42:48.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.296+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e802f070 con 0x7f47d8004d00 2023-11-18T20:42:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.296+0000 7f47ee7fc640 1 -- 172.21.15.60:0/1657092962 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e80349d0 con 0x7f47d8004d00 2023-11-18T20:42:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.296+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1657092962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d8004d00 msgr2=0x7f47d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.296+0000 7f47f5d0c640 1 --2- 172.21.15.60:0/1657092962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47d8004d00 0x7f47d8005180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f47e800a8d0 tx=0x7f47d80057f0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.296+0000 7f47effff640 1 -- 172.21.15.60:0/1657092962 reap_dead start 2023-11-18T20:42:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.296+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1657092962 shutdown_connections 2023-11-18T20:42:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.296+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1657092962 >> 172.21.15.60:0/1657092962 conn(0x7f47f0072230 msgr2=0x7f47f00cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.297+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1657092962 shutdown_connections 2023-11-18T20:42:48.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.297+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1657092962 wait complete. 2023-11-18T20:42:48.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.298+0000 7f47f5d0c640 1 Processor -- start 2023-11-18T20:42:48.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.298+0000 7f47f5d0c640 1 -- start start 2023-11-18T20:42:48.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47f5d0c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f01585a0 0x7f47f015e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47f5d0c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f0158980 0x7f47f015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47f5d0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f01606e0 0x7f47f015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47f5d0c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0160540 con 0x7f47f01606e0 2023-11-18T20:42:48.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47f5d0c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f015ebd0 con 0x7f47f01585a0 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47f5d0c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f015ff30 con 0x7f47f0158980 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f01606e0 0x7f47f015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f01606e0 0x7f47f015b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41588/0 (socket says 172.21.15.60:41588) 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47effff640 1 -- 172.21.15.60:0/1891197468 learned_addr learned my addr 172.21.15.60:0/1891197468 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47eeffd640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f0158980 0x7f47f015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47ef7fe640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f01585a0 0x7f47f015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47effff640 1 -- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f0158980 msgr2=0x7f47f015b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47effff640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f0158980 0x7f47f015b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47effff640 1 -- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f01585a0 msgr2=0x7f47f015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47effff640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f01585a0 0x7f47f015e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47effff640 1 -- 172.21.15.60:0/1891197468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f015ee90 con 0x7f47f01606e0 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47ef7fe640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f01585a0 0x7f47f015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47effff640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f01606e0 0x7f47f015b9a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f47e800a8d0 tx=0x7f47e8001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47d7fff640 1 -- 172.21.15.60:0/1891197468 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8034840 con 0x7f47f01606e0 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47d7fff640 1 -- 172.21.15.60:0/1891197468 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e80349a0 con 0x7f47f01606e0 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1891197468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f015f170 con 0x7f47f01606e0 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47d7fff640 1 -- 172.21.15.60:0/1891197468 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8034d50 con 0x7f47f01606e0 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.299+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1891197468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f015f730 con 0x7f47f01606e0 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.300+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1891197468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f0069400 con 0x7f47f01606e0 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.301+0000 7f47d7fff640 1 -- 172.21.15.60:0/1891197468 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47e8031030 con 0x7f47f01606e0 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.301+0000 7f47d7fff640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f47dc06ae40 0x7f47dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.301+0000 7f47d7fff640 1 -- 172.21.15.60:0/1891197468 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47e80ab8d0 con 0x7f47f01606e0 2023-11-18T20:42:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.301+0000 7f47ef7fe640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f47dc06ae40 0x7f47dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.302+0000 7f47ef7fe640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f47dc06ae40 0x7f47dc06d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009b40 tx=0x7f47e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:48.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.305+0000 7f47d7fff640 1 -- 172.21.15.60:0/1891197468 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47e8076b60 con 0x7f47f01606e0 2023-11-18T20:42:48.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.479+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1891197468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47f0069400 con 0x7f47f01606e0 2023-11-18T20:42:48.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.480+0000 7f47d7fff640 1 -- 172.21.15.60:0/1891197468 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47e8002000 con 0x7f47f01606e0 2023-11-18T20:42:48.481 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:42:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.481+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f47dc06ae40 msgr2=0x7f47dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.481+0000 7f47f5d0c640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f47dc06ae40 0x7f47dc06d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009b40 tx=0x7f47e0008040 comp rx=0 tx=0).stop 2023-11-18T20:42:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.481+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1891197468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f01606e0 msgr2=0x7f47f015b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.481+0000 7f47f5d0c640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f01606e0 0x7f47f015b9a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f47e800a8d0 tx=0x7f47e8001360 comp rx=0 tx=0).stop 2023-11-18T20:42:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.482+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1891197468 shutdown_connections 2023-11-18T20:42:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.482+0000 7f47f5d0c640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f47dc06ae40 0x7f47dc06d300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.482+0000 7f47f5d0c640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47f01606e0 0x7f47f015b9a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.482+0000 7f47f5d0c640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47f0158980 0x7f47f015b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.482+0000 7f47f5d0c640 1 --2- 172.21.15.60:0/1891197468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47f01585a0 0x7f47f015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.482+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1891197468 >> 172.21.15.60:0/1891197468 conn(0x7f47f0072230 msgr2=0x7f47f015c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.482+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1891197468 shutdown_connections 2023-11-18T20:42:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.482+0000 7f47f5d0c640 1 -- 172.21.15.60:0/1891197468 wait complete. 2023-11-18T20:42:48.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:42:48.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.601+0000 7f50f6826640 1 Processor -- start 2023-11-18T20:42:48.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.601+0000 7f50f6826640 1 -- start start 2023-11-18T20:42:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.602+0000 7f50f6826640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50f0160fb0 0x7f50f0161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.602+0000 7f50f6826640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f50f00d0ed0 0x7f50f00d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.602+0000 7f50f6826640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f0161d20 con 0x7f50f006ea50 2023-11-18T20:42:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.602+0000 7f50f6826640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f015e760 con 0x7f50f0160fb0 2023-11-18T20:42:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.602+0000 7f50f6826640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f015d9a0 con 0x7f50f00d0ed0 2023-11-18T20:42:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.602+0000 7f50ef7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50f0160fb0 0x7f50f0161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.602+0000 7f50effff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f50f006ea50 0x7f50f006ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39266/0 (socket says 172.21.15.60:39266) 2023-11-18T20:42:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.602+0000 7f50effff640 1 -- 172.21.15.60:0/3884785891 learned_addr learned my addr 172.21.15.60:0/3884785891 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.602+0000 7f50f4dc3640 1 --2- 172.21.15.60:0/3884785891 >> v2:172.21.15.100:3300/0 conn(0x7f50f00d0ed0 0x7f50f00d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.603+0000 7f50ef7fe640 1 -- 172.21.15.60:0/3884785891 >> v2:172.21.15.100:3300/0 conn(0x7f50f00d0ed0 msgr2=0x7f50f00d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.603+0000 7f50ef7fe640 1 --2- 172.21.15.60:0/3884785891 >> v2:172.21.15.100:3300/0 conn(0x7f50f00d0ed0 0x7f50f00d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.603+0000 7f50ef7fe640 1 -- 172.21.15.60:0/3884785891 >> v1:172.21.15.60:6789/0 conn(0x7f50f006ea50 legacy=0x7f50f006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.603+0000 7f50ef7fe640 1 -- 172.21.15.60:0/3884785891 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f015dc80 con 0x7f50f0160fb0 2023-11-18T20:42:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.603+0000 7f50eeffd640 1 -- 172.21.15.60:0/3884785891 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183689385 0 0) 0x7f50f0161d20 con 0x7f50f006ea50 2023-11-18T20:42:48.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.603+0000 7f50f4dc3640 1 --2- 172.21.15.60:0/3884785891 >> v2:172.21.15.100:3300/0 conn(0x7f50f00d0ed0 0x7f50f00d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.603+0000 7f50ef7fe640 1 --2- 172.21.15.60:0/3884785891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f0160fb0 0x7f50f0161440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f50f00b55a0 tx=0x7f50e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ea5ab8642d0513 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.603+0000 7f50eeffd640 1 -- 172.21.15.60:0/3884785891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e002e070 con 0x7f50f0160fb0 2023-11-18T20:42:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.603+0000 7f50eeffd640 1 -- 172.21.15.60:0/3884785891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e00315a0 con 0x7f50f0160fb0 2023-11-18T20:42:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.603+0000 7f50eeffd640 1 -- 172.21.15.60:0/3884785891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0031930 con 0x7f50f0160fb0 2023-11-18T20:42:48.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.604+0000 7f50f6826640 1 -- 172.21.15.60:0/3884785891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f0160fb0 msgr2=0x7f50f0161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.604+0000 7f50f6826640 1 --2- 172.21.15.60:0/3884785891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f0160fb0 0x7f50f0161440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f50f00b55a0 tx=0x7f50e002b840 comp rx=0 tx=0).stop 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.604+0000 7f50f6826640 1 -- 172.21.15.60:0/3884785891 shutdown_connections 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.604+0000 7f50f6826640 1 --2- 172.21.15.60:0/3884785891 >> v2:172.21.15.100:3300/0 conn(0x7f50f00d0ed0 0x7f50f00d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.604+0000 7f50f6826640 1 --2- 172.21.15.60:0/3884785891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f0160fb0 0x7f50f0161440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.604+0000 7f50f6826640 1 -- 172.21.15.60:0/3884785891 >> 172.21.15.60:0/3884785891 conn(0x7f50f006a6b0 msgr2=0x7f50f006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.604+0000 7f50f6826640 1 -- 172.21.15.60:0/3884785891 shutdown_connections 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.604+0000 7f50f6826640 1 -- 172.21.15.60:0/3884785891 wait complete. 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.605+0000 7f50f6826640 1 Processor -- start 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.605+0000 7f50f6826640 1 -- start start 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50f6826640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f006ea50 0x7f50f013de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50f6826640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f00d0ed0 0x7f50f01e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50f6826640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f0160fb0 0x7f50f01e6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50f6826640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01587e0 con 0x7f50f006ea50 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50f6826640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f013e3b0 con 0x7f50f00d0ed0 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50f6826640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f006c580 con 0x7f50f0160fb0 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f006ea50 0x7f50f013de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f006ea50 0x7f50f013de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41612/0 (socket says 172.21.15.60:41612) 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50effff640 1 -- 172.21.15.60:0/2682850734 learned_addr learned my addr 172.21.15.60:0/2682850734 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50ef7fe640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f00d0ed0 0x7f50f01e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50f4dc3640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f0160fb0 0x7f50f01e6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50effff640 1 -- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f0160fb0 msgr2=0x7f50f01e6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50effff640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f0160fb0 0x7f50f01e6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50effff640 1 -- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f00d0ed0 msgr2=0x7f50f01e6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50effff640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f00d0ed0 0x7f50f01e6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.606+0000 7f50effff640 1 -- 172.21.15.60:0/2682850734 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0009400 con 0x7f50f006ea50 2023-11-18T20:42:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.607+0000 7f50effff640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f006ea50 0x7f50f013de70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c430 tx=0x7f50d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.607+0000 7f50ecff9640 1 -- 172.21.15.60:0/2682850734 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80074e0 con 0x7f50f006ea50 2023-11-18T20:42:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.607+0000 7f50ecff9640 1 -- 172.21.15.60:0/2682850734 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d8007660 con 0x7f50f006ea50 2023-11-18T20:42:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.607+0000 7f50ecff9640 1 -- 172.21.15.60:0/2682850734 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8004a20 con 0x7f50f006ea50 2023-11-18T20:42:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.607+0000 7f50f6826640 1 -- 172.21.15.60:0/2682850734 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f006c860 con 0x7f50f006ea50 2023-11-18T20:42:48.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.607+0000 7f50f6826640 1 -- 172.21.15.60:0/2682850734 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f006ce20 con 0x7f50f006ea50 2023-11-18T20:42:48.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.608+0000 7f50f6826640 1 -- 172.21.15.60:0/2682850734 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f0069400 con 0x7f50f006ea50 2023-11-18T20:42:48.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.608+0000 7f50ecff9640 1 -- 172.21.15.60:0/2682850734 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50d8004b80 con 0x7f50f006ea50 2023-11-18T20:42:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.609+0000 7f50ecff9640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f50d006ae40 0x7f50d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.609+0000 7f50ecff9640 1 -- 172.21.15.60:0/2682850734 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50d80894b0 con 0x7f50f006ea50 2023-11-18T20:42:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.609+0000 7f50ef7fe640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f50d006ae40 0x7f50d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.609+0000 7f50ef7fe640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f50d006ae40 0x7f50d006d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50e0036000 tx=0x7f50e003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:48.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.613+0000 7f50ecff9640 1 -- 172.21.15.60:0/2682850734 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50d8054550 con 0x7f50f006ea50 2023-11-18T20:42:48.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.791+0000 7f50f6826640 1 -- 172.21.15.60:0/2682850734 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50f0070d30 con 0x7f50f006ea50 2023-11-18T20:42:48.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.791+0000 7f50ecff9640 1 -- 172.21.15.60:0/2682850734 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f50d8057b70 con 0x7f50f006ea50 2023-11-18T20:42:48.793 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:42:48.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 -- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f50d006ae40 msgr2=0x7f50d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f50d006ae40 0x7f50d006d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50e0036000 tx=0x7f50e003d040 comp rx=0 tx=0).stop 2023-11-18T20:42:48.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 -- 172.21.15.60:0/2682850734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f006ea50 msgr2=0x7f50f013de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f006ea50 0x7f50f013de70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c430 tx=0x7f50d800c8f0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 -- 172.21.15.60:0/2682850734 shutdown_connections 2023-11-18T20:42:48.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f50d006ae40 0x7f50d006d300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50f0160fb0 0x7f50f01e6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f00d0ed0 0x7f50f01e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 --2- 172.21.15.60:0/2682850734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50f006ea50 0x7f50f013de70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 -- 172.21.15.60:0/2682850734 >> 172.21.15.60:0/2682850734 conn(0x7f50f006a6b0 msgr2=0x7f50f00b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:48.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 -- 172.21.15.60:0/2682850734 shutdown_connections 2023-11-18T20:42:48.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.793+0000 7f50f6826640 1 -- 172.21.15.60:0/2682850734 wait complete. 2023-11-18T20:42:48.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:42:48.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.913+0000 7fcf1a9c9640 1 Processor -- start 2023-11-18T20:42:48.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.913+0000 7fcf1a9c9640 1 -- start start 2023-11-18T20:42:48.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.914+0000 7fcf1a9c9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcf1415d330 0x7fcf140b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.914+0000 7fcf1a9c9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf140d3900 0x7fcf140b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.914+0000 7fcf1a9c9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf140b2b10 con 0x7fcf140d2910 2023-11-18T20:42:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.914+0000 7fcf1a9c9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf140d1590 con 0x7fcf140d3900 2023-11-18T20:42:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.914+0000 7fcf1a9c9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf140d1870 con 0x7fcf1415d330 2023-11-18T20:42:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.914+0000 7fcf18f66640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf140d3900 0x7fcf140b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.914+0000 7fcf18f66640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcf140d3900 0x7fcf140b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41622/0 (socket says 172.21.15.60:41622) 2023-11-18T20:42:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.914+0000 7fcf18f66640 1 -- 172.21.15.60:0/2201673869 learned_addr learned my addr 172.21.15.60:0/2201673869 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.915+0000 7fcf18f66640 1 -- 172.21.15.60:0/2201673869 >> v2:172.21.15.100:3300/0 conn(0x7fcf1415d330 msgr2=0x7fcf140b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.915+0000 7fcf13fff640 1 --2- 172.21.15.60:0/2201673869 >> v2:172.21.15.100:3300/0 conn(0x7fcf1415d330 0x7fcf140b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.915+0000 7fcf18f66640 1 --2- 172.21.15.60:0/2201673869 >> v2:172.21.15.100:3300/0 conn(0x7fcf1415d330 0x7fcf140b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.915+0000 7fcf18f66640 1 -- 172.21.15.60:0/2201673869 >> v1:172.21.15.100:6789/0 conn(0x7fcf140d2910 legacy=0x7fcf140d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.915+0000 7fcf18f66640 1 -- 172.21.15.60:0/2201673869 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf140d1b50 con 0x7fcf140d3900 2023-11-18T20:42:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.915+0000 7fcf13fff640 1 --2- 172.21.15.60:0/2201673869 >> v2:172.21.15.100:3300/0 conn(0x7fcf1415d330 0x7fcf140b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.915+0000 7fcf18f66640 1 --2- 172.21.15.60:0/2201673869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 0x7fcf140b6390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf140d0d10 tx=0x7fcf0c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f83f9a0e51fce7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf12ffd640 1 -- 172.21.15.60:0/2201673869 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c02c9f0 con 0x7fcf140d3900 2023-11-18T20:42:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf12ffd640 1 -- 172.21.15.60:0/2201673869 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c02cb50 con 0x7fcf140d3900 2023-11-18T20:42:48.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf12ffd640 1 -- 172.21.15.60:0/2201673869 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0343f0 con 0x7fcf140d3900 2023-11-18T20:42:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/2201673869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 msgr2=0x7fcf140b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf1a9c9640 1 --2- 172.21.15.60:0/2201673869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 0x7fcf140b6390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf140d0d10 tx=0x7fcf0c02be50 comp rx=0 tx=0).stop 2023-11-18T20:42:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/2201673869 shutdown_connections 2023-11-18T20:42:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf1a9c9640 1 --2- 172.21.15.60:0/2201673869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 0x7fcf140b6390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf1a9c9640 1 --2- 172.21.15.60:0/2201673869 >> v2:172.21.15.100:3300/0 conn(0x7fcf1415d330 0x7fcf140b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/2201673869 >> 172.21.15.60:0/2201673869 conn(0x7fcf14072020 msgr2=0x7fcf14072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:48.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/2201673869 shutdown_connections 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.916+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/2201673869 wait complete. 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.917+0000 7fcf1a9c9640 1 Processor -- start 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.918+0000 7fcf1a9c9640 1 -- start start 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.918+0000 7fcf1a9c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf140d2910 0x7fcf1415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.918+0000 7fcf1a9c9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 0x7fcf140ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.918+0000 7fcf1a9c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf1415d330 0x7fcf14154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.918+0000 7fcf1a9c9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14154f50 con 0x7fcf140d3900 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.918+0000 7fcf1a9c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1415ece0 con 0x7fcf140d2910 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.918+0000 7fcf1a9c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140caa30 con 0x7fcf1415d330 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.918+0000 7fcf137fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 0x7fcf140ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.918+0000 7fcf137fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 0x7fcf140ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41638/0 (socket says 172.21.15.60:41638) 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.918+0000 7fcf137fe640 1 -- 172.21.15.60:0/3479993917 learned_addr learned my addr 172.21.15.60:0/3479993917 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf13fff640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf140d2910 0x7fcf1415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf18f66640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf1415d330 0x7fcf14154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf137fe640 1 -- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf1415d330 msgr2=0x7fcf14154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf137fe640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf1415d330 0x7fcf14154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf137fe640 1 -- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf140d2910 msgr2=0x7fcf1415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf137fe640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf140d2910 0x7fcf1415f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf137fe640 1 -- 172.21.15.60:0/3479993917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c009b30 con 0x7fcf140d3900 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf137fe640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 0x7fcf140ca330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800c430 tx=0x7fcf0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf10ff9640 1 -- 172.21.15.60:0/3479993917 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08011790 con 0x7fcf140d3900 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf10ff9640 1 -- 172.21.15.60:0/3479993917 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf080118f0 con 0x7fcf140d3900 2023-11-18T20:42:48.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf10ff9640 1 -- 172.21.15.60:0/3479993917 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08011ce0 con 0x7fcf140d3900 2023-11-18T20:42:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/3479993917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf140cad10 con 0x7fcf140d3900 2023-11-18T20:42:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.919+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/3479993917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf140cb2d0 con 0x7fcf140d3900 2023-11-18T20:42:48.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.920+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/3479993917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf14069400 con 0x7fcf140d3900 2023-11-18T20:42:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.921+0000 7fcf10ff9640 1 -- 172.21.15.60:0/3479993917 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcf0801c460 con 0x7fcf140d3900 2023-11-18T20:42:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.921+0000 7fcf10ff9640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcf0006ae40 0x7fcf0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.921+0000 7fcf10ff9640 1 -- 172.21.15.60:0/3479993917 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf08089550 con 0x7fcf140d3900 2023-11-18T20:42:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.921+0000 7fcf13fff640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcf0006ae40 0x7fcf0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:48.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.922+0000 7fcf13fff640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcf0006ae40 0x7fcf0006d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400a950 tx=0x7fcf04008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:48.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:48.925+0000 7fcf10ff9640 1 -- 172.21.15.60:0/3479993917 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf08054690 con 0x7fcf140d3900 2023-11-18T20:42:49.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.100+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/3479993917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf140b6710 con 0x7fcf140d3900 2023-11-18T20:42:49.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.100+0000 7fcf10ff9640 1 -- 172.21.15.60:0/3479993917 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf08057cb0 con 0x7fcf140d3900 2023-11-18T20:42:49.101 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:42:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcf0006ae40 msgr2=0x7fcf0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcf0006ae40 0x7fcf0006d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400a950 tx=0x7fcf04008040 comp rx=0 tx=0).stop 2023-11-18T20:42:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/3479993917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 msgr2=0x7fcf140ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 0x7fcf140ca330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800c430 tx=0x7fcf0800c8f0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/3479993917 shutdown_connections 2023-11-18T20:42:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcf0006ae40 0x7fcf0006d300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf1415d330 0x7fcf14154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140d3900 0x7fcf140ca330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 --2- 172.21.15.60:0/3479993917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf140d2910 0x7fcf1415f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/3479993917 >> 172.21.15.60:0/3479993917 conn(0x7fcf14072020 msgr2=0x7fcf140d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:49.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/3479993917 shutdown_connections 2023-11-18T20:42:49.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.102+0000 7fcf1a9c9640 1 -- 172.21.15.60:0/3479993917 wait complete. 2023-11-18T20:42:49.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:42:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.219+0000 7f3545a75640 1 Processor -- start 2023-11-18T20:42:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.219+0000 7f3545a75640 1 -- start start 2023-11-18T20:42:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.220+0000 7f3545a75640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354015d330 0x7f35400b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.220+0000 7f3545a75640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35400d2910 0x7f35400d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.220+0000 7f3545a75640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35400b2b10 con 0x7f35400d2d90 2023-11-18T20:42:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.220+0000 7f3545a75640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35400d1590 con 0x7f35400d2910 2023-11-18T20:42:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.220+0000 7f3545a75640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35400d1870 con 0x7f354015d330 2023-11-18T20:42:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.220+0000 7f353effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35400d2910 0x7f35400d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.220+0000 7f353effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35400d2910 0x7f35400d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41654/0 (socket says 172.21.15.60:41654) 2023-11-18T20:42:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.220+0000 7f353effd640 1 -- 172.21.15.60:0/239492042 learned_addr learned my addr 172.21.15.60:0/239492042 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.221+0000 7f353f7fe640 1 --2- 172.21.15.60:0/239492042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354015d330 0x7f35400b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.221+0000 7f353effd640 1 -- 172.21.15.60:0/239492042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354015d330 msgr2=0x7f35400b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.221+0000 7f353effd640 1 --2- 172.21.15.60:0/239492042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354015d330 0x7f35400b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.221+0000 7f353effd640 1 -- 172.21.15.60:0/239492042 >> v1:172.21.15.100:6789/0 conn(0x7f35400d2d90 legacy=0x7f35400b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.221+0000 7f353effd640 1 -- 172.21.15.60:0/239492042 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35400d1b50 con 0x7f35400d2910 2023-11-18T20:42:49.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.221+0000 7f353effd640 1 --2- 172.21.15.60:0/239492042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35400d2910 0x7f35400d31f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3534009870 tx=0x7f353402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21ae948c12b9494 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:49.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f353e7fc640 1 -- 172.21.15.60:0/239492042 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534031640 con 0x7f35400d2910 2023-11-18T20:42:49.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f353e7fc640 1 -- 172.21.15.60:0/239492042 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35340317a0 con 0x7f35400d2910 2023-11-18T20:42:49.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f353e7fc640 1 -- 172.21.15.60:0/239492042 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534031b30 con 0x7f35400d2910 2023-11-18T20:42:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f3545a75640 1 -- 172.21.15.60:0/239492042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35400d2910 msgr2=0x7f35400d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f3545a75640 1 --2- 172.21.15.60:0/239492042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35400d2910 0x7f35400d31f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3534009870 tx=0x7f353402b8f0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f3545a75640 1 -- 172.21.15.60:0/239492042 shutdown_connections 2023-11-18T20:42:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f3545a75640 1 --2- 172.21.15.60:0/239492042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35400d2910 0x7f35400d31f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f3545a75640 1 --2- 172.21.15.60:0/239492042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354015d330 0x7f35400b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f3545a75640 1 -- 172.21.15.60:0/239492042 >> 172.21.15.60:0/239492042 conn(0x7f3540072020 msgr2=0x7f3540072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f3545a75640 1 -- 172.21.15.60:0/239492042 shutdown_connections 2023-11-18T20:42:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.222+0000 7f3545a75640 1 -- 172.21.15.60:0/239492042 wait complete. 2023-11-18T20:42:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.223+0000 7f3545a75640 1 Processor -- start 2023-11-18T20:42:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f3545a75640 1 -- start start 2023-11-18T20:42:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f3545a75640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35400d2910 0x7f354014d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f3545a75640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35400d2d90 0x7f354015d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f3545a75640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354015d330 0x7f35400cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f3545a75640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35400c8670 con 0x7f354015d330 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f3545a75640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354014dc50 con 0x7f35400d2910 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f3545a75640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354014e0f0 con 0x7f35400d2d90 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f353ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354015d330 0x7f35400cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f353ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354015d330 0x7f35400cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41664/0 (socket says 172.21.15.60:41664) 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f353ffff640 1 -- 172.21.15.60:0/3871808031 learned_addr learned my addr 172.21.15.60:0/3871808031 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f353f7fe640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35400d2910 0x7f354014d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.224+0000 7f353effd640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35400d2d90 0x7f354015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f353ffff640 1 -- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35400d2d90 msgr2=0x7f354015d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f353ffff640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35400d2d90 0x7f354015d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f353ffff640 1 -- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35400d2910 msgr2=0x7f354014d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f353ffff640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35400d2910 0x7f354014d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f353ffff640 1 -- 172.21.15.60:0/3871808031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340094d0 con 0x7f354015d330 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f353f7fe640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35400d2910 0x7f354014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f353ffff640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354015d330 0x7f35400cb880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f353800c730 tx=0x7f353800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f3527fff640 1 -- 172.21.15.60:0/3871808031 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538003b70 con 0x7f354015d330 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f3527fff640 1 -- 172.21.15.60:0/3871808031 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3538003cd0 con 0x7f354015d330 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f3545a75640 1 -- 172.21.15.60:0/3871808031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354015de60 con 0x7f354015d330 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f3527fff640 1 -- 172.21.15.60:0/3871808031 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800aa90 con 0x7f354015d330 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.225+0000 7f3545a75640 1 -- 172.21.15.60:0/3871808031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35400c7cd0 con 0x7f354015d330 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.226+0000 7f3545a75640 1 -- 172.21.15.60:0/3871808031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35400c87b0 con 0x7f354015d330 2023-11-18T20:42:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.226+0000 7f3527fff640 1 -- 172.21.15.60:0/3871808031 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f353800abf0 con 0x7f354015d330 2023-11-18T20:42:49.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.227+0000 7f3527fff640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f352c06ae40 0x7f352c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.227+0000 7f3527fff640 1 -- 172.21.15.60:0/3871808031 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3538089150 con 0x7f354015d330 2023-11-18T20:42:49.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.227+0000 7f353f7fe640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f352c06ae40 0x7f352c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.228+0000 7f353f7fe640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f352c06ae40 0x7f352c06d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3530006fd0 tx=0x7f3530008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:49.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.231+0000 7f3527fff640 1 -- 172.21.15.60:0/3871808031 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35400c87b0 con 0x7f354015d330 2023-11-18T20:42:49.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.443+0000 7f3545a75640 1 -- 172.21.15.60:0/3871808031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f354015de60 con 0x7f354015d330 2023-11-18T20:42:49.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.443+0000 7f3527fff640 1 -- 172.21.15.60:0/3871808031 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f354015de60 con 0x7f354015d330 2023-11-18T20:42:49.445 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:49.445 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1530496056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1530496056},{"type":"v1","addr":"172.21.15.60:6833","nonce":1530496056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3842389676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3842389676},{"type":"v1","addr":"172.21.15.60:6835","nonce":3842389676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4248907695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4248907695},{"type":"v1","addr":"172.21.15.100:6836","nonce":4248907695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:42:37.459556+0000","modified":"2023-11-18T20:42:39.458016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1851816776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1851816776},{"type":"v1","addr":"172.21.15.100:6837","nonce":1851816776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-18T20:42:49.445 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2023-11-18T20:42:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 -- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f352c06ae40 msgr2=0x7f352c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f352c06ae40 0x7f352c06d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3530006fd0 tx=0x7f3530008040 comp rx=0 tx=0).stop 2023-11-18T20:42:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 -- 172.21.15.60:0/3871808031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354015d330 msgr2=0x7f35400cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354015d330 0x7f35400cb880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f353800c730 tx=0x7f353800cbf0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 -- 172.21.15.60:0/3871808031 shutdown_connections 2023-11-18T20:42:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f352c06ae40 0x7f352c06d300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354015d330 0x7f35400cb880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35400d2d90 0x7f354015d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.447 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 --2- 172.21.15.60:0/3871808031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35400d2910 0x7f354014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 -- 172.21.15.60:0/3871808031 >> 172.21.15.60:0/3871808031 conn(0x7f3540072020 msgr2=0x7f35400b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:49.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 -- 172.21.15.60:0/3871808031 shutdown_connections 2023-11-18T20:42:49.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.445+0000 7f3545a75640 1 -- 172.21.15.60:0/3871808031 wait complete. 2023-11-18T20:42:49.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:49.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.562+0000 7fdecb4af640 1 Processor -- start 2023-11-18T20:42:49.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.562+0000 7fdecb4af640 1 -- start start 2023-11-18T20:42:49.565 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdecb4af640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdec415d2d0 0x7fdec40b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdecb4af640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec40d2040 0x7fdec40b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdecb4af640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdec40cda40 0x7fdec40cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdecb4af640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec40d1bb0 con 0x7fdec40cda40 2023-11-18T20:42:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdecb4af640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec415d6b0 con 0x7fdec415d2d0 2023-11-18T20:42:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdecb4af640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40ce700 con 0x7fdec40d2040 2023-11-18T20:42:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec9a4c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdec40cda40 0x7fdec40cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec9a4c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdec40cda40 0x7fdec40cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41672/0 (socket says 172.21.15.60:41672) 2023-11-18T20:42:49.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec9a4c640 1 -- 172.21.15.60:0/1871870662 learned_addr learned my addr 172.21.15.60:0/1871870662 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:49.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec8a4a640 1 --2- 172.21.15.60:0/1871870662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec40d2040 0x7fdec40b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec924b640 1 --2- 172.21.15.60:0/1871870662 >> v2:172.21.15.100:3300/0 conn(0x7fdec415d2d0 0x7fdec40b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec9a4c640 1 -- 172.21.15.60:0/1871870662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec40d2040 msgr2=0x7fdec40b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:49.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec9a4c640 1 --2- 172.21.15.60:0/1871870662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec40d2040 0x7fdec40b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec9a4c640 1 -- 172.21.15.60:0/1871870662 >> v2:172.21.15.100:3300/0 conn(0x7fdec415d2d0 msgr2=0x7fdec40b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:49.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec9a4c640 1 --2- 172.21.15.60:0/1871870662 >> v2:172.21.15.100:3300/0 conn(0x7fdec415d2d0 0x7fdec40b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec9a4c640 1 -- 172.21.15.60:0/1871870662 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec40d1280 con 0x7fdec40cda40 2023-11-18T20:42:49.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec8a4a640 1 --2- 172.21.15.60:0/1871870662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec40d2040 0x7fdec40b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:49.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.563+0000 7fdec924b640 1 --2- 172.21.15.60:0/1871870662 >> v2:172.21.15.100:3300/0 conn(0x7fdec415d2d0 0x7fdec40b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:49.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.564+0000 7fdec9a4c640 1 --2- 172.21.15.60:0/1871870662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 0x7fdec40cded0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdec406c5d0 tx=0x7fdeb402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3187d40952c4dd5a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:49.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.564+0000 7fdec3fff640 1 -- 172.21.15.60:0/1871870662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb402ca70 con 0x7fdec40cda40 2023-11-18T20:42:49.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.564+0000 7fdec3fff640 1 -- 172.21.15.60:0/1871870662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb402cbd0 con 0x7fdec40cda40 2023-11-18T20:42:49.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.564+0000 7fdec3fff640 1 -- 172.21.15.60:0/1871870662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40343f0 con 0x7fdec40cda40 2023-11-18T20:42:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.565+0000 7fdecb4af640 1 -- 172.21.15.60:0/1871870662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 msgr2=0x7fdec40cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.565+0000 7fdecb4af640 1 --2- 172.21.15.60:0/1871870662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 0x7fdec40cded0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdec406c5d0 tx=0x7fdeb402bed0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.565+0000 7fdecb4af640 1 -- 172.21.15.60:0/1871870662 shutdown_connections 2023-11-18T20:42:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.565+0000 7fdecb4af640 1 --2- 172.21.15.60:0/1871870662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 0x7fdec40cded0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.565+0000 7fdecb4af640 1 --2- 172.21.15.60:0/1871870662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec40d2040 0x7fdec40b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.565+0000 7fdecb4af640 1 --2- 172.21.15.60:0/1871870662 >> v2:172.21.15.100:3300/0 conn(0x7fdec415d2d0 0x7fdec40b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.565+0000 7fdecb4af640 1 -- 172.21.15.60:0/1871870662 >> 172.21.15.60:0/1871870662 conn(0x7fdec4072050 msgr2=0x7fdec4072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.565+0000 7fdecb4af640 1 -- 172.21.15.60:0/1871870662 shutdown_connections 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.565+0000 7fdecb4af640 1 -- 172.21.15.60:0/1871870662 wait complete. 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.566+0000 7fdecb4af640 1 Processor -- start 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.566+0000 7fdecb4af640 1 -- start start 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdecb4af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 0x7fdec4156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdecb4af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec40d2040 0x7fdec40c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdecb4af640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec415d2d0 0x7fdec40cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdecb4af640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40cb990 con 0x7fdec40cda40 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdecb4af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40cbc70 con 0x7fdec40d2040 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdecb4af640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40cbf50 con 0x7fdec415d2d0 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec924b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 0x7fdec4156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec924b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 0x7fdec4156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41686/0 (socket says 172.21.15.60:41686) 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec924b640 1 -- 172.21.15.60:0/288424688 learned_addr learned my addr 172.21.15.60:0/288424688 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec9a4c640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec415d2d0 0x7fdec40cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec924b640 1 -- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec415d2d0 msgr2=0x7fdec40cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec8a4a640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec40d2040 0x7fdec40c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec924b640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec415d2d0 0x7fdec40cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec924b640 1 -- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec40d2040 msgr2=0x7fdec40c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec924b640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec40d2040 0x7fdec40c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec924b640 1 -- 172.21.15.60:0/288424688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb4009b30 con 0x7fdec40cda40 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.567+0000 7fdec924b640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 0x7fdec4156580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00d7d0 tx=0x7fdebc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.568+0000 7fdec1ffb640 1 -- 172.21.15.60:0/288424688 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc004030 con 0x7fdec40cda40 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.568+0000 7fdec1ffb640 1 -- 172.21.15.60:0/288424688 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc004190 con 0x7fdec40cda40 2023-11-18T20:42:49.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.568+0000 7fdecb4af640 1 -- 172.21.15.60:0/288424688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec40cc230 con 0x7fdec40cda40 2023-11-18T20:42:49.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.568+0000 7fdec1ffb640 1 -- 172.21.15.60:0/288424688 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc004520 con 0x7fdec40cda40 2023-11-18T20:42:49.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.568+0000 7fdecb4af640 1 -- 172.21.15.60:0/288424688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec4156ac0 con 0x7fdec40cda40 2023-11-18T20:42:49.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.569+0000 7fdecb4af640 1 -- 172.21.15.60:0/288424688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec4069400 con 0x7fdec40cda40 2023-11-18T20:42:49.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.569+0000 7fdec1ffb640 1 -- 172.21.15.60:0/288424688 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdebc0096a0 con 0x7fdec40cda40 2023-11-18T20:42:49.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.569+0000 7fdec1ffb640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdeb006ae40 0x7fdeb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.570+0000 7fdec1ffb640 1 -- 172.21.15.60:0/288424688 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdebc0897c0 con 0x7fdec40cda40 2023-11-18T20:42:49.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.570+0000 7fdec8a4a640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdeb006ae40 0x7fdeb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.570+0000 7fdec8a4a640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdeb006ae40 0x7fdeb006d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8006fd0 tx=0x7fdeb8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:49.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.574+0000 7fdec1ffb640 1 -- 172.21.15.60:0/288424688 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdebc054870 con 0x7fdec40cda40 2023-11-18T20:42:49.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.753+0000 7fdecb4af640 1 -- 172.21.15.60:0/288424688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdec40cf2d0 con 0x7fdec40cda40 2023-11-18T20:42:49.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.754+0000 7fdec1ffb640 1 -- 172.21.15.60:0/288424688 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fdebc057e90 con 0x7fdec40cda40 2023-11-18T20:42:49.755 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:49.755 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:49.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 -- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdeb006ae40 msgr2=0x7fdeb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdeb006ae40 0x7fdeb006d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8006fd0 tx=0x7fdeb8008040 comp rx=0 tx=0).stop 2023-11-18T20:42:49.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 -- 172.21.15.60:0/288424688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 msgr2=0x7fdec4156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 0x7fdec4156580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00d7d0 tx=0x7fdebc00dc90 comp rx=0 tx=0).stop 2023-11-18T20:42:49.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 -- 172.21.15.60:0/288424688 shutdown_connections 2023-11-18T20:42:49.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdeb006ae40 0x7fdeb006d300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdec415d2d0 0x7fdec40cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdec40d2040 0x7fdec40c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 --2- 172.21.15.60:0/288424688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdec40cda40 0x7fdec4156580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 -- 172.21.15.60:0/288424688 >> 172.21.15.60:0/288424688 conn(0x7fdec4072050 msgr2=0x7fdec415dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:49.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 -- 172.21.15.60:0/288424688 shutdown_connections 2023-11-18T20:42:49.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.756+0000 7fdecb4af640 1 -- 172.21.15.60:0/288424688 wait complete. 2023-11-18T20:42:49.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:42:49.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.873+0000 7f8523651640 1 Processor -- start 2023-11-18T20:42:49.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.873+0000 7f8523651640 1 -- start start 2023-11-18T20:42:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8523651640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 0x7f851c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8523651640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f851c0d3730 0x7f851c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8523651640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851c0b2b10 con 0x7f851c15d330 2023-11-18T20:42:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8523651640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c0d3b50 con 0x7f851c0d3730 2023-11-18T20:42:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8523651640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0d1590 con 0x7f851c0d2910 2023-11-18T20:42:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8520bec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 0x7f851c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8521bee640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f851c0d3730 0x7f851c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f85213ed640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f851c15d330 0x7f851c0b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:53140/0 (socket says 172.21.15.60:53140) 2023-11-18T20:42:49.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f85213ed640 1 -- 172.21.15.60:0/1144577084 learned_addr learned my addr 172.21.15.60:0/1144577084 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:49.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f851bfff640 1 -- 172.21.15.60:0/1144577084 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454220682 0 0) 0x7f851c0b2b10 con 0x7f851c15d330 2023-11-18T20:42:49.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f851bfff640 1 -- 172.21.15.60:0/1144577084 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8504003520 con 0x7f851c15d330 2023-11-18T20:42:49.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8520bec640 1 -- 172.21.15.60:0/1144577084 >> v2:172.21.15.100:3300/0 conn(0x7f851c0d3730 msgr2=0x7f851c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:49.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8520bec640 1 --2- 172.21.15.60:0/1144577084 >> v2:172.21.15.100:3300/0 conn(0x7f851c0d3730 0x7f851c0b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8520bec640 1 -- 172.21.15.60:0/1144577084 >> v1:172.21.15.100:6789/0 conn(0x7f851c15d330 legacy=0x7f851c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8520bec640 1 -- 172.21.15.60:0/1144577084 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0d1870 con 0x7f851c0d2910 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.874+0000 7f8521bee640 1 --2- 172.21.15.60:0/1144577084 >> v2:172.21.15.100:3300/0 conn(0x7f851c0d3730 0x7f851c0b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.875+0000 7f8520bec640 1 --2- 172.21.15.60:0/1144577084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 0x7f851c0d31f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85100097a0 tx=0x7f851002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9377ba1164d5205e server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.875+0000 7f851bfff640 1 -- 172.21.15.60:0/1144577084 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851002e070 con 0x7f851c0d2910 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.875+0000 7f851bfff640 1 -- 172.21.15.60:0/1144577084 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85100315a0 con 0x7f851c0d2910 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.875+0000 7f851bfff640 1 -- 172.21.15.60:0/1144577084 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510031910 con 0x7f851c0d2910 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.876+0000 7f8523651640 1 -- 172.21.15.60:0/1144577084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 msgr2=0x7f851c0d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.876+0000 7f8523651640 1 --2- 172.21.15.60:0/1144577084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 0x7f851c0d31f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85100097a0 tx=0x7f851002b840 comp rx=0 tx=0).stop 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.876+0000 7f8523651640 1 -- 172.21.15.60:0/1144577084 shutdown_connections 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.876+0000 7f8523651640 1 --2- 172.21.15.60:0/1144577084 >> v2:172.21.15.100:3300/0 conn(0x7f851c0d3730 0x7f851c0b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.876+0000 7f8523651640 1 --2- 172.21.15.60:0/1144577084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 0x7f851c0d31f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.876+0000 7f8523651640 1 -- 172.21.15.60:0/1144577084 >> 172.21.15.60:0/1144577084 conn(0x7f851c072020 msgr2=0x7f851c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:49.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.876+0000 7f8523651640 1 -- 172.21.15.60:0/1144577084 shutdown_connections 2023-11-18T20:42:49.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.876+0000 7f8523651640 1 -- 172.21.15.60:0/1144577084 wait complete. 2023-11-18T20:42:49.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.877+0000 7f8523651640 1 Processor -- start 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.877+0000 7f8523651640 1 -- start start 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8523651640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 0x7f851c0cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8523651640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f851c0d3730 0x7f851c0b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8523651640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f851c15d330 0x7f851c0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8523651640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0b8c50 con 0x7f851c15d330 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8523651640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0cd5c0 con 0x7f851c0d3730 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8523651640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c152900 con 0x7f851c0d2910 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8521bee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f851c15d330 0x7f851c0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8521bee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f851c15d330 0x7f851c0ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41692/0 (socket says 172.21.15.60:41692) 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8521bee640 1 -- 172.21.15.60:0/3339004225 learned_addr learned my addr 172.21.15.60:0/3339004225 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f85213ed640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 0x7f851c0cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8520bec640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f851c0d3730 0x7f851c0b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8521bee640 1 -- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 msgr2=0x7f851c0cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8521bee640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 0x7f851c0cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8521bee640 1 -- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f851c0d3730 msgr2=0x7f851c0b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8521bee640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f851c0d3730 0x7f851c0b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8521bee640 1 -- 172.21.15.60:0/3339004225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c009a70 con 0x7f851c15d330 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.878+0000 7f8520bec640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f851c0d3730 0x7f851c0b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.879+0000 7f8521bee640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f851c15d330 0x7f851c0ca8d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f850c00efc0 tx=0x7f850c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.879+0000 7f8519ffb640 1 -- 172.21.15.60:0/3339004225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c009d70 con 0x7f851c15d330 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.879+0000 7f8519ffb640 1 -- 172.21.15.60:0/3339004225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c00f040 con 0x7f851c15d330 2023-11-18T20:42:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.879+0000 7f8523651640 1 -- 172.21.15.60:0/3339004225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8510009400 con 0x7f851c15d330 2023-11-18T20:42:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.879+0000 7f8519ffb640 1 -- 172.21.15.60:0/3339004225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c003990 con 0x7f851c15d330 2023-11-18T20:42:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.879+0000 7f8523651640 1 -- 172.21.15.60:0/3339004225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c0ccc90 con 0x7f851c15d330 2023-11-18T20:42:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.880+0000 7f8523651640 1 -- 172.21.15.60:0/3339004225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c143520 con 0x7f851c15d330 2023-11-18T20:42:49.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.880+0000 7f8519ffb640 1 -- 172.21.15.60:0/3339004225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f850c0144a0 con 0x7f851c15d330 2023-11-18T20:42:49.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.881+0000 7f8519ffb640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f850806ae40 0x7f850806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:49.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.881+0000 7f8519ffb640 1 -- 172.21.15.60:0/3339004225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f850c08a2a0 con 0x7f851c15d330 2023-11-18T20:42:49.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.881+0000 7f85213ed640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f850806ae40 0x7f850806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:49.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.881+0000 7f85213ed640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f850806ae40 0x7f850806d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85140011e0 tx=0x7f8514009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:49.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:49.885+0000 7f8519ffb640 1 -- 172.21.15.60:0/3339004225 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f851c143520 con 0x7f851c15d330 2023-11-18T20:42:50.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.100+0000 7f8523651640 1 -- 172.21.15.60:0/3339004225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f851c143520 con 0x7f851c15d330 2023-11-18T20:42:50.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.100+0000 7f8519ffb640 1 -- 172.21.15.60:0/3339004225 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f851c143520 con 0x7f851c15d330 2023-11-18T20:42:50.102 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:50.102 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1530496056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1530496056},{"type":"v1","addr":"172.21.15.60:6833","nonce":1530496056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3842389676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3842389676},{"type":"v1","addr":"172.21.15.60:6835","nonce":3842389676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4248907695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4248907695},{"type":"v1","addr":"172.21.15.100:6836","nonce":4248907695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:42:37.459556+0000","modified":"2023-11-18T20:42:39.458016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1851816776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1851816776},{"type":"v1","addr":"172.21.15.100:6837","nonce":1851816776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-18T20:42:50.102 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2023-11-18T20:42:50.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.102+0000 7f8523651640 1 -- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f850806ae40 msgr2=0x7f850806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.102+0000 7f8523651640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f850806ae40 0x7f850806d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85140011e0 tx=0x7f8514009040 comp rx=0 tx=0).stop 2023-11-18T20:42:50.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.102+0000 7f8523651640 1 -- 172.21.15.60:0/3339004225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f851c15d330 msgr2=0x7f851c0ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.102+0000 7f8523651640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f851c15d330 0x7f851c0ca8d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f850c00efc0 tx=0x7f850c009f90 comp rx=0 tx=0).stop 2023-11-18T20:42:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.103+0000 7f8523651640 1 -- 172.21.15.60:0/3339004225 shutdown_connections 2023-11-18T20:42:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.103+0000 7f8523651640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f850806ae40 0x7f850806d300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.103+0000 7f8523651640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f851c15d330 0x7f851c0ca8d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.103+0000 7f8523651640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f851c0d3730 0x7f851c0b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.103+0000 7f8523651640 1 --2- 172.21.15.60:0/3339004225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f851c0d2910 0x7f851c0cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.103+0000 7f8523651640 1 -- 172.21.15.60:0/3339004225 >> 172.21.15.60:0/3339004225 conn(0x7f851c072020 msgr2=0x7f851c0b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.103+0000 7f8523651640 1 -- 172.21.15.60:0/3339004225 shutdown_connections 2023-11-18T20:42:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.103+0000 7f8523651640 1 -- 172.21.15.60:0/3339004225 wait complete. 2023-11-18T20:42:50.517 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-18T20:42:37.459556+0000', 'modified': '2023-11-18T20:42:39.458016+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4303}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4303': {'gid': 4303, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6837/1851816776', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6835', 'nonce': 1851816776}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 1851816776}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-18T20:42:50.517 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-18T20:42:50.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-18T20:42:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.628+0000 7f1108bf5640 1 Processor -- start 2023-11-18T20:42:50.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.628+0000 7f1108bf5640 1 -- start start 2023-11-18T20:42:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.629+0000 7f1108bf5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1104140310 0x7f11041618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.629+0000 7f1108bf5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f110406abe0 0x7f110406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.629+0000 7f1108bf5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11041406f0 con 0x7f11041409d0 2023-11-18T20:42:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.629+0000 7f1108bf5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1104161e20 con 0x7f1104140310 2023-11-18T20:42:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.629+0000 7f1108bf5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1104143960 con 0x7f110406abe0 2023-11-18T20:42:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.629+0000 7f110259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1104140310 0x7f11041618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.629+0000 7f1101d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f11041409d0 0x7f110406a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56868/0 (socket says 172.21.15.60:56868) 2023-11-18T20:42:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.629+0000 7f1101d9b640 1 -- 172.21.15.60:0/4058729944 learned_addr learned my addr 172.21.15.60:0/4058729944 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.629+0000 7f1102d9d640 1 --2- 172.21.15.60:0/4058729944 >> v2:172.21.15.100:3300/0 conn(0x7f110406abe0 0x7f110406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.630+0000 7f110259c640 1 -- 172.21.15.60:0/4058729944 >> v2:172.21.15.100:3300/0 conn(0x7f110406abe0 msgr2=0x7f110406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.630+0000 7f110259c640 1 --2- 172.21.15.60:0/4058729944 >> v2:172.21.15.100:3300/0 conn(0x7f110406abe0 0x7f110406efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.630+0000 7f110259c640 1 -- 172.21.15.60:0/4058729944 >> v1:172.21.15.60:6789/0 conn(0x7f11041409d0 legacy=0x7f110406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.630+0000 7f110259c640 1 -- 172.21.15.60:0/4058729944 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1104142de0 con 0x7f1104140310 2023-11-18T20:42:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.630+0000 7f110159a640 1 -- 172.21.15.60:0/4058729944 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129122245 0 0) 0x7f11041406f0 con 0x7f11041409d0 2023-11-18T20:42:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.630+0000 7f110259c640 1 --2- 172.21.15.60:0/4058729944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 0x7f11041618e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1104154930 tx=0x7f10fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae216811edec0b02 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.630+0000 7f110159a640 1 -- 172.21.15.60:0/4058729944 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc02e070 con 0x7f1104140310 2023-11-18T20:42:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.630+0000 7f110159a640 1 -- 172.21.15.60:0/4058729944 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc0315a0 con 0x7f1104140310 2023-11-18T20:42:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.630+0000 7f110159a640 1 -- 172.21.15.60:0/4058729944 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc031930 con 0x7f1104140310 2023-11-18T20:42:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.631+0000 7f1108bf5640 1 -- 172.21.15.60:0/4058729944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 msgr2=0x7f11041618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.631+0000 7f1108bf5640 1 --2- 172.21.15.60:0/4058729944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 0x7f11041618e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1104154930 tx=0x7f10fc02b840 comp rx=0 tx=0).stop 2023-11-18T20:42:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.631+0000 7f1108bf5640 1 -- 172.21.15.60:0/4058729944 shutdown_connections 2023-11-18T20:42:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.631+0000 7f1108bf5640 1 --2- 172.21.15.60:0/4058729944 >> v2:172.21.15.100:3300/0 conn(0x7f110406abe0 0x7f110406efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.631+0000 7f1108bf5640 1 --2- 172.21.15.60:0/4058729944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 0x7f11041618e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.631+0000 7f1108bf5640 1 -- 172.21.15.60:0/4058729944 >> 172.21.15.60:0/4058729944 conn(0x7f11040b1e50 msgr2=0x7f11040b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.631+0000 7f1108bf5640 1 -- 172.21.15.60:0/4058729944 shutdown_connections 2023-11-18T20:42:50.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.631+0000 7f1108bf5640 1 -- 172.21.15.60:0/4058729944 wait complete. 2023-11-18T20:42:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.632+0000 7f1108bf5640 1 Processor -- start 2023-11-18T20:42:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.633+0000 7f1108bf5640 1 -- start start 2023-11-18T20:42:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.633+0000 7f1108bf5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110406abe0 0x7f1104143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.633+0000 7f1108bf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 0x7f11041f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.633+0000 7f1108bf5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11041409d0 0x7f1104145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.633+0000 7f1108bf5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11041441c0 con 0x7f1104140310 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.633+0000 7f1108bf5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11041444a0 con 0x7f110406abe0 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.633+0000 7f1108bf5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1104144780 con 0x7f11041409d0 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.633+0000 7f1101d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 0x7f11041f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.633+0000 7f1101d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 0x7f11041f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57706/0 (socket says 172.21.15.60:57706) 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.633+0000 7f1101d9b640 1 -- 172.21.15.60:0/3660749523 learned_addr learned my addr 172.21.15.60:0/3660749523 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f110259c640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110406abe0 0x7f1104143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f1101d9b640 1 -- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11041409d0 msgr2=0x7f1104145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f1101d9b640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11041409d0 0x7f1104145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f1101d9b640 1 -- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110406abe0 msgr2=0x7f1104143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f1101d9b640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110406abe0 0x7f1104143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f1101d9b640 1 -- 172.21.15.60:0/3660749523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc009400 con 0x7f1104140310 2023-11-18T20:42:50.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f110259c640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110406abe0 0x7f1104143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f1101d9b640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 0x7f11041f3920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00c4d0 tx=0x7f10ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f10faffd640 1 -- 172.21.15.60:0/3660749523 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0074e0 con 0x7f1104140310 2023-11-18T20:42:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f10faffd640 1 -- 172.21.15.60:0/3660749523 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec007660 con 0x7f1104140310 2023-11-18T20:42:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f10faffd640 1 -- 172.21.15.60:0/3660749523 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec004a20 con 0x7f1104140310 2023-11-18T20:42:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f1108bf5640 1 -- 172.21.15.60:0/3660749523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1104144a60 con 0x7f1104140310 2023-11-18T20:42:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.634+0000 7f1108bf5640 1 -- 172.21.15.60:0/3660749523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1104146440 con 0x7f1104140310 2023-11-18T20:42:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.635+0000 7f10faffd640 1 -- 172.21.15.60:0/3660749523 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f10ec004b80 con 0x7f1104140310 2023-11-18T20:42:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.636+0000 7f10faffd640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10e406ae40 0x7f10e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.636+0000 7f10faffd640 1 -- 172.21.15.60:0/3660749523 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10ec0887a0 con 0x7f1104140310 2023-11-18T20:42:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.636+0000 7f110259c640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10e406ae40 0x7f10e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.637+0000 7f110259c640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10e406ae40 0x7f10e406d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f10fc036000 tx=0x7f10fc008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.637+0000 7f110b789740 1 Processor -- start 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.637+0000 7f110b789740 1 -- start start 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110b789740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121637b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110b789740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 0x560121667920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110b789740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121638070 0x560121667e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110b789740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601215f88c0 con 0x560121664b90 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110b789740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601215aab90 con 0x560121664f70 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110b789740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601215a8280 con 0x560121638070 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121637b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121637b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57712/0 (socket says 172.21.15.60:57712) 2023-11-18T20:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110259c640 1 -- 172.21.15.60:0/1449007204 learned_addr learned my addr 172.21.15.60:0/1449007204 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f1101d9b640 1 --2- 172.21.15.60:0/1449007204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 0x560121667920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f1102d9d640 1 --2- 172.21.15.60:0/1449007204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121638070 0x560121667e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110259c640 1 -- 172.21.15.60:0/1449007204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121638070 msgr2=0x560121667e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110259c640 1 --2- 172.21.15.60:0/1449007204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121638070 0x560121667e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110259c640 1 -- 172.21.15.60:0/1449007204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 msgr2=0x560121667920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110259c640 1 --2- 172.21.15.60:0/1449007204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 0x560121667920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110259c640 1 -- 172.21.15.60:0/1449007204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560121569500 con 0x560121664b90 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f110259c640 1 --2- 172.21.15.60:0/1449007204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121637b30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009860 tx=0x7f10fc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b2131e9942ce55 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f10e3fff640 1 -- 172.21.15.60:0/1449007204 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc04c070 con 0x560121664b90 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f10e3fff640 1 -- 172.21.15.60:0/1449007204 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc031e40 con 0x560121664b90 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.638+0000 7f10e3fff640 1 -- 172.21.15.60:0/1449007204 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc040070 con 0x560121664b90 2023-11-18T20:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.639+0000 7f110b789740 1 -- 172.21.15.60:0/1449007204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 msgr2=0x560121637b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.639+0000 7f110b789740 1 --2- 172.21.15.60:0/1449007204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121637b30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009860 tx=0x7f10fc030000 comp rx=0 tx=0).stop 2023-11-18T20:42:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.639+0000 7f110b789740 1 -- 172.21.15.60:0/1449007204 shutdown_connections 2023-11-18T20:42:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.639+0000 7f110b789740 1 --2- 172.21.15.60:0/1449007204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121638070 0x560121667e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.639+0000 7f110b789740 1 --2- 172.21.15.60:0/1449007204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 0x560121667920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.639+0000 7f110b789740 1 --2- 172.21.15.60:0/1449007204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121637b30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.639+0000 7f110b789740 1 -- 172.21.15.60:0/1449007204 >> 172.21.15.60:0/1449007204 conn(0x560121639da0 msgr2=0x560121682920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.639+0000 7f110b789740 1 -- 172.21.15.60:0/1449007204 shutdown_connections 2023-11-18T20:42:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.639+0000 7f110b789740 1 -- 172.21.15.60:0/1449007204 wait complete. 2023-11-18T20:42:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.640+0000 7f110b789740 1 -- start start 2023-11-18T20:42:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.640+0000 7f110b789740 1 Processor -- start 2023-11-18T20:42:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f110b789740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121664b90 0x5601216847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f110b789740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664f70 0x560121682a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f110b789740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601216804f0 0x560121680970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f110b789740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601215a3b70 con 0x560121664f70 2023-11-18T20:42:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f110b789740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56012155de00 con 0x5601216804f0 2023-11-18T20:42:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f110b789740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56012155daa0 con 0x560121664b90 2023-11-18T20:42:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f110b789740 10 client.0 _ll_register_callbacks cb 0x5601212b5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f1101d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664f70 0x560121682a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f1101d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664f70 0x560121682a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57722/0 (socket says 172.21.15.60:57722) 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f1101d9b640 1 -- 172.21.15.60:0/3150594318 learned_addr learned my addr 172.21.15.60:0/3150594318 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f110259c640 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121664b90 0x5601216847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f1102d9d640 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601216804f0 0x560121680970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f1101d9b640 1 -- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121664b90 msgr2=0x5601216847c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f1101d9b640 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121664b90 0x5601216847c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f1101d9b640 1 -- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601216804f0 msgr2=0x560121680970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f1101d9b640 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601216804f0 0x560121680970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f1101d9b640 1 -- 172.21.15.60:0/3150594318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1104142de0 con 0x560121664f70 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f1101d9b640 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664f70 0x560121682a20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0539c0 tx=0x7f10ec05b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2174fe968afe5 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f10e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-18T20:42:50.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f10e17fa640 1 -- 172.21.15.60:0/3150594318 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec05f580 con 0x560121664f70 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f10e17fa640 1 -- 172.21.15.60:0/3150594318 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10ec05d440 con 0x560121664f70 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.641+0000 7f110b789740 1 -- 172.21.15.60:0/3150594318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56012159cb50 con 0x560121664f70 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f10e17fa640 1 -- 172.21.15.60:0/3150594318 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10ec05d600 con 0x560121664f70 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f110b789740 10 client.4638 fetch_fsmap learned FSMap version 6 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f110b789740 1 -- 172.21.15.60:0/3150594318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x5601215a21f0 con 0x560121664f70 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f10e17fa640 1 -- 172.21.15.60:0/3150594318 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56012159cb50 con 0x560121664f70 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f10e17fa640 1 -- 172.21.15.60:0/3150594318 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10ec05b130 con 0x560121664f70 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f110b789740 10 client.4638 fetch_fsmap finished waiting for FSMap version 6 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f110b789740 10 client.4638 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f110b789740 20 client.4638 populate_metadata read hostname 'smithi060' 2023-11-18T20:42:50.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f110b789740 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:6835/1851816776,v1:172.21.15.100:6837/1851816776] conn(0x56012166b7c0 0x56012167dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f110b789740 4 client.4638 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f110b789740 1 -- 172.21.15.60:0/3150594318 --> [v2:172.21.15.100:6835/1851816776,v1:172.21.15.100:6837/1851816776] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56012159c870 con 0x56012166b7c0 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f110259c640 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:6835/1851816776,v1:172.21.15.100:6837/1851816776] conn(0x56012166b7c0 0x56012167dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f110259c640 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:6835/1851816776,v1:172.21.15.100:6837/1851816776] conn(0x56012166b7c0 0x56012167dc40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d5d7fa7356fb87 server_cookie=f0107750d1d4b8cc in_seq=0 out_seq=0 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.642+0000 7f10e17fa640 10 client.4638 ms_handle_connect on v2:172.21.15.100:6835/1851816776 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f10e17fa640 1 -- 172.21.15.60:0/3150594318 <== mds.0 v2:172.21.15.100:6835/1851816776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56012159c870 con 0x56012166b7c0 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f10e17fa640 10 client.4638 handle_command_reply: tid=0 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 client.4638 shutdown 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 -- 172.21.15.60:0/3150594318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664f70 msgr2=0x560121682a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664f70 0x560121682a20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0539c0 tx=0x7f10ec05b660 comp rx=0 tx=0).stop 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 -- 172.21.15.60:0/3150594318 shutdown_connections 2023-11-18T20:42:50.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:6835/1851816776,v1:172.21.15.100:6837/1851816776] conn(0x56012166b7c0 0x56012167dc40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5601216804f0 0x560121680970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664f70 0x560121682a20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 --2- 172.21.15.60:0/3150594318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121664b90 0x5601216847c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f10e17fa640 0 client.4638 ms_handle_reset on v2:172.21.15.100:6835/1851816776 2023-11-18T20:42:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 -- 172.21.15.60:0/3150594318 >> 172.21.15.60:0/3150594318 conn(0x560121638070 msgr2=0x56012163bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 -- 172.21.15.60:0/3150594318 shutdown_connections 2023-11-18T20:42:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.643+0000 7f110b789740 1 -- 172.21.15.60:0/3150594318 wait complete. 2023-11-18T20:42:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.644+0000 7f110b789740 20 client.4638 trim_cache size 0 max 16384 2023-11-18T20:42:50.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.659+0000 7f110b789740 1 Processor -- start 2023-11-18T20:42:50.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.659+0000 7f110b789740 1 -- start start 2023-11-18T20:42:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f110b789740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664b90 0x5601216a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f110b789740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121664f70 0x560121667c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f110b789740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5601216804f0 0x560121685e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f110b789740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560121523270 con 0x5601216804f0 2023-11-18T20:42:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f110b789740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56012167fae0 con 0x560121664b90 2023-11-18T20:42:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f110b789740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560121639850 con 0x560121664f70 2023-11-18T20:42:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f1102d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5601216804f0 0x560121685e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f1102d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5601216804f0 0x560121685e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57728/0 (socket says 172.21.15.60:57728) 2023-11-18T20:42:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f1102d9d640 1 -- 172.21.15.60:0/2677955766 learned_addr learned my addr 172.21.15.60:0/2677955766 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:50.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f110259c640 1 --2- 172.21.15.60:0/2677955766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664b90 0x5601216a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f1101d9b640 1 --2- 172.21.15.60:0/2677955766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121664f70 0x560121667c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f1102d9d640 1 -- 172.21.15.60:0/2677955766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121664f70 msgr2=0x560121667c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f1102d9d640 1 --2- 172.21.15.60:0/2677955766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121664f70 0x560121667c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f1102d9d640 1 -- 172.21.15.60:0/2677955766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664b90 msgr2=0x5601216a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f1102d9d640 1 --2- 172.21.15.60:0/2677955766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664b90 0x5601216a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f1102d9d640 1 -- 172.21.15.60:0/2677955766 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560121682540 con 0x5601216804f0 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.660+0000 7f1102d9d640 1 --2- 172.21.15.60:0/2677955766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5601216804f0 0x560121685e30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10f4007e60 tx=0x7f10f4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94831c390559b75b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f10f8ff9640 1 -- 172.21.15.60:0/2677955766 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4003c60 con 0x5601216804f0 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f10f8ff9640 1 -- 172.21.15.60:0/2677955766 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4003dc0 con 0x5601216804f0 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f10f8ff9640 1 -- 172.21.15.60:0/2677955766 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4004230 con 0x5601216804f0 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f110b789740 1 -- 172.21.15.60:0/2677955766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5601216804f0 msgr2=0x560121685e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f110b789740 1 --2- 172.21.15.60:0/2677955766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5601216804f0 0x560121685e30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10f4007e60 tx=0x7f10f4012820 comp rx=0 tx=0).stop 2023-11-18T20:42:50.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f110b789740 1 -- 172.21.15.60:0/2677955766 shutdown_connections 2023-11-18T20:42:50.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f110b789740 1 --2- 172.21.15.60:0/2677955766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5601216804f0 0x560121685e30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f110b789740 1 --2- 172.21.15.60:0/2677955766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560121664f70 0x560121667c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f110b789740 1 --2- 172.21.15.60:0/2677955766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664b90 0x5601216a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f110b789740 1 -- 172.21.15.60:0/2677955766 >> 172.21.15.60:0/2677955766 conn(0x560121638070 msgr2=0x5601216845d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:50.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f110b789740 1 -- 172.21.15.60:0/2677955766 shutdown_connections 2023-11-18T20:42:50.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.661+0000 7f110b789740 1 -- 172.21.15.60:0/2677955766 wait complete. 2023-11-18T20:42:50.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.662+0000 7f110b789740 1 -- start start 2023-11-18T20:42:50.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.662+0000 7f110b789740 1 Processor -- start 2023-11-18T20:42:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110b789740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121685e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110b789740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 0x560121686370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110b789740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5601216804f0 0x560121698f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110b789740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601216811a0 con 0x560121664b90 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110b789740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56012166bba0 con 0x560121664f70 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110b789740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56012169c010 con 0x5601216804f0 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110b789740 10 client.0 _ll_register_callbacks cb 0x7f10fc009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121685e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121685e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57732/0 (socket says 172.21.15.60:57732) 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110259c640 1 -- 172.21.15.60:0/1892505109 learned_addr learned my addr 172.21.15.60:0/1892505109 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f1102d9d640 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5601216804f0 0x560121698f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110259c640 1 -- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5601216804f0 msgr2=0x560121698f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f1101d9b640 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 0x560121686370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110259c640 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5601216804f0 0x560121698f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110259c640 1 -- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 msgr2=0x560121686370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110259c640 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 0x560121686370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110259c640 1 -- 172.21.15.60:0/1892505109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f4008040 con 0x560121664b90 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f1101d9b640 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 0x560121686370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110259c640 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121685e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f10fc031bd0 tx=0x7f10fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217c14892c60b03f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f10e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f10e17fa640 1 -- 172.21.15.60:0/1892505109 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc064070 con 0x560121664b90 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.663+0000 7f110b789740 1 -- 172.21.15.60:0/1892505109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10fc031e40 con 0x560121664b90 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f10e17fa640 1 -- 172.21.15.60:0/1892505109 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10fc032b50 con 0x560121664b90 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f10e17fa640 1 -- 172.21.15.60:0/1892505109 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10fc04c040 con 0x560121664b90 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f110b789740 10 client.4644 fetch_fsmap learned FSMap version 6 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f110b789740 1 -- 172.21.15.60:0/1892505109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x56012153fd00 con 0x560121664b90 2023-11-18T20:42:50.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f10e17fa640 1 -- 172.21.15.60:0/1892505109 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc031e40 con 0x560121664b90 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f10e17fa640 1 -- 172.21.15.60:0/1892505109 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10fc067020 con 0x560121664b90 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f110b789740 10 client.4644 fetch_fsmap finished waiting for FSMap version 6 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f110b789740 10 client.4644 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f110b789740 20 client.4644 populate_metadata read hostname 'smithi060' 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f110b789740 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:6835/1851816776,v1:172.21.15.100:6837/1851816776] conn(0x560121663d90 0x560121699440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f110b789740 4 client.4644 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f110b789740 1 -- 172.21.15.60:0/1892505109 --> [v2:172.21.15.100:6835/1851816776,v1:172.21.15.100:6837/1851816776] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5601212a46c0 con 0x560121663d90 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.664+0000 7f1101d9b640 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:6835/1851816776,v1:172.21.15.100:6837/1851816776] conn(0x560121663d90 0x560121699440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f10e17fa640 10 client.4644 ms_handle_connect on v2:172.21.15.100:6835/1851816776 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f1101d9b640 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:6835/1851816776,v1:172.21.15.100:6837/1851816776] conn(0x560121663d90 0x560121699440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd8f2885ad46163 server_cookie=25d9b8c15a7de8c in_seq=0 out_seq=0 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f10e17fa640 1 -- 172.21.15.60:0/1892505109 <== mds.0 v2:172.21.15.100:6835/1851816776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5601212a46c0 con 0x560121663d90 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f10e17fa640 10 client.4644 handle_command_reply: tid=0 2023-11-18T20:42:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 client.4644 shutdown 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 -- 172.21.15.60:0/1892505109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 msgr2=0x560121685e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121685e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f10fc031bd0 tx=0x7f10fc004730 comp rx=0 tx=0).stop 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 -- 172.21.15.60:0/1892505109 shutdown_connections 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:6835/1851816776,v1:172.21.15.100:6837/1851816776] conn(0x560121663d90 0x560121699440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5601216804f0 0x560121698f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560121664f70 0x560121686370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f10e17fa640 0 client.4644 ms_handle_reset on v2:172.21.15.100:6835/1851816776 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 --2- 172.21.15.60:0/1892505109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560121664b90 0x560121685e30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 -- 172.21.15.60:0/1892505109 >> 172.21.15.60:0/1892505109 conn(0x560121638070 msgr2=0x560121684e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 -- 172.21.15.60:0/1892505109 shutdown_connections 2023-11-18T20:42:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 1 -- 172.21.15.60:0/1892505109 wait complete. 2023-11-18T20:42:50.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.665+0000 7f110b789740 20 client.4644 trim_cache size 0 max 16384 2023-11-18T20:42:50.674 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:42:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "ae4682cb-f89a-4edd-b6d6-55eae03fd1b8", 2023-11-18T20:42:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-11-18T20:42:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4303, 2023-11-18T20:42:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-11-18T20:42:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-11-18T20:42:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-11-18T20:42:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 12.553086376, 2023-11-18T20:42:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 6, 2023-11-18T20:42:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 21, 2023-11-18T20:42:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 21, 2023-11-18T20:42:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 19.588491824999998 2023-11-18T20:42:50.675 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:42:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 -- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10e406ae40 msgr2=0x7f10e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10e406ae40 0x7f10e406d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f10fc036000 tx=0x7f10fc008030 comp rx=0 tx=0).stop 2023-11-18T20:42:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 -- 172.21.15.60:0/3660749523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 msgr2=0x7f11041f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 0x7f11041f3920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00c4d0 tx=0x7f10ec00c990 comp rx=0 tx=0).stop 2023-11-18T20:42:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 -- 172.21.15.60:0/3660749523 shutdown_connections 2023-11-18T20:42:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10e406ae40 0x7f10e406d300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11041409d0 0x7f1104145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1104140310 0x7f11041f3920 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 --2- 172.21.15.60:0/3660749523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f110406abe0 0x7f1104143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 -- 172.21.15.60:0/3660749523 >> 172.21.15.60:0/3660749523 conn(0x7f11040b1e50 msgr2=0x7f11040733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 -- 172.21.15.60:0/3660749523 shutdown_connections 2023-11-18T20:42:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:50.666+0000 7f1108bf5640 1 -- 172.21.15.60:0/3660749523 wait complete. 2023-11-18T20:42:50.678 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-18T20:42:50.699 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-18T20:42:50.700 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-18T20:42:50.700 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-18T20:42:50.700 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-18T20:42:50.700 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-18T20:42:50.700 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-18T20:42:50.700 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-18T20:42:50.700 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-18T20:42:50.700 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-18T20:42:50.700 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-18T20:42:50.701 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-18T20:42:50.701 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-18T20:42:50.701 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-18T20:42:50.701 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-18T20:42:50.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:50.701 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2023-11-18T20:42:50.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:50.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2023-11-18T20:42:50.794 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2023-11-18T20:42:50.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:42:50.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:50.796 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-11-18T20:42:50.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:50.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2023-11-18T20:42:50.885 INFO:teuthology.orchestra.run.smithi060.stderr:Cannot find device "ceph-brx" 2023-11-18T20:42:50.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:42:50.886 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-18T20:42:50.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-18T20:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.032+0000 7f5a6f32c640 1 Processor -- start 2023-11-18T20:42:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.032+0000 7f5a6f32c640 1 -- start start 2023-11-18T20:42:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.033+0000 7f5a6f32c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5a68150560 0x7f5a680b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.033+0000 7f5a6f32c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 0x7f5a6815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.033+0000 7f5a6f32c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a680b49f0 con 0x7f5a68150940 2023-11-18T20:42:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.033+0000 7f5a6f32c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a680b3de0 con 0x7f5a68150560 2023-11-18T20:42:51.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.033+0000 7f5a6f32c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680b40c0 con 0x7f5a680b3390 2023-11-18T20:42:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6c8c7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 0x7f5a6815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6d0c8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5a68150560 0x7f5a680b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6d8c9640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5a68150940 0x7f5a680ce0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:35208/0 (socket says 172.21.15.60:35208) 2023-11-18T20:42:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6d0c8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5a68150560 0x7f5a680b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38266/0 (socket says 172.21.15.60:38266) 2023-11-18T20:42:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6c8c7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 0x7f5a6815d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:57388/0 (socket says 172.21.15.60:57388) 2023-11-18T20:42:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6d8c9640 1 -- 172.21.15.60:0/2989479128 learned_addr learned my addr 172.21.15.60:0/2989479128 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:51.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6d0c8640 1 -- 172.21.15.60:0/2989479128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 msgr2=0x7f5a6815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6d0c8640 1 --2- 172.21.15.60:0/2989479128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 0x7f5a6815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6d0c8640 1 -- 172.21.15.60:0/2989479128 >> v1:172.21.15.100:6789/0 conn(0x7f5a68150940 legacy=0x7f5a680ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a67fff640 1 -- 172.21.15.60:0/2989479128 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172469973 0 0) 0x7f5a680b49f0 con 0x7f5a68150940 2023-11-18T20:42:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6d0c8640 1 -- 172.21.15.60:0/2989479128 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680b43a0 con 0x7f5a68150560 2023-11-18T20:42:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.034+0000 7f5a6c8c7640 1 --2- 172.21.15.60:0/2989479128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 0x7f5a6815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:42:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.035+0000 7f5a6d0c8640 1 --2- 172.21.15.60:0/2989479128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 0x7f5a680b2e50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5a600097a0 tx=0x7f5a6002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5564442bfddf96e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.035+0000 7f5a67fff640 1 -- 172.21.15.60:0/2989479128 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60031830 con 0x7f5a68150560 2023-11-18T20:42:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.035+0000 7f5a67fff640 1 -- 172.21.15.60:0/2989479128 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60031990 con 0x7f5a68150560 2023-11-18T20:42:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.035+0000 7f5a67fff640 1 -- 172.21.15.60:0/2989479128 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60031d20 con 0x7f5a68150560 2023-11-18T20:42:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.035+0000 7f5a6f32c640 1 -- 172.21.15.60:0/2989479128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 msgr2=0x7f5a680b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.035+0000 7f5a6f32c640 1 --2- 172.21.15.60:0/2989479128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 0x7f5a680b2e50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5a600097a0 tx=0x7f5a6002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:42:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.035+0000 7f5a6f32c640 1 -- 172.21.15.60:0/2989479128 shutdown_connections 2023-11-18T20:42:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.035+0000 7f5a6f32c640 1 --2- 172.21.15.60:0/2989479128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 0x7f5a6815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.035+0000 7f5a6f32c640 1 --2- 172.21.15.60:0/2989479128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 0x7f5a680b2e50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.036+0000 7f5a6f32c640 1 -- 172.21.15.60:0/2989479128 >> 172.21.15.60:0/2989479128 conn(0x7f5a6806a4d0 msgr2=0x7f5a6806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.036+0000 7f5a6f32c640 1 -- 172.21.15.60:0/2989479128 shutdown_connections 2023-11-18T20:42:51.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.036+0000 7f5a6f32c640 1 -- 172.21.15.60:0/2989479128 wait complete. 2023-11-18T20:42:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.037+0000 7f5a6f32c640 1 Processor -- start 2023-11-18T20:42:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.037+0000 7f5a6f32c640 1 -- start start 2023-11-18T20:42:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.037+0000 7f5a6f32c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 0x7f5a6815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.037+0000 7f5a6f32c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 0x7f5a6814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.037+0000 7f5a6f32c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6814fa50 0x7f5a681527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.037+0000 7f5a6f32c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6814ff00 con 0x7f5a6814fa50 2023-11-18T20:42:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.037+0000 7f5a6f32c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6813a0f0 con 0x7f5a68150560 2023-11-18T20:42:51.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.037+0000 7f5a6f32c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68140d80 con 0x7f5a680b3390 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6d8c9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6814fa50 0x7f5a681527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6c8c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 0x7f5a6814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6d0c8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 0x7f5a6815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6c8c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 0x7f5a6814f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38280/0 (socket says 172.21.15.60:38280) 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6c8c7640 1 -- 172.21.15.60:0/277653461 learned_addr learned my addr 172.21.15.60:0/277653461 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6c8c7640 1 -- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 msgr2=0x7f5a6815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6c8c7640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 0x7f5a6815c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6c8c7640 1 -- 172.21.15.60:0/277653461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6814fa50 msgr2=0x7f5a681527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6c8c7640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6814fa50 0x7f5a681527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6c8c7640 1 -- 172.21.15.60:0/277653461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c009400 con 0x7f5a68150560 2023-11-18T20:42:51.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6c8c7640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 0x7f5a6814f510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00ec10 tx=0x7f5a5c00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a65ffb640 1 -- 172.21.15.60:0/277653461 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c009760 con 0x7f5a68150560 2023-11-18T20:42:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a65ffb640 1 -- 172.21.15.60:0/277653461 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c00cc30 con 0x7f5a68150560 2023-11-18T20:42:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a6f32c640 1 -- 172.21.15.60:0/277653461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a60009400 con 0x7f5a68150560 2023-11-18T20:42:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.038+0000 7f5a65ffb640 1 -- 172.21.15.60:0/277653461 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c013470 con 0x7f5a68150560 2023-11-18T20:42:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.039+0000 7f5a6f32c640 1 -- 172.21.15.60:0/277653461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a68140fd0 con 0x7f5a68150560 2023-11-18T20:42:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.039+0000 7f5a6f32c640 1 -- 172.21.15.60:0/277653461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a680b4e60 con 0x7f5a68150560 2023-11-18T20:42:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.040+0000 7f5a65ffb640 1 -- 172.21.15.60:0/277653461 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a5c005990 con 0x7f5a68150560 2023-11-18T20:42:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.040+0000 7f5a65ffb640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a5406ae70 0x7f5a5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.041+0000 7f5a65ffb640 1 -- 172.21.15.60:0/277653461 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a5c0897a0 con 0x7f5a68150560 2023-11-18T20:42:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.041+0000 7f5a6d0c8640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a5406ae70 0x7f5a5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.041+0000 7f5a6d0c8640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a5406ae70 0x7f5a5406d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5a60036000 tx=0x7f5a6003e040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:51.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.045+0000 7f5a65ffb640 1 -- 172.21.15.60:0/277653461 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a5c0548e0 con 0x7f5a68150560 2023-11-18T20:42:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.262+0000 7f5a6f32c640 1 -- 172.21.15.60:0/277653461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5a680b4e60 con 0x7f5a68150560 2023-11-18T20:42:51.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.263+0000 7f5a65ffb640 1 -- 172.21.15.60:0/277653461 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5a5c017060 con 0x7f5a68150560 2023-11-18T20:42:51.265 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-18T20:42:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 -- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a5406ae70 msgr2=0x7f5a5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a5406ae70 0x7f5a5406d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5a60036000 tx=0x7f5a6003e040 comp rx=0 tx=0).stop 2023-11-18T20:42:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 -- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 msgr2=0x7f5a6814f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 0x7f5a6814f510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00ec10 tx=0x7f5a5c00c520 comp rx=0 tx=0).stop 2023-11-18T20:42:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 -- 172.21.15.60:0/277653461 shutdown_connections 2023-11-18T20:42:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a5406ae70 0x7f5a5406d330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6814fa50 0x7f5a681527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a68150560 0x7f5a6814f510 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 --2- 172.21.15.60:0/277653461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b3390 0x7f5a6815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 -- 172.21.15.60:0/277653461 >> 172.21.15.60:0/277653461 conn(0x7f5a6806a4d0 msgr2=0x7f5a680b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 -- 172.21.15.60:0/277653461 shutdown_connections 2023-11-18T20:42:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:51.265+0000 7f5a6f32c640 1 -- 172.21.15.60:0/277653461 wait complete. 2023-11-18T20:42:51.277 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-18T20:42:51.277 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-18T20:42:51.277 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-11-18T20:42:51.277 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-18T20:42:51.277 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-18T20:42:51.278 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-18T20:42:51.278 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-18T20:42:51.278 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-18T20:42:51.278 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:42:51.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:51.278 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-11-18T20:42:51.294 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:42:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:42:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:42:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-11-18T20:42:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:51.294 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-11-18T20:42:51.294 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42685sec preferred_lft 42685sec 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-11-18T20:42:51.296 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-18T20:42:51.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:51.296 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:42:51.296 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:42:51.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2023-11-18T20:42:51.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2023-11-18T20:42:51.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2023-11-18T20:42:51.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-18T20:42:51.297 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:42:51.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:51.487 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-18T20:42:51.517 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-11-18T20:42:51.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:51.519 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-11-18T20:42:51.591 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-11-18T20:42:51.592 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-18T20:42:51.592 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-18T20:42:51.592 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-18T20:42:51.592 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-18T20:42:51.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:51.592 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:42:51.592 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:42:51.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-18T20:42:51.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-18T20:42:51.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-18T20:42:51.593 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:42:51.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:51.849 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-11-18T20:42:51.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:51.865 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2023-11-18T20:42:51.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:51.921 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:42:51.921 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:42:51.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:42:51.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-18T20:42:51.921 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:42:52.009 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-18T20:42:52.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:52.010 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:42:52.010 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:42:52.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-18T20:42:52.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-18T20:42:52.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-18T20:42:52.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-18T20:42:52.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-18T20:42:52.010 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:42:52.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:52.125 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:42:52.125 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:42:52.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-11-18T20:42:52.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2023-11-18T20:42:52.125 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:42:52.175 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-18T20:42:52.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:42:52.175 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-18T20:42:52.189 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:42:52.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:42:52.190 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:42:52.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:42:52.316 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:42:52.373 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:42:52.373 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:42:52.373 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:42:52.373 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:42:52.374 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:42:52.375 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:42:52.375 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:42:52.375 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:42:52.375 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:42:52.375 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:42:52.375 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:42:52.375 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:42:52.375 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:42:52.375 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:42:52.375 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:42:52.376 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:42:52.376 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:42:52.376 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:42:52.376 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:42:52.376 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:42:52.376 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:42:52.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:52.376 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:42:52.429 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-18T20:42:52.429 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-18T20:42:52.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:42:52.502 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:42:52.558 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:42:52.558 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:42:52.558 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:42:52.558 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:42:52.558 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:42:52.558 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:42:52.558 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:42:52.558 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:42:52.558 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:42:52.558 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:42:52.559 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:42:52.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:52.561 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:42:52.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-11-18T20:42:52.570+0000 7fd1a4a1f480 -1 init, newargv = 0x55ad1778f350 newargc=15 2023-11-18T20:42:52.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[95584]: starting ceph client 2023-11-18T20:42:52.576 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[95584]: starting fuse 2023-11-18T20:42:52.586 INFO:teuthology.orchestra.run.smithi060.stdout:41 2023-11-18T20:42:52.586 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-18T20:42:52.587 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2023-11-18T20:42:52.587 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-11-18T20:42:52.587 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-11-18T20:42:52.587 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-11-18T20:42:52.587 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-11-18T20:42:52.587 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:42:52.587 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-11-18T20:42:52.587 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-18T20:42:52.587 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-11-18T20:42:52.588 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-18T20:42:52.588 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:42:52.588 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-11-18T20:42:52.588 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-11-18T20:42:52.588 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-11-18T20:42:52.588 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-11-18T20:42:52.588 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:42:52.589 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-11-18T20:42:52.589 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-18T20:42:52.589 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-18T20:42:52.589 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-18T20:42:52.589 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-11-18T20:42:52.589 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-11-18T20:42:52.589 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-11-18T20:42:52.590 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-18T20:42:52.590 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:42:52.590 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-11-18T20:42:52.590 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-11-18T20:42:52.712 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.95584.asok 2023-11-18T20:42:52.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:42:52.722 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95584.asok 2023-11-18T20:42:52.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:52.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95584.asok status 2023-11-18T20:42:52.864 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe", 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-7418-gb0f45a51 (b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe) reef (dev)", 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "95584", 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-11-18T20:42:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4653, 2023-11-18T20:42:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-11-18T20:42:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-11-18T20:42:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-11-18T20:42:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 4653 2023-11-18T20:42:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:42:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:42:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:42:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:42:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1269341982 2023-11-18T20:42:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:42:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:42:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:42:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:42:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:42:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1269341982 2023-11-18T20:42:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:42:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.4653 192.168.0.1:0/1269341982", 2023-11-18T20:42:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/1269341982", 2023-11-18T20:42:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-11-18T20:42:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 6, 2023-11-18T20:42:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 21, 2023-11-18T20:42:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-11-18T20:42:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-11-18T20:42:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-11-18T20:42:52.868 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:42:52.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-18T20:42:52.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.977+0000 7f4c7b0a2640 1 Processor -- start 2023-11-18T20:42:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.977+0000 7f4c7b0a2640 1 -- start start 2023-11-18T20:42:52.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.978+0000 7f4c7b0a2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4c7415d910 0x7f4c740ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:52.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.978+0000 7f4c7b0a2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c740b49f0 con 0x7f4c74150940 2023-11-18T20:42:52.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.978+0000 7f4c7b0a2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c740b3560 con 0x7f4c74150560 2023-11-18T20:42:52.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.978+0000 7f4c7b0a2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c740b3de0 con 0x7f4c7415d910 2023-11-18T20:42:52.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.978+0000 7f4c73fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4c74150940 0x7f4c7415d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56880/0 (socket says 172.21.15.60:56880) 2023-11-18T20:42:52.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.978+0000 7f4c73fff640 1 -- 172.21.15.60:0/2498533907 learned_addr learned my addr 172.21.15.60:0/2498533907 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:52.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.978+0000 7f4c7963f640 1 --2- 172.21.15.60:0/2498533907 >> v2:172.21.15.100:3300/0 conn(0x7f4c7415d910 0x7f4c740ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.979+0000 7f4c737fe640 1 -- 172.21.15.60:0/2498533907 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664446285 0 0) 0x7f4c740b49f0 con 0x7f4c74150940 2023-11-18T20:42:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.979+0000 7f4c737fe640 1 -- 172.21.15.60:0/2498533907 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c5c003520 con 0x7f4c74150940 2023-11-18T20:42:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.979+0000 7f4c7963f640 1 -- 172.21.15.60:0/2498533907 >> v1:172.21.15.100:6789/0 conn(0x7f4c74150560 legacy=0x7f4c740b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.979+0000 7f4c737fe640 1 -- 172.21.15.60:0/2498533907 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1827126912 0 0) 0x7f4c5c003520 con 0x7f4c74150940 2023-11-18T20:42:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.979+0000 7f4c7963f640 1 -- 172.21.15.60:0/2498533907 >> v1:172.21.15.60:6789/0 conn(0x7f4c74150940 legacy=0x7f4c7415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.979+0000 7f4c7963f640 1 -- 172.21.15.60:0/2498533907 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740b40c0 con 0x7f4c7415d910 2023-11-18T20:42:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c7963f640 1 --2- 172.21.15.60:0/2498533907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c7415d910 0x7f4c740ce0e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4c640097a0 tx=0x7f4c6402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5294c66400bf445 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c737fe640 1 -- 172.21.15.60:0/2498533907 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6402f050 con 0x7f4c7415d910 2023-11-18T20:42:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c737fe640 1 -- 172.21.15.60:0/2498533907 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64031500 con 0x7f4c7415d910 2023-11-18T20:42:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c737fe640 1 -- 172.21.15.60:0/2498533907 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68001670 con 0x7f4c7415d910 2023-11-18T20:42:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2498533907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c7415d910 msgr2=0x7f4c740ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c7b0a2640 1 --2- 172.21.15.60:0/2498533907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c7415d910 0x7f4c740ce0e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4c640097a0 tx=0x7f4c6402b820 comp rx=0 tx=0).stop 2023-11-18T20:42:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2498533907 shutdown_connections 2023-11-18T20:42:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c7b0a2640 1 --2- 172.21.15.60:0/2498533907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c7415d910 0x7f4c740ce0e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2498533907 >> 172.21.15.60:0/2498533907 conn(0x7f4c7406a4d0 msgr2=0x7f4c7406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:52.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2498533907 shutdown_connections 2023-11-18T20:42:52.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.980+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2498533907 wait complete. 2023-11-18T20:42:52.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.981+0000 7f4c7b0a2640 1 Processor -- start 2023-11-18T20:42:52.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.981+0000 7f4c7b0a2640 1 -- start start 2023-11-18T20:42:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.982+0000 7f4c7b0a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c74150560 0x7f4c741559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.982+0000 7f4c7b0a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c7415d910 0x7f4c7413d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.982+0000 7f4c7b0a2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c74156e40 0x7f4c74152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.982+0000 7f4c7b0a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74156240 con 0x7f4c7415d910 2023-11-18T20:42:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.982+0000 7f4c7b0a2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74156520 con 0x7f4c74150560 2023-11-18T20:42:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.982+0000 7f4c7b0a2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74151f70 con 0x7f4c74156e40 2023-11-18T20:42:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.982+0000 7f4c73fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c7415d910 0x7f4c7413d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:52.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.982+0000 7f4c73fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c7415d910 0x7f4c7413d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57756/0 (socket says 172.21.15.60:57756) 2023-11-18T20:42:52.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.982+0000 7f4c73fff640 1 -- 172.21.15.60:0/2182745999 learned_addr learned my addr 172.21.15.60:0/2182745999 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:52.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c78e3e640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c74150560 0x7f4c741559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:52.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c7963f640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c74156e40 0x7f4c74152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:52.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c73fff640 1 -- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c74156e40 msgr2=0x7f4c74152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:52.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c73fff640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c74156e40 0x7f4c74152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:52.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c73fff640 1 -- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c74150560 msgr2=0x7f4c741559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:52.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c73fff640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c74150560 0x7f4c741559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c73fff640 1 -- 172.21.15.60:0/2182745999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c64009400 con 0x7f4c7415d910 2023-11-18T20:42:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c73fff640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c7415d910 0x7f4c7413d9f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4c680011e0 tx=0x7f4c6800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c717fa640 1 -- 172.21.15.60:0/2182745999 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68005ad0 con 0x7f4c7415d910 2023-11-18T20:42:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c717fa640 1 -- 172.21.15.60:0/2182745999 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c68005c50 con 0x7f4c7415d910 2023-11-18T20:42:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2182745999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c74152250 con 0x7f4c7415d910 2023-11-18T20:42:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c717fa640 1 -- 172.21.15.60:0/2182745999 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c680072b0 con 0x7f4c7415d910 2023-11-18T20:42:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.983+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2182745999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7413df30 con 0x7f4c7415d910 2023-11-18T20:42:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.984+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2182745999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c74152490 con 0x7f4c7415d910 2023-11-18T20:42:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.985+0000 7f4c717fa640 1 -- 172.21.15.60:0/2182745999 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c68007410 con 0x7f4c7415d910 2023-11-18T20:42:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.985+0000 7f4c717fa640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4c6006ae40 0x7f4c6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:52.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.985+0000 7f4c717fa640 1 -- 172.21.15.60:0/2182745999 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c68088c80 con 0x7f4c7415d910 2023-11-18T20:42:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.986+0000 7f4c78e3e640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4c6006ae40 0x7f4c6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.986+0000 7f4c78e3e640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4c6006ae40 0x7f4c6006d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0012e0 tx=0x7f4c6c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:52.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:52.990+0000 7f4c717fa640 1 -- 172.21.15.60:0/2182745999 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c74152490 con 0x7f4c7415d910 2023-11-18T20:42:53.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.203+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2182745999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4c74152490 con 0x7f4c7415d910 2023-11-18T20:42:53.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.203+0000 7f4c717fa640 1 -- 172.21.15.60:0/2182745999 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4c74152490 con 0x7f4c7415d910 2023-11-18T20:42:53.205 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-18T20:42:53.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4c6006ae40 msgr2=0x7f4c6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:53.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4c6006ae40 0x7f4c6006d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0012e0 tx=0x7f4c6c009040 comp rx=0 tx=0).stop 2023-11-18T20:42:53.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2182745999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c7415d910 msgr2=0x7f4c7413d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:53.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c7415d910 0x7f4c7413d9f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4c680011e0 tx=0x7f4c6800cbe0 comp rx=0 tx=0).stop 2023-11-18T20:42:53.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2182745999 shutdown_connections 2023-11-18T20:42:53.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4c6006ae40 0x7f4c6006d300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:53.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c74156e40 0x7f4c74152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:53.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c7415d910 0x7f4c7413d9f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:53.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 --2- 172.21.15.60:0/2182745999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c74150560 0x7f4c741559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:53.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2182745999 >> 172.21.15.60:0/2182745999 conn(0x7f4c7406a4d0 msgr2=0x7f4c740b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:53.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2182745999 shutdown_connections 2023-11-18T20:42:53.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:53.205+0000 7f4c7b0a2640 1 -- 172.21.15.60:0/2182745999 wait complete. 2023-11-18T20:42:53.217 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-18T20:42:53.217 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-18T20:42:53.217 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-11-18T20:42:53.217 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-18T20:42:53.218 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-18T20:42:53.218 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-18T20:42:53.218 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-18T20:42:53.218 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-18T20:42:53.218 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-18T20:42:53.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.219 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-11-18T20:42:53.234 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:53.235 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42683sec preferred_lft 42683sec 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:42:53.236 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0a:c3:0c:c2:e6:72 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:53.237 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-18T20:42:53.237 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.237 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::d896:cfff:fef3:88dc/64 scope link tentative 2023-11-18T20:42:53.237 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.237 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:42:53.237 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0a:c3:0c:c2:e6:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:42:53.238 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8c3:cff:fec2:e672/64 scope link tentative 2023-11-18T20:42:53.238 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.238 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-18T20:42:53.304 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-11-18T20:42:53.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.306 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-11-18T20:42:53.362 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-11-18T20:42:53.362 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-18T20:42:53.362 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-18T20:42:53.362 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-18T20:42:53.362 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-18T20:42:53.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.363 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:42:53.363 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:42:53.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-18T20:42:53.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-18T20:42:53.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-18T20:42:53.364 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:42:53.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.468 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-11-18T20:42:53.483 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-18T20:42:53.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.484 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2023-11-18T20:42:53.539 INFO:teuthology.orchestra.run.smithi060.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-18T20:42:53.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.540 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:42:53.540 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:42:53.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:42:53.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-18T20:42:53.540 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:42:53.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-18T20:42:53.661 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:42:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:42:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:42:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-11-18T20:42:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.662 INFO:teuthology.orchestra.run.smithi060.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:42:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 1a:63:52:46:7c:7d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-18T20:42:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-18T20:42:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::1863:52ff:fe46:7c7d/64 scope link 2023-11-18T20:42:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-18T20:42:53.732 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:42:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:42:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:42:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-11-18T20:42:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.733 INFO:teuthology.orchestra.run.smithi060.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:42:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 1a:63:52:46:7c:7d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-18T20:42:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-18T20:42:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::1863:52ff:fe46:7c7d/64 scope link 2023-11-18T20:42:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:53.734 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-18T20:42:53.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.734 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:42:53.734 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:42:53.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-18T20:42:53.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-18T20:42:53.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-18T20:42:53.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-18T20:42:53.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-18T20:42:53.735 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:42:53.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:53.876 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:42:53.876 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:42:53.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.1 up 2023-11-18T20:42:53.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.1 master ceph-brx 2023-11-18T20:42:53.876 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:42:53.925 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-18T20:42:53.925 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:42:53.925 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-18T20:42:53.940 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-18T20:42:53.940 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:42:53.940 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-18T20:42:53.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:42:54.066 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:42:54.122 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:42:54.122 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:42:54.122 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:42:54.122 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:42:54.122 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:42:54.122 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:42:54.122 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:42:54.122 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:42:54.123 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:42:54.124 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:42:54.125 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-18T20:42:54.125 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:42:54.125 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:42:54.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:54.125 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:42:54.177 INFO:teuthology.orchestra.run.smithi060.stdout:41 2023-11-18T20:42:54.177 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-11-18T20:42:54.177 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-18T20:42:54.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:42:54.312 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi060.stderr:2023-11-18T20:42:54.311+0000 7f13c77fb480 -1 init, newargv = 0x55fffb8f27b0 newargc=15 2023-11-18T20:42:54.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi060.stderr:ceph-fuse[96052]: starting ceph client 2023-11-18T20:42:54.317 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi060.stderr:ceph-fuse[96052]: starting fuse 2023-11-18T20:42:54.331 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:42:54.387 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:42:54.387 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:42:54.387 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:42:54.388 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:42:54.388 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:42:54.388 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:42:54.388 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:42:54.388 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:42:54.388 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:42:54.389 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:42:54.389 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:42:54.389 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:42:54.389 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:42:54.389 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:42:54.390 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:42:54.391 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:42:54.391 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:42:54.391 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:42:54.391 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:42:54.391 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:42:54.391 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:42:54.391 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:42:54.391 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-18T20:42:54.391 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:42:54.391 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:42:54.392 INFO:teuthology.orchestra.run.smithi060.stdout:/home/ubuntu/cephtest/mnt.2 2023-11-18T20:42:54.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:54.392 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:42:54.442 INFO:teuthology.orchestra.run.smithi060.stdout:41 2023-11-18T20:42:54.442 INFO:teuthology.orchestra.run.smithi060.stdout:44 2023-11-18T20:42:54.443 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-11-18T20:42:54.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2023-11-18T20:42:54.443 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-11-18T20:42:54.443 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-11-18T20:42:54.443 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-11-18T20:42:54.443 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-11-18T20:42:54.443 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:42:54.443 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-11-18T20:42:54.443 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-18T20:42:54.443 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-11-18T20:42:54.444 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-11-18T20:42:54.445 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-11-18T20:42:54.445 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-18T20:42:54.445 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:42:54.445 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.2")) 2023-11-18T20:42:54.445 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-11-18T20:42:54.567 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.2.96052.asok 2023-11-18T20:42:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:42:54.577 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.96052.asok 2023-11-18T20:42:54.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:54.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.96052.asok status 2023-11-18T20:42:54.719 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe", 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-7418-gb0f45a51 (b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe) reef (dev)", 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "2", 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "96052", 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-11-18T20:42:54.720 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4665, 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 4665 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.2:0", 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2303241108 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:42:54.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:42:54.722 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:42:54.722 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:42:54.722 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.2:0", 2023-11-18T20:42:54.722 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2303241108 2023-11-18T20:42:54.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:42:54.722 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.4665 192.168.0.2:0/2303241108", 2023-11-18T20:42:54.722 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.2:0/2303241108", 2023-11-18T20:42:54.722 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-11-18T20:42:54.722 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 6, 2023-11-18T20:42:54.722 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 21, 2023-11-18T20:42:54.723 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-11-18T20:42:54.723 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-11-18T20:42:54.723 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-11-18T20:42:54.723 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:42:54.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-18T20:42:54.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.832+0000 7f20a410f640 1 Processor -- start 2023-11-18T20:42:54.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.832+0000 7f20a410f640 1 -- start start 2023-11-18T20:42:54.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.833+0000 7f20a410f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f209c15d340 0x7f209c06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:54.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.833+0000 7f20a410f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209c161ce0 con 0x7f209c0ad640 2023-11-18T20:42:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.833+0000 7f20a410f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209c15d720 con 0x7f209c0aeb80 2023-11-18T20:42:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.833+0000 7f20a410f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209c0adae0 con 0x7f209c15d340 2023-11-18T20:42:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.833+0000 7f20a1eab640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f209c15d340 0x7f209c06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.833+0000 7f20a1eab640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f209c15d340 0x7f209c06b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57768/0 (socket says 172.21.15.60:57768) 2023-11-18T20:42:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.833+0000 7f20a16aa640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f209c0ad640 0x7f209c160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56884/0 (socket says 172.21.15.60:56884) 2023-11-18T20:42:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.833+0000 7f20a1eab640 1 -- 172.21.15.60:0/2055557430 learned_addr learned my addr 172.21.15.60:0/2055557430 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.833+0000 7f20a26ac640 1 --1- 172.21.15.60:0/2055557430 >> v1:172.21.15.100:6789/0 conn(0x7f209c0aeb80 0x7f209c1615d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:35224/0 (socket says 172.21.15.60:35224) 2023-11-18T20:42:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.834+0000 7f20a1eab640 1 -- 172.21.15.60:0/2055557430 >> v1:172.21.15.100:6789/0 conn(0x7f209c0aeb80 legacy=0x7f209c1615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.834+0000 7f20a1eab640 1 -- 172.21.15.60:0/2055557430 >> v1:172.21.15.60:6789/0 conn(0x7f209c0ad640 legacy=0x7f209c160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:54.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.834+0000 7f20a1eab640 1 -- 172.21.15.60:0/2055557430 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c0addc0 con 0x7f209c15d340 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.834+0000 7f20a1eab640 1 --2- 172.21.15.60:0/2055557430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 0x7f209c06b4e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2094009870 tx=0x7f209402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed9cf51200e6c6c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.834+0000 7f20a0ea9640 1 -- 172.21.15.60:0/2055557430 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209402f850 con 0x7f209c15d340 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.835+0000 7f20a0ea9640 1 -- 172.21.15.60:0/2055557430 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209402f9b0 con 0x7f209c15d340 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.835+0000 7f20a0ea9640 1 -- 172.21.15.60:0/2055557430 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209402fd40 con 0x7f209c15d340 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.835+0000 7f20a410f640 1 -- 172.21.15.60:0/2055557430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 msgr2=0x7f209c06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.835+0000 7f20a410f640 1 --2- 172.21.15.60:0/2055557430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 0x7f209c06b4e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2094009870 tx=0x7f209402ba30 comp rx=0 tx=0).stop 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.835+0000 7f20a410f640 1 -- 172.21.15.60:0/2055557430 shutdown_connections 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.835+0000 7f20a410f640 1 --2- 172.21.15.60:0/2055557430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 0x7f209c06b4e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.835+0000 7f20a410f640 1 -- 172.21.15.60:0/2055557430 >> 172.21.15.60:0/2055557430 conn(0x7f209c070e70 msgr2=0x7f209c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.835+0000 7f20a410f640 1 -- 172.21.15.60:0/2055557430 shutdown_connections 2023-11-18T20:42:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.835+0000 7f20a410f640 1 -- 172.21.15.60:0/2055557430 wait complete. 2023-11-18T20:42:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.836+0000 7f20a410f640 1 Processor -- start 2023-11-18T20:42:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.836+0000 7f20a410f640 1 -- start start 2023-11-18T20:42:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a410f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f209c0ad640 0x7f209c1f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a410f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209c0aeb80 0x7f209c1f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a410f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 0x7f209c0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a410f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1f6e30 con 0x7f209c15d340 2023-11-18T20:42:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a410f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1f7250 con 0x7f209c0aeb80 2023-11-18T20:42:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a410f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1f2be0 con 0x7f209c0ad640 2023-11-18T20:42:54.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a26ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 0x7f209c0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a26ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 0x7f209c0ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57780/0 (socket says 172.21.15.60:57780) 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a26ac640 1 -- 172.21.15.60:0/4024139375 learned_addr learned my addr 172.21.15.60:0/4024139375 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a1eab640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f209c0ad640 0x7f209c1f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a16aa640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209c0aeb80 0x7f209c1f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a26ac640 1 -- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f209c0ad640 msgr2=0x7f209c1f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a26ac640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f209c0ad640 0x7f209c1f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a26ac640 1 -- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209c0aeb80 msgr2=0x7f209c1f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a26ac640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209c0aeb80 0x7f209c1f26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a26ac640 1 -- 172.21.15.60:0/4024139375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940094d0 con 0x7f209c15d340 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a16aa640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209c0aeb80 0x7f209c1f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.837+0000 7f20a1eab640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f209c0ad640 0x7f209c1f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.838+0000 7f20a26ac640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 0x7f209c0ced80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f208c00c7b0 tx=0x7f208c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.838+0000 7f209a7fc640 1 -- 172.21.15.60:0/4024139375 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c00a640 con 0x7f209c15d340 2023-11-18T20:42:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.838+0000 7f209a7fc640 1 -- 172.21.15.60:0/4024139375 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c00a7a0 con 0x7f209c15d340 2023-11-18T20:42:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.838+0000 7f20a410f640 1 -- 172.21.15.60:0/4024139375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c1f6490 con 0x7f209c15d340 2023-11-18T20:42:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.838+0000 7f209a7fc640 1 -- 172.21.15.60:0/4024139375 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c00ab10 con 0x7f209c15d340 2023-11-18T20:42:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.838+0000 7f20a410f640 1 -- 172.21.15.60:0/4024139375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c1f6a50 con 0x7f209c15d340 2023-11-18T20:42:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.839+0000 7f20a410f640 1 -- 172.21.15.60:0/4024139375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209c1f6f70 con 0x7f209c15d340 2023-11-18T20:42:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.839+0000 7f209a7fc640 1 -- 172.21.15.60:0/4024139375 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f208c00ac70 con 0x7f209c15d340 2023-11-18T20:42:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.840+0000 7f209a7fc640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208806ae40 0x7f208806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.840+0000 7f209a7fc640 1 -- 172.21.15.60:0/4024139375 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f208c088c60 con 0x7f209c15d340 2023-11-18T20:42:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.840+0000 7f20a1eab640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208806ae40 0x7f208806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:54.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.841+0000 7f20a1eab640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208806ae40 0x7f208806d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2094036000 tx=0x7f2094008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:54.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:54.844+0000 7f209a7fc640 1 -- 172.21.15.60:0/4024139375 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f209c1f6f70 con 0x7f209c15d340 2023-11-18T20:42:55.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.055+0000 7f20a410f640 1 -- 172.21.15.60:0/4024139375 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f209c1f6f70 con 0x7f209c15d340 2023-11-18T20:42:55.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.056+0000 7f209a7fc640 1 -- 172.21.15.60:0/4024139375 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f209c1f6f70 con 0x7f209c15d340 2023-11-18T20:42:55.057 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-18T20:42:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.057+0000 7f20a410f640 1 -- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208806ae40 msgr2=0x7f208806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.057+0000 7f20a410f640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208806ae40 0x7f208806d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2094036000 tx=0x7f2094008030 comp rx=0 tx=0).stop 2023-11-18T20:42:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.057+0000 7f20a410f640 1 -- 172.21.15.60:0/4024139375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 msgr2=0x7f209c0ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.057+0000 7f20a410f640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 0x7f209c0ced80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f208c00c7b0 tx=0x7f208c00cc70 comp rx=0 tx=0).stop 2023-11-18T20:42:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.057+0000 7f20a410f640 1 -- 172.21.15.60:0/4024139375 shutdown_connections 2023-11-18T20:42:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.057+0000 7f20a410f640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208806ae40 0x7f208806d300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.057+0000 7f20a410f640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f209c15d340 0x7f209c0ced80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:55.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.057+0000 7f20a410f640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f209c0aeb80 0x7f209c1f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:55.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.057+0000 7f20a410f640 1 --2- 172.21.15.60:0/4024139375 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f209c0ad640 0x7f209c1f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:55.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.057+0000 7f20a410f640 1 -- 172.21.15.60:0/4024139375 >> 172.21.15.60:0/4024139375 conn(0x7f209c070e70 msgr2=0x7f209c0aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:55.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.058+0000 7f20a410f640 1 -- 172.21.15.60:0/4024139375 shutdown_connections 2023-11-18T20:42:55.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:55.058+0000 7f20a410f640 1 -- 172.21.15.60:0/4024139375 wait complete. 2023-11-18T20:42:55.069 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-18T20:42:55.070 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-18T20:42:55.070 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2023-11-18T20:42:55.070 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-18T20:42:55.070 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-18T20:42:55.070 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-18T20:42:55.070 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-18T20:42:55.070 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-18T20:42:55.070 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-18T20:42:55.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:55.071 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2023-11-18T20:42:55.089 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:42:55.089 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:42:55.089 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:55.090 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:55.091 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-18T20:42:55.091 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:55.091 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-18T20:42:55.091 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42684sec preferred_lft 42684sec 2023-11-18T20:42:55.091 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2023-11-18T20:42:55.091 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:55.091 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-18T20:42:55.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:55.091 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2023-11-18T20:42:55.091 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2023-11-18T20:42:55.092 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2023-11-18T20:42:55.092 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2023-11-18T20:42:55.092 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2023-11-18T20:42:55.092 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-18T20:42:55.092 DEBUG:teuthology.orchestra.run.smithi100:> ') 2023-11-18T20:42:55.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:55.256 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-18T20:42:55.284 INFO:teuthology.orchestra.run.smithi100.stdout:1 2023-11-18T20:42:55.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:55.285 DEBUG:teuthology.orchestra.run.smithi100:> route 2023-11-18T20:42:55.372 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2023-11-18T20:42:55.372 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-18T20:42:55.372 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-18T20:42:55.373 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-18T20:42:55.373 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-18T20:42:55.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:55.373 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2023-11-18T20:42:55.373 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2023-11-18T20:42:55.373 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-18T20:42:55.374 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-18T20:42:55.374 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-18T20:42:55.374 DEBUG:teuthology.orchestra.run.smithi100:> ') 2023-11-18T20:42:55.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:55.586 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2023-11-18T20:42:55.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:55.601 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2023-11-18T20:42:55.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:55.656 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2023-11-18T20:42:55.656 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2023-11-18T20:42:55.656 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:42:55.656 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-18T20:42:55.656 DEBUG:teuthology.orchestra.run.smithi100:> ') 2023-11-18T20:42:55.738 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-18T20:42:55.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:55.739 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2023-11-18T20:42:55.739 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2023-11-18T20:42:55.739 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-18T20:42:55.739 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-18T20:42:55.739 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-18T20:42:55.739 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-18T20:42:55.739 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-18T20:42:55.739 DEBUG:teuthology.orchestra.run.smithi100:> ') 2023-11-18T20:42:55.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:55.937 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2023-11-18T20:42:55.937 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2023-11-18T20:42:55.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2023-11-18T20:42:55.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2023-11-18T20:42:55.938 DEBUG:teuthology.orchestra.run.smithi100:> ') 2023-11-18T20:42:55.978 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-18T20:42:55.978 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:42:55.979 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-18T20:42:56.033 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-18T20:42:56.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:42:56.034 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-18T20:42:56.088 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2023-11-18T20:42:56.157 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:42:56.212 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2023-11-18T20:42:56.213 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:42:56.214 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:42:56.215 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2023-11-18T20:42:56.215 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2023-11-18T20:42:56.215 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2023-11-18T20:42:56.215 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2023-11-18T20:42:56.215 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2023-11-18T20:42:56.215 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2023-11-18T20:42:56.215 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2023-11-18T20:42:56.215 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:42:56.215 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:42:56.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:56.216 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2023-11-18T20:42:56.267 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-18T20:42:56.267 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-18T20:42:56.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2023-11-18T20:42:56.335 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:42:56.391 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2023-11-18T20:42:56.391 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2023-11-18T20:42:56.391 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2023-11-18T20:42:56.391 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2023-11-18T20:42:56.391 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2023-11-18T20:42:56.391 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2023-11-18T20:42:56.391 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2023-11-18T20:42:56.391 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2023-11-18T20:42:56.391 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:42:56.392 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2023-11-18T20:42:56.393 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2023-11-18T20:42:56.393 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2023-11-18T20:42:56.393 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2023-11-18T20:42:56.393 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2023-11-18T20:42:56.393 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2023-11-18T20:42:56.393 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2023-11-18T20:42:56.393 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:42:56.393 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:42:56.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:56.393 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2023-11-18T20:42:56.404 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi100.stderr:2023-11-18T20:42:56.403+0000 7fcf91435480 -1 init, newargv = 0x55acbe410220 newargc=15 2023-11-18T20:42:56.404 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi100.stderr:ceph-fuse[93815]: starting ceph client 2023-11-18T20:42:56.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi100.stderr:ceph-fuse[93815]: starting fuse 2023-11-18T20:42:56.418 INFO:teuthology.orchestra.run.smithi100.stdout:41 2023-11-18T20:42:56.419 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-18T20:42:56.419 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2023-11-18T20:42:56.419 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2023-11-18T20:42:56.419 DEBUG:teuthology.orchestra.run.smithi100:> import re 2023-11-18T20:42:56.419 DEBUG:teuthology.orchestra.run.smithi100:> import os 2023-11-18T20:42:56.419 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2023-11-18T20:42:56.419 DEBUG:teuthology.orchestra.run.smithi100:> 2023-11-18T20:42:56.419 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2023-11-18T20:42:56.419 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-18T20:42:56.419 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2023-11-18T20:42:56.419 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2023-11-18T20:42:56.420 DEBUG:teuthology.orchestra.run.smithi100:> return f 2023-11-18T20:42:56.421 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-18T20:42:56.421 DEBUG:teuthology.orchestra.run.smithi100:> 2023-11-18T20:42:56.421 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.1")) 2023-11-18T20:42:56.421 DEBUG:teuthology.orchestra.run.smithi100:> ' 2023-11-18T20:42:56.538 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.1.93815.asok 2023-11-18T20:42:56.541 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:42:56.549 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.93815.asok 2023-11-18T20:42:56.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:56.549 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.93815.asok status 2023-11-18T20:42:56.685 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2023-11-18T20:42:56.685 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2023-11-18T20:42:56.685 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe", 2023-11-18T20:42:56.685 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 18.0.0-7418-gb0f45a51 (b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe) reef (dev)", 2023-11-18T20:42:56.685 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "1", 2023-11-18T20:42:56.685 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2023-11-18T20:42:56.685 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "93815", 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4657, 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4657 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2023-11-18T20:42:56.686 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2023-11-18T20:42:56.687 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2023-11-18T20:42:56.687 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:42:56.687 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 274274222 2023-11-18T20:42:56.687 INFO:teuthology.orchestra.run.smithi100.stdout: } 2023-11-18T20:42:56.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2023-11-18T20:42:56.687 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2023-11-18T20:42:56.687 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2023-11-18T20:42:56.687 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:42:56.687 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 274274222 2023-11-18T20:42:56.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2023-11-18T20:42:56.688 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4657 192.168.0.1:0/274274222", 2023-11-18T20:42:56.688 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.1:0/274274222", 2023-11-18T20:42:56.688 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2023-11-18T20:42:56.688 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 6, 2023-11-18T20:42:56.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 21, 2023-11-18T20:42:56.688 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2023-11-18T20:42:56.688 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2023-11-18T20:42:56.688 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2023-11-18T20:42:56.688 INFO:teuthology.orchestra.run.smithi100.stdout:} 2023-11-18T20:42:56.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-18T20:42:56.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.811+0000 7f245941f640 1 Processor -- start 2023-11-18T20:42:56.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.811+0000 7f245941f640 1 -- start start 2023-11-18T20:42:56.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.812+0000 7f245941f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f24541409d0 0x7f24541618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:56.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.812+0000 7f245941f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24541406f0 con 0x7f245406abe0 2023-11-18T20:42:56.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.812+0000 7f245941f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2454143960 con 0x7f2454140310 2023-11-18T20:42:56.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.812+0000 7f245941f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2454161e20 con 0x7f24541409d0 2023-11-18T20:42:56.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.812+0000 7f24527fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f245406abe0 0x7f245406efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56900/0 (socket says 172.21.15.60:56900) 2023-11-18T20:42:56.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.812+0000 7f24527fc640 1 -- 172.21.15.60:0/357455963 learned_addr learned my addr 172.21.15.60:0/357455963 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:56.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.812+0000 7f24537fe640 1 --2- 172.21.15.60:0/357455963 >> v2:172.21.15.100:3300/0 conn(0x7f24541409d0 0x7f24541618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:56.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.812+0000 7f2451ffb640 1 -- 172.21.15.60:0/357455963 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800916341 0 0) 0x7f24541406f0 con 0x7f245406abe0 2023-11-18T20:42:56.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.812+0000 7f2451ffb640 1 -- 172.21.15.60:0/357455963 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2440003520 con 0x7f245406abe0 2023-11-18T20:42:56.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.813+0000 7f2451ffb640 1 -- 172.21.15.60:0/357455963 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352588937 0 0) 0x7f2454143960 con 0x7f2454140310 2023-11-18T20:42:56.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.813+0000 7f2451ffb640 1 -- 172.21.15.60:0/357455963 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24541406f0 con 0x7f2454140310 2023-11-18T20:42:56.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.813+0000 7f24537fe640 1 -- 172.21.15.60:0/357455963 >> v1:172.21.15.100:6789/0 conn(0x7f2454140310 legacy=0x7f245406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:56.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.813+0000 7f24537fe640 1 -- 172.21.15.60:0/357455963 >> v1:172.21.15.60:6789/0 conn(0x7f245406abe0 legacy=0x7f245406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:56.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.813+0000 7f24537fe640 1 -- 172.21.15.60:0/357455963 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2454142de0 con 0x7f24541409d0 2023-11-18T20:42:56.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.813+0000 7f24537fe640 1 --2- 172.21.15.60:0/357455963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24541409d0 0x7f24541618e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f244c0097a0 tx=0x7f244c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=425cd098ad96727 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:56.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.814+0000 7f2451ffb640 1 -- 172.21.15.60:0/357455963 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c031830 con 0x7f24541409d0 2023-11-18T20:42:56.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.814+0000 7f2451ffb640 1 -- 172.21.15.60:0/357455963 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c031990 con 0x7f24541409d0 2023-11-18T20:42:56.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.814+0000 7f2451ffb640 1 -- 172.21.15.60:0/357455963 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c031d20 con 0x7f24541409d0 2023-11-18T20:42:56.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.814+0000 7f245941f640 1 -- 172.21.15.60:0/357455963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24541409d0 msgr2=0x7f24541618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:56.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.814+0000 7f245941f640 1 --2- 172.21.15.60:0/357455963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24541409d0 0x7f24541618e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f244c0097a0 tx=0x7f244c02b8d0 comp rx=0 tx=0).stop 2023-11-18T20:42:56.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.814+0000 7f245941f640 1 -- 172.21.15.60:0/357455963 shutdown_connections 2023-11-18T20:42:56.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.814+0000 7f245941f640 1 --2- 172.21.15.60:0/357455963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24541409d0 0x7f24541618e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:56.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.814+0000 7f245941f640 1 -- 172.21.15.60:0/357455963 >> 172.21.15.60:0/357455963 conn(0x7f24540b1e50 msgr2=0x7f24540b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:56.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.814+0000 7f245941f640 1 -- 172.21.15.60:0/357455963 shutdown_connections 2023-11-18T20:42:56.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.814+0000 7f245941f640 1 -- 172.21.15.60:0/357455963 wait complete. 2023-11-18T20:42:56.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.816+0000 7f245941f640 1 Processor -- start 2023-11-18T20:42:56.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.816+0000 7f245941f640 1 -- start start 2023-11-18T20:42:56.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.816+0000 7f245941f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f245406abe0 0x7f2454146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:56.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.816+0000 7f245941f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2454140310 0x7f24540b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:56.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.816+0000 7f245941f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24541409d0 0x7f24540b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:56.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.816+0000 7f245941f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24540b5060 con 0x7f24541409d0 2023-11-18T20:42:56.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.816+0000 7f245941f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24540b3a80 con 0x7f245406abe0 2023-11-18T20:42:56.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.816+0000 7f245941f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24540b3d60 con 0x7f2454140310 2023-11-18T20:42:56.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24541409d0 0x7f24540b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:56.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24541409d0 0x7f24540b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57790/0 (socket says 172.21.15.60:57790) 2023-11-18T20:42:56.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24537fe640 1 -- 172.21.15.60:0/1153408431 learned_addr learned my addr 172.21.15.60:0/1153408431 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f2452ffd640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f245406abe0 0x7f2454146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24527fc640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2454140310 0x7f24540b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24537fe640 1 -- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2454140310 msgr2=0x7f24540b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24537fe640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2454140310 0x7f24540b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24537fe640 1 -- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f245406abe0 msgr2=0x7f2454146780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24537fe640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f245406abe0 0x7f2454146780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24537fe640 1 -- 172.21.15.60:0/1153408431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c009400 con 0x7f24541409d0 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24527fc640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2454140310 0x7f24540b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f2452ffd640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f245406abe0 0x7f2454146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f24537fe640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24541409d0 0x7f24540b4b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f244c0097a0 tx=0x7f244c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f244b7fe640 1 -- 172.21.15.60:0/1153408431 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c03a630 con 0x7f24541409d0 2023-11-18T20:42:56.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f244b7fe640 1 -- 172.21.15.60:0/1153408431 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c03a790 con 0x7f24541409d0 2023-11-18T20:42:56.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f245941f640 1 -- 172.21.15.60:0/1153408431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24540b4040 con 0x7f24541409d0 2023-11-18T20:42:56.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f244b7fe640 1 -- 172.21.15.60:0/1153408431 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c03ab40 con 0x7f24541409d0 2023-11-18T20:42:56.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.817+0000 7f245941f640 1 -- 172.21.15.60:0/1153408431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24540b4600 con 0x7f24541409d0 2023-11-18T20:42:56.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.818+0000 7f245941f640 1 -- 172.21.15.60:0/1153408431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2454161030 con 0x7f24541409d0 2023-11-18T20:42:56.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.819+0000 7f244b7fe640 1 -- 172.21.15.60:0/1153408431 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f244c03aca0 con 0x7f24541409d0 2023-11-18T20:42:56.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.819+0000 7f244b7fe640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f243406ae40 0x7f243406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:56.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.819+0000 7f244b7fe640 1 -- 172.21.15.60:0/1153408431 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f244c0aaf50 con 0x7f24541409d0 2023-11-18T20:42:56.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.819+0000 7f2452ffd640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f243406ae40 0x7f243406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:56.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.820+0000 7f2452ffd640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f243406ae40 0x7f243406d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2444007fd0 tx=0x7f2444009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:56.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:56.823+0000 7f244b7fe640 1 -- 172.21.15.60:0/1153408431 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f244c075f80 con 0x7f24541409d0 2023-11-18T20:42:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.038+0000 7f245941f640 1 -- 172.21.15.60:0/1153408431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2454069400 con 0x7f24541409d0 2023-11-18T20:42:57.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.038+0000 7f244b7fe640 1 -- 172.21.15.60:0/1153408431 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f244c048070 con 0x7f24541409d0 2023-11-18T20:42:57.040 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-18T20:42:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.040+0000 7f245941f640 1 -- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f243406ae40 msgr2=0x7f243406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.040+0000 7f245941f640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f243406ae40 0x7f243406d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2444007fd0 tx=0x7f2444009040 comp rx=0 tx=0).stop 2023-11-18T20:42:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.040+0000 7f245941f640 1 -- 172.21.15.60:0/1153408431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24541409d0 msgr2=0x7f24540b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.040+0000 7f245941f640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24541409d0 0x7f24540b4b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f244c0097a0 tx=0x7f244c0046c0 comp rx=0 tx=0).stop 2023-11-18T20:42:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.040+0000 7f245941f640 1 -- 172.21.15.60:0/1153408431 shutdown_connections 2023-11-18T20:42:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.040+0000 7f245941f640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f243406ae40 0x7f243406d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.040+0000 7f245941f640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24541409d0 0x7f24540b4b20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:57.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.040+0000 7f245941f640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2454140310 0x7f24540b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.040+0000 7f245941f640 1 --2- 172.21.15.60:0/1153408431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f245406abe0 0x7f2454146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.040+0000 7f245941f640 1 -- 172.21.15.60:0/1153408431 >> 172.21.15.60:0/1153408431 conn(0x7f24540b1e50 msgr2=0x7f245406f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.041+0000 7f245941f640 1 -- 172.21.15.60:0/1153408431 shutdown_connections 2023-11-18T20:42:57.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:57.041+0000 7f245941f640 1 -- 172.21.15.60:0/1153408431 wait complete. 2023-11-18T20:42:57.052 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-18T20:42:57.052 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-18T20:42:57.052 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2023-11-18T20:42:57.052 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-18T20:42:57.052 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-18T20:42:57.053 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-18T20:42:57.053 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-18T20:42:57.053 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-18T20:42:57.053 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-18T20:42:57.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.053 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2023-11-18T20:42:57.068 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:42:57.068 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:42:57.069 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42682sec preferred_lft 42682sec 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether a2:ec:2f:1d:b2:3b brd ff:ff:ff:ff:ff:ff 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::a0e7:81ff:fe1d:9141/64 scope link 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.070 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:42:57.071 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether a2:ec:2f:1d:b2:3b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:42:57.071 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::a0ec:2fff:fe1d:b23b/64 scope link tentative 2023-11-18T20:42:57.071 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.071 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-18T20:42:57.134 INFO:teuthology.orchestra.run.smithi100.stdout:1 2023-11-18T20:42:57.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.135 DEBUG:teuthology.orchestra.run.smithi100:> route 2023-11-18T20:42:57.192 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2023-11-18T20:42:57.192 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-18T20:42:57.192 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-18T20:42:57.192 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-18T20:42:57.192 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-18T20:42:57.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.193 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2023-11-18T20:42:57.193 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2023-11-18T20:42:57.193 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-18T20:42:57.193 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-18T20:42:57.193 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-18T20:42:57.193 DEBUG:teuthology.orchestra.run.smithi100:> ') 2023-11-18T20:42:57.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.287 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2023-11-18T20:42:57.301 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-18T20:42:57.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.302 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2023-11-18T20:42:57.356 INFO:teuthology.orchestra.run.smithi100.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-18T20:42:57.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.356 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2023-11-18T20:42:57.357 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2023-11-18T20:42:57.357 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-18T20:42:57.357 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-18T20:42:57.357 DEBUG:teuthology.orchestra.run.smithi100:> ') 2023-11-18T20:42:57.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.439 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-18T20:42:57.505 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:42:57.505 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:42:57.505 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:42:57.505 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.506 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2023-11-18T20:42:57.506 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.506 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:42:57.506 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether a2:8e:86:1d:cb:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-18T20:42:57.506 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-18T20:42:57.506 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.506 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::a08e:86ff:fe1d:cb21/64 scope link tentative 2023-11-18T20:42:57.506 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.508 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-18T20:42:57.575 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:42:57.575 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:42:57.575 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:42:57.575 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.575 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2023-11-18T20:42:57.575 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.575 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:42:57.576 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether a2:8e:86:1d:cb:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-18T20:42:57.576 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-18T20:42:57.576 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.576 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::a08e:86ff:fe1d:cb21/64 scope link tentative 2023-11-18T20:42:57.576 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:42:57.577 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-18T20:42:57.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.577 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2023-11-18T20:42:57.577 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2023-11-18T20:42:57.577 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-18T20:42:57.577 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-18T20:42:57.578 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-18T20:42:57.578 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-18T20:42:57.578 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-18T20:42:57.578 DEBUG:teuthology.orchestra.run.smithi100:> ') 2023-11-18T20:42:57.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.702 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2023-11-18T20:42:57.702 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2023-11-18T20:42:57.702 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.1 up 2023-11-18T20:42:57.702 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.1 master ceph-brx 2023-11-18T20:42:57.702 DEBUG:teuthology.orchestra.run.smithi100:> ') 2023-11-18T20:42:57.744 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-18T20:42:57.744 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:42:57.744 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-18T20:42:57.803 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-18T20:42:57.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:42:57.803 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-18T20:42:57.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2023-11-18T20:42:57.925 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:42:57.980 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2023-11-18T20:42:57.980 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2023-11-18T20:42:57.980 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2023-11-18T20:42:57.980 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2023-11-18T20:42:57.981 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2023-11-18T20:42:57.982 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2023-11-18T20:42:57.983 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2023-11-18T20:42:57.983 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2023-11-18T20:42:57.983 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2023-11-18T20:42:57.983 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2023-11-18T20:42:57.983 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:42:57.983 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:42:57.983 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-18T20:42:57.983 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-18T20:42:57.983 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-18T20:42:57.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:57.984 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2023-11-18T20:42:58.035 INFO:teuthology.orchestra.run.smithi100.stdout:41 2023-11-18T20:42:58.035 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-11-18T20:42:58.035 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-18T20:42:58.077 DEBUG:teuthology.orchestra.run.smithi100:> sudo modprobe fuse 2023-11-18T20:42:58.103 DEBUG:teuthology.orchestra.run.smithi100:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:42:58.159 INFO:teuthology.orchestra.run.smithi100.stdout:/proc 2023-11-18T20:42:58.159 INFO:teuthology.orchestra.run.smithi100.stdout:/sys 2023-11-18T20:42:58.159 INFO:teuthology.orchestra.run.smithi100.stdout:/dev 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/security 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/shm 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/pts 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/run 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/cgroup 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/pstore 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/bpf 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/ 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/selinux 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/hugepages 2023-11-18T20:42:58.160 INFO:teuthology.orchestra.run.smithi100.stdout:/dev/mqueue 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/debug 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/tracing 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/fs/fuse/connections 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/sys/kernel/config 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/run/user/1000 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-4 2023-11-18T20:42:58.161 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-5 2023-11-18T20:42:58.162 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-6 2023-11-18T20:42:58.162 INFO:teuthology.orchestra.run.smithi100.stdout:/var/lib/ceph/osd/ceph-7 2023-11-18T20:42:58.162 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns 2023-11-18T20:42:58.162 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:42:58.162 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:42:58.162 INFO:teuthology.orchestra.run.smithi100.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-18T20:42:58.162 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-18T20:42:58.162 INFO:teuthology.orchestra.run.smithi100.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-18T20:42:58.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:58.162 DEBUG:teuthology.orchestra.run.smithi100:> ls /sys/fs/fuse/connections 2023-11-18T20:42:58.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi100.stderr:2023-11-18T20:42:58.162+0000 7fafb1e41480 -1 init, newargv = 0x5572946918d0 newargc=15 2023-11-18T20:42:58.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi100.stderr:ceph-fuse[94259]: starting ceph client 2023-11-18T20:42:58.169 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi100.stderr:ceph-fuse[94259]: starting fuse 2023-11-18T20:42:58.177 INFO:teuthology.orchestra.run.smithi100.stdout:41 2023-11-18T20:42:58.177 INFO:teuthology.orchestra.run.smithi100.stdout:44 2023-11-18T20:42:58.178 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-11-18T20:42:58.178 DEBUG:teuthology.orchestra.run.smithi100:> sudo stdin-killer -- python3 -c ' 2023-11-18T20:42:58.178 DEBUG:teuthology.orchestra.run.smithi100:> import glob 2023-11-18T20:42:58.178 DEBUG:teuthology.orchestra.run.smithi100:> import re 2023-11-18T20:42:58.178 DEBUG:teuthology.orchestra.run.smithi100:> import os 2023-11-18T20:42:58.178 DEBUG:teuthology.orchestra.run.smithi100:> import subprocess 2023-11-18T20:42:58.178 DEBUG:teuthology.orchestra.run.smithi100:> 2023-11-18T20:42:58.178 DEBUG:teuthology.orchestra.run.smithi100:> def _find_admin_socket(client_name): 2023-11-18T20:42:58.178 DEBUG:teuthology.orchestra.run.smithi100:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-18T20:42:58.178 DEBUG:teuthology.orchestra.run.smithi100:> files = glob.glob(asok_path) 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> # Given a non-glob path, it better be there 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> if "*" not in asok_path: 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> assert(len(files) == 1) 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> return files[0] 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> for f in files: 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> contents = proc_f.read() 2023-11-18T20:42:58.179 DEBUG:teuthology.orchestra.run.smithi100:> if mountpoint in contents: 2023-11-18T20:42:58.180 DEBUG:teuthology.orchestra.run.smithi100:> return f 2023-11-18T20:42:58.180 DEBUG:teuthology.orchestra.run.smithi100:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-18T20:42:58.180 DEBUG:teuthology.orchestra.run.smithi100:> 2023-11-18T20:42:58.180 DEBUG:teuthology.orchestra.run.smithi100:> print(_find_admin_socket("client.3")) 2023-11-18T20:42:58.180 DEBUG:teuthology.orchestra.run.smithi100:> ' 2023-11-18T20:42:58.298 INFO:teuthology.orchestra.run.smithi100.stdout:/var/run/ceph/ceph-client.3.94259.asok 2023-11-18T20:42:58.301 INFO:teuthology.orchestra.run.smithi100.stderr:2023-11-18T20:42:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:42:58.309 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.94259.asok 2023-11-18T20:42:58.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:58.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.94259.asok status 2023-11-18T20:42:58.445 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2023-11-18T20:42:58.445 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": { 2023-11-18T20:42:58.445 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_sha1": "b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe", 2023-11-18T20:42:58.445 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 18.0.0-7418-gb0f45a51 (b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe) reef (dev)", 2023-11-18T20:42:58.445 INFO:teuthology.orchestra.run.smithi100.stdout: "entity_id": "3", 2023-11-18T20:42:58.445 INFO:teuthology.orchestra.run.smithi100.stdout: "hostname": "smithi100", 2023-11-18T20:42:58.445 INFO:teuthology.orchestra.run.smithi100.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-18T20:42:58.445 INFO:teuthology.orchestra.run.smithi100.stdout: "pid": "94259", 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: "root": "/" 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_count": 0, 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: "dentry_pinned_count": 0, 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4511, 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: "inst": { 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: "name": { 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "client", 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: "num": 4511 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2023-11-18T20:42:58.446 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.2:0", 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1823468247 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: } 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": { 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "192.168.0.2:0", 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1823468247 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: "inst_str": "client.4511 192.168.0.2:0/1823468247", 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: "addr_str": "192.168.0.2:0/1823468247", 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: "inode_count": 1, 2023-11-18T20:42:58.447 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_epoch": 6, 2023-11-18T20:42:58.448 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch": 21, 2023-11-18T20:42:58.448 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_epoch_barrier": 0, 2023-11-18T20:42:58.448 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklisted": false, 2023-11-18T20:42:58.448 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs" 2023-11-18T20:42:58.448 INFO:teuthology.orchestra.run.smithi100.stdout:} 2023-11-18T20:42:58.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:58.454 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-11-18T20:42:58.454 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-18T20:42:58.473 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-11-18T20:42:58.473 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-18T20:42:58.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:58.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:42:58.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:58.553 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-11-18T20:42:58.553 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-18T20:42:58.610 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-11-18T20:42:58.610 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-18T20:42:58.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:58.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-18T20:42:58.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:58.687 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2023-11-18T20:42:58.687 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-18T20:42:58.703 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2023-11-18T20:42:58.703 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-18T20:42:58.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:58.704 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-18T20:42:58.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:58.772 DEBUG:teuthology.orchestra.run.smithi100:> stat --file-system '--printf=%T 2023-11-18T20:42:58.772 DEBUG:teuthology.orchestra.run.smithi100:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-18T20:42:58.827 INFO:teuthology.orchestra.run.smithi100.stdout:fuseblk 2023-11-18T20:42:58.827 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-18T20:42:58.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:42:58.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-18T20:42:58.895 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-18T20:42:58.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:42:59.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.032+0000 7fa3bd251640 1 Processor -- start 2023-11-18T20:42:59.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.032+0000 7fa3bd251640 1 -- start start 2023-11-18T20:42:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.033+0000 7fa3bd251640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3b80d3900 0x7fa3b80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.033+0000 7fa3bd251640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b80b2b10 con 0x7fa3b815d330 2023-11-18T20:42:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.033+0000 7fa3bd251640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b80d1590 con 0x7fa3b80d2910 2023-11-18T20:42:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.033+0000 7fa3bd251640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b80d1870 con 0x7fa3b80d3900 2023-11-18T20:42:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.033+0000 7fa3b759e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3b80d3900 0x7fa3b80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.033+0000 7fa3b759e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3b80d3900 0x7fa3b80b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57806/0 (socket says 172.21.15.60:57806) 2023-11-18T20:42:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.033+0000 7fa3b6d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa3b815d330 0x7fa3b80b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56904/0 (socket says 172.21.15.60:56904) 2023-11-18T20:42:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.033+0000 7fa3b759e640 1 -- 172.21.15.60:0/1118038818 learned_addr learned my addr 172.21.15.60:0/1118038818 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.034+0000 7fa3b759e640 1 -- 172.21.15.60:0/1118038818 >> v1:172.21.15.100:6789/0 conn(0x7fa3b80d2910 legacy=0x7fa3b80d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.034+0000 7fa3b759e640 1 -- 172.21.15.60:0/1118038818 >> v1:172.21.15.60:6789/0 conn(0x7fa3b815d330 legacy=0x7fa3b80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.034+0000 7fa3b5d9b640 1 -- 172.21.15.60:0/1118038818 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070562857 0 0) 0x7fa3b80d1590 con 0x7fa3b80d2910 2023-11-18T20:42:59.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.034+0000 7fa3b759e640 1 -- 172.21.15.60:0/1118038818 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80d1b50 con 0x7fa3b80d3900 2023-11-18T20:42:59.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.034+0000 7fa3b759e640 1 --2- 172.21.15.60:0/1118038818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d3900 0x7fa3b80b6390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009e50 tx=0x7fa3b002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117648fc938aa20d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.035+0000 7fa3b5d9b640 1 -- 172.21.15.60:0/1118038818 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b002c8d0 con 0x7fa3b80d3900 2023-11-18T20:42:59.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.035+0000 7fa3b5d9b640 1 -- 172.21.15.60:0/1118038818 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b002ca30 con 0x7fa3b80d3900 2023-11-18T20:42:59.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.035+0000 7fa3b5d9b640 1 -- 172.21.15.60:0/1118038818 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b002cdc0 con 0x7fa3b80d3900 2023-11-18T20:42:59.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.035+0000 7fa3bd251640 1 -- 172.21.15.60:0/1118038818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d3900 msgr2=0x7fa3b80b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.035+0000 7fa3bd251640 1 --2- 172.21.15.60:0/1118038818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d3900 0x7fa3b80b6390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009e50 tx=0x7fa3b002bdb0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.035+0000 7fa3bd251640 1 -- 172.21.15.60:0/1118038818 shutdown_connections 2023-11-18T20:42:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.035+0000 7fa3bd251640 1 --2- 172.21.15.60:0/1118038818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d3900 0x7fa3b80b6390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.035+0000 7fa3bd251640 1 -- 172.21.15.60:0/1118038818 >> 172.21.15.60:0/1118038818 conn(0x7fa3b8072020 msgr2=0x7fa3b8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.035+0000 7fa3bd251640 1 -- 172.21.15.60:0/1118038818 shutdown_connections 2023-11-18T20:42:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.035+0000 7fa3bd251640 1 -- 172.21.15.60:0/1118038818 wait complete. 2023-11-18T20:42:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.037+0000 7fa3bd251640 1 Processor -- start 2023-11-18T20:42:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.037+0000 7fa3bd251640 1 -- start start 2023-11-18T20:42:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3bd251640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d2910 0x7fa3b815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3bd251640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3b80d3900 0x7fa3b81544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3bd251640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3b815d330 0x7fa3b80b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3bd251640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80cc5a0 con 0x7fa3b80d2910 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3bd251640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80cb990 con 0x7fa3b80d3900 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3bd251640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80cbc70 con 0x7fa3b815d330 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d2910 0x7fa3b815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d2910 0x7fa3b815e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57818/0 (socket says 172.21.15.60:57818) 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b6d9d640 1 -- 172.21.15.60:0/4273031981 learned_addr learned my addr 172.21.15.60:0/4273031981 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b659c640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3b80d3900 0x7fa3b81544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b759e640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3b815d330 0x7fa3b80b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b6d9d640 1 -- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3b815d330 msgr2=0x7fa3b80b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b6d9d640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3b815d330 0x7fa3b80b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b6d9d640 1 -- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3b80d3900 msgr2=0x7fa3b81544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b6d9d640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3b80d3900 0x7fa3b81544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b6d9d640 1 -- 172.21.15.60:0/4273031981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b0009b30 con 0x7fa3b80d2910 2023-11-18T20:42:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.038+0000 7fa3b6d9d640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d2910 0x7fa3b815e810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00c780 tx=0x7fa3ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.039+0000 7fa39f7fe640 1 -- 172.21.15.60:0/4273031981 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0072d0 con 0x7fa3b80d2910 2023-11-18T20:42:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.039+0000 7fa39f7fe640 1 -- 172.21.15.60:0/4273031981 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac007430 con 0x7fa3b80d2910 2023-11-18T20:42:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.039+0000 7fa3bd251640 1 -- 172.21.15.60:0/4273031981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b80cbf50 con 0x7fa3b80d2910 2023-11-18T20:42:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.039+0000 7fa39f7fe640 1 -- 172.21.15.60:0/4273031981 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0047e0 con 0x7fa3b80d2910 2023-11-18T20:42:59.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.039+0000 7fa3bd251640 1 -- 172.21.15.60:0/4273031981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b8154a20 con 0x7fa3b80d2910 2023-11-18T20:42:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.040+0000 7fa3bd251640 1 -- 172.21.15.60:0/4273031981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b8069400 con 0x7fa3b80d2910 2023-11-18T20:42:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.040+0000 7fa39f7fe640 1 -- 172.21.15.60:0/4273031981 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3ac004940 con 0x7fa3b80d2910 2023-11-18T20:42:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.041+0000 7fa39f7fe640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa3a406ae40 0x7fa3a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.041+0000 7fa39f7fe640 1 -- 172.21.15.60:0/4273031981 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3ac088ed0 con 0x7fa3b80d2910 2023-11-18T20:42:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.041+0000 7fa3b659c640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa3a406ae40 0x7fa3a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.042+0000 7fa3b659c640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa3a406ae40 0x7fa3a406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80c9770 tx=0x7fa3a8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.045+0000 7fa39f7fe640 1 -- 172.21.15.60:0/4273031981 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3ac054110 con 0x7fa3b80d2910 2023-11-18T20:42:59.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.230+0000 7fa3bd251640 1 -- 172.21.15.60:0/4273031981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3b80b6350 con 0x7fa3b80d2910 2023-11-18T20:42:59.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.232+0000 7fa39f7fe640 1 -- 172.21.15.60:0/4273031981 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fa3ac018090 con 0x7fa3b80d2910 2023-11-18T20:42:59.233 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:42:59.233 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:42:59.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 -- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa3a406ae40 msgr2=0x7fa3a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa3a406ae40 0x7fa3a406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80c9770 tx=0x7fa3a8009040 comp rx=0 tx=0).stop 2023-11-18T20:42:59.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 -- 172.21.15.60:0/4273031981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d2910 msgr2=0x7fa3b815e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d2910 0x7fa3b815e810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00c780 tx=0x7fa3ac00cc40 comp rx=0 tx=0).stop 2023-11-18T20:42:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 -- 172.21.15.60:0/4273031981 shutdown_connections 2023-11-18T20:42:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa3a406ae40 0x7fa3a406d300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3b815d330 0x7fa3b80b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3b80d3900 0x7fa3b81544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 --2- 172.21.15.60:0/4273031981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3b80d2910 0x7fa3b815e810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 -- 172.21.15.60:0/4273031981 >> 172.21.15.60:0/4273031981 conn(0x7fa3b8072020 msgr2=0x7fa3b80d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 -- 172.21.15.60:0/4273031981 shutdown_connections 2023-11-18T20:42:59.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.234+0000 7fa3bd251640 1 -- 172.21.15.60:0/4273031981 wait complete. 2023-11-18T20:42:59.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:42:59.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:42:59.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.358+0000 7fa51b50f640 1 Processor -- start 2023-11-18T20:42:59.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.358+0000 7fa51b50f640 1 -- start start 2023-11-18T20:42:59.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.359+0000 7fa51b50f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa51413e090 0x7fa51406e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.359+0000 7fa51b50f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa51415f830 0x7fa51406ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.359+0000 7fa51b50f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51413e470 con 0x7fa51415b5c0 2023-11-18T20:42:59.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.359+0000 7fa51b50f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51415c5d0 con 0x7fa51415f830 2023-11-18T20:42:59.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.359+0000 7fa51b50f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51415c2f0 con 0x7fa51413e090 2023-11-18T20:42:59.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.359+0000 7fa518aaa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa51415f830 0x7fa51406ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.360+0000 7fa518aaa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa51415f830 0x7fa51406ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57824/0 (socket says 172.21.15.60:57824) 2023-11-18T20:42:59.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.360+0000 7fa518aaa640 1 -- 172.21.15.60:0/3228620052 learned_addr learned my addr 172.21.15.60:0/3228620052 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:59.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.360+0000 7fa5192ab640 1 --2- 172.21.15.60:0/3228620052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa51413e090 0x7fa51406e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.360+0000 7fa518aaa640 1 -- 172.21.15.60:0/3228620052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa51413e090 msgr2=0x7fa51406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:59.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.360+0000 7fa518aaa640 1 --2- 172.21.15.60:0/3228620052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa51413e090 0x7fa51406e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.360+0000 7fa518aaa640 1 -- 172.21.15.60:0/3228620052 >> v1:172.21.15.100:6789/0 conn(0x7fa51415b5c0 legacy=0x7fa51415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.360+0000 7fa518aaa640 1 -- 172.21.15.60:0/3228620052 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51415ed50 con 0x7fa51415f830 2023-11-18T20:42:59.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.360+0000 7fa50bfff640 1 -- 172.21.15.60:0/3228620052 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622992865 0 0) 0x7fa51413e470 con 0x7fa51415b5c0 2023-11-18T20:42:59.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.361+0000 7fa518aaa640 1 --2- 172.21.15.60:0/3228620052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415f830 0x7fa51406ed10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa504009870 tx=0x7fa50402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72740949f64d4b09 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.361+0000 7fa50bfff640 1 -- 172.21.15.60:0/3228620052 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504031640 con 0x7fa51415f830 2023-11-18T20:42:59.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.361+0000 7fa50bfff640 1 -- 172.21.15.60:0/3228620052 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5040317a0 con 0x7fa51415f830 2023-11-18T20:42:59.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.361+0000 7fa50bfff640 1 -- 172.21.15.60:0/3228620052 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504031b30 con 0x7fa51415f830 2023-11-18T20:42:59.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.362+0000 7fa51b50f640 1 -- 172.21.15.60:0/3228620052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415f830 msgr2=0x7fa51406ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.362+0000 7fa51b50f640 1 --2- 172.21.15.60:0/3228620052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415f830 0x7fa51406ed10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa504009870 tx=0x7fa50402b8f0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.362+0000 7fa51b50f640 1 -- 172.21.15.60:0/3228620052 shutdown_connections 2023-11-18T20:42:59.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.362+0000 7fa51b50f640 1 --2- 172.21.15.60:0/3228620052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415f830 0x7fa51406ed10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.362+0000 7fa51b50f640 1 --2- 172.21.15.60:0/3228620052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa51413e090 0x7fa51406e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.362+0000 7fa51b50f640 1 -- 172.21.15.60:0/3228620052 >> 172.21.15.60:0/3228620052 conn(0x7fa51406a8c0 msgr2=0x7fa51406acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:59.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.362+0000 7fa51b50f640 1 -- 172.21.15.60:0/3228620052 shutdown_connections 2023-11-18T20:42:59.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.362+0000 7fa51b50f640 1 -- 172.21.15.60:0/3228620052 wait complete. 2023-11-18T20:42:59.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.363+0000 7fa51b50f640 1 Processor -- start 2023-11-18T20:42:59.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.363+0000 7fa51b50f640 1 -- start start 2023-11-18T20:42:59.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa51b50f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa51413e090 0x7fa51415e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa51b50f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415b5c0 0x7fa51413e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa51b50f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51415f830 0x7fa5140b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa51b50f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51406c660 con 0x7fa51415b5c0 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa51b50f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51406ba50 con 0x7fa51415f830 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa51b50f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51406bd30 con 0x7fa51413e090 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa518aaa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415b5c0 0x7fa51413e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa518aaa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415b5c0 0x7fa51413e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57826/0 (socket says 172.21.15.60:57826) 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa518aaa640 1 -- 172.21.15.60:0/794545267 learned_addr learned my addr 172.21.15.60:0/794545267 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa5192ab640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa51413e090 0x7fa51415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.364+0000 7fa519aac640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51415f830 0x7fa5140b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa518aaa640 1 -- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa51413e090 msgr2=0x7fa51415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa518aaa640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa51413e090 0x7fa51415e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa518aaa640 1 -- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51415f830 msgr2=0x7fa5140b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa518aaa640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51415f830 0x7fa5140b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa518aaa640 1 -- 172.21.15.60:0/794545267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040094d0 con 0x7fa51415b5c0 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa519aac640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51415f830 0x7fa5140b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa518aaa640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415b5c0 0x7fa51413e470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa504034040 tx=0x7fa5040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa509ffb640 1 -- 172.21.15.60:0/794545267 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504004230 con 0x7fa51415b5c0 2023-11-18T20:42:59.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa509ffb640 1 -- 172.21.15.60:0/794545267 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5040043b0 con 0x7fa51415b5c0 2023-11-18T20:42:59.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa51b50f640 1 -- 172.21.15.60:0/794545267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51406c010 con 0x7fa51415b5c0 2023-11-18T20:42:59.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa509ffb640 1 -- 172.21.15.60:0/794545267 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504004520 con 0x7fa51415b5c0 2023-11-18T20:42:59.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.365+0000 7fa51b50f640 1 -- 172.21.15.60:0/794545267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51415e580 con 0x7fa51415b5c0 2023-11-18T20:42:59.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.366+0000 7fa51b50f640 1 -- 172.21.15.60:0/794545267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa514067fa0 con 0x7fa51415b5c0 2023-11-18T20:42:59.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.367+0000 7fa509ffb640 1 -- 172.21.15.60:0/794545267 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa504042070 con 0x7fa51415b5c0 2023-11-18T20:42:59.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.367+0000 7fa509ffb640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa4f806ae40 0x7fa4f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.368+0000 7fa509ffb640 1 -- 172.21.15.60:0/794545267 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5040ab290 con 0x7fa51415b5c0 2023-11-18T20:42:59.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.368+0000 7fa5192ab640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa4f806ae40 0x7fa4f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.368+0000 7fa5192ab640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa4f806ae40 0x7fa4f806d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa510009b40 tx=0x7fa510009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.372+0000 7fa509ffb640 1 -- 172.21.15.60:0/794545267 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa504076340 con 0x7fa51415b5c0 2023-11-18T20:42:59.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.548+0000 7fa51b50f640 1 -- 172.21.15.60:0/794545267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa514067420 con 0x7fa51415b5c0 2023-11-18T20:42:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.548+0000 7fa509ffb640 1 -- 172.21.15.60:0/794545267 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa504079960 con 0x7fa51415b5c0 2023-11-18T20:42:59.550 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:42:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.550+0000 7fa51b50f640 1 -- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa4f806ae40 msgr2=0x7fa4f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.550+0000 7fa51b50f640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa4f806ae40 0x7fa4f806d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa510009b40 tx=0x7fa510009190 comp rx=0 tx=0).stop 2023-11-18T20:42:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.550+0000 7fa51b50f640 1 -- 172.21.15.60:0/794545267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415b5c0 msgr2=0x7fa51413e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.550+0000 7fa51b50f640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415b5c0 0x7fa51413e470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa504034040 tx=0x7fa5040046c0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.551+0000 7fa51b50f640 1 -- 172.21.15.60:0/794545267 shutdown_connections 2023-11-18T20:42:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.551+0000 7fa51b50f640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa4f806ae40 0x7fa4f806d300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.551+0000 7fa51b50f640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa51415f830 0x7fa5140b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.551+0000 7fa51b50f640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa51415b5c0 0x7fa51413e470 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.551+0000 7fa51b50f640 1 --2- 172.21.15.60:0/794545267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa51413e090 0x7fa51415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.551+0000 7fa51b50f640 1 -- 172.21.15.60:0/794545267 >> 172.21.15.60:0/794545267 conn(0x7fa51406a8c0 msgr2=0x7fa514157b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.551+0000 7fa51b50f640 1 -- 172.21.15.60:0/794545267 shutdown_connections 2023-11-18T20:42:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.551+0000 7fa51b50f640 1 -- 172.21.15.60:0/794545267 wait complete. 2023-11-18T20:42:59.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:42:59.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.672+0000 7f040f361640 1 Processor -- start 2023-11-18T20:42:59.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.672+0000 7f040f361640 1 -- start start 2023-11-18T20:42:59.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f040f361640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0408143c50 0x7f04080721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f040f361640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04080b3d40 con 0x7f04080b4440 2023-11-18T20:42:59.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f040f361640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04080716f0 con 0x7f04080b3760 2023-11-18T20:42:59.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f040f361640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04080b4020 con 0x7f0408143c50 2023-11-18T20:42:59.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f040d0fd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0408143c50 0x7f04080721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f040c8fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f04080b4440 0x7f04080b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56906/0 (socket says 172.21.15.60:56906) 2023-11-18T20:42:59.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f040c8fc640 1 -- 172.21.15.60:0/285504383 learned_addr learned my addr 172.21.15.60:0/285504383 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:59.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f040d0fd640 1 -- 172.21.15.60:0/285504383 >> v1:172.21.15.100:6789/0 conn(0x7f04080b3760 legacy=0x7f040806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f040d0fd640 1 -- 172.21.15.60:0/285504383 >> v1:172.21.15.60:6789/0 conn(0x7f04080b4440 legacy=0x7f04080b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f03fffff640 1 -- 172.21.15.60:0/285504383 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990103781 0 0) 0x7f04080b3d40 con 0x7f04080b4440 2023-11-18T20:42:59.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.674+0000 7f040d0fd640 1 -- 172.21.15.60:0/285504383 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04080719d0 con 0x7f0408143c50 2023-11-18T20:42:59.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.675+0000 7f040d0fd640 1 --2- 172.21.15.60:0/285504383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0408143c50 0x7f04080721d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0404009d90 tx=0x7f040402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff74c9c29b04b1f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.675+0000 7f03fffff640 1 -- 172.21.15.60:0/285504383 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040402cc60 con 0x7f0408143c50 2023-11-18T20:42:59.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.675+0000 7f03fffff640 1 -- 172.21.15.60:0/285504383 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040402cdc0 con 0x7f0408143c50 2023-11-18T20:42:59.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.675+0000 7f03fffff640 1 -- 172.21.15.60:0/285504383 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404034660 con 0x7f0408143c50 2023-11-18T20:42:59.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.676+0000 7f040f361640 1 -- 172.21.15.60:0/285504383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0408143c50 msgr2=0x7f04080721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.676+0000 7f040f361640 1 --2- 172.21.15.60:0/285504383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0408143c50 0x7f04080721d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0404009d90 tx=0x7f040402beb0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.676+0000 7f040f361640 1 -- 172.21.15.60:0/285504383 shutdown_connections 2023-11-18T20:42:59.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.676+0000 7f040f361640 1 --2- 172.21.15.60:0/285504383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0408143c50 0x7f04080721d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.676+0000 7f040f361640 1 -- 172.21.15.60:0/285504383 >> 172.21.15.60:0/285504383 conn(0x7f04080b1d50 msgr2=0x7f04080b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:59.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.676+0000 7f040f361640 1 -- 172.21.15.60:0/285504383 shutdown_connections 2023-11-18T20:42:59.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.676+0000 7f040f361640 1 -- 172.21.15.60:0/285504383 wait complete. 2023-11-18T20:42:59.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.678+0000 7f040f361640 1 Processor -- start 2023-11-18T20:42:59.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.678+0000 7f040f361640 1 -- start start 2023-11-18T20:42:59.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040f361640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04080b3760 0x7f0408155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040f361640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04080b4440 0x7f040815de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040f361640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0408143c50 0x7f040815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040f361640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040812f090 con 0x7f04080b3760 2023-11-18T20:42:59.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040f361640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040815b9f0 con 0x7f0408143c50 2023-11-18T20:42:59.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040f361640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04081562b0 con 0x7f04080b4440 2023-11-18T20:42:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040d0fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04080b3760 0x7f0408155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040d0fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04080b3760 0x7f0408155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57858/0 (socket says 172.21.15.60:57858) 2023-11-18T20:42:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040d0fd640 1 -- 172.21.15.60:0/3446863202 learned_addr learned my addr 172.21.15.60:0/3446863202 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040d8fe640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0408143c50 0x7f040815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.679+0000 7f040c8fc640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04080b4440 0x7f040815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f040d0fd640 1 -- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04080b4440 msgr2=0x7f040815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f040d0fd640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04080b4440 0x7f040815de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f040d0fd640 1 -- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0408143c50 msgr2=0x7f040815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f040d0fd640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0408143c50 0x7f040815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f040d0fd640 1 -- 172.21.15.60:0/3446863202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0404009a70 con 0x7f04080b3760 2023-11-18T20:42:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f040d8fe640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0408143c50 0x7f040815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f040c8fc640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04080b4440 0x7f040815de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f040d0fd640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04080b3760 0x7f0408155d70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0404009eb0 tx=0x7f0404004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f03fdffb640 1 -- 172.21.15.60:0/3446863202 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040403e610 con 0x7f04080b3760 2023-11-18T20:42:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f03fdffb640 1 -- 172.21.15.60:0/3446863202 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040403e770 con 0x7f04080b3760 2023-11-18T20:42:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f040f361640 1 -- 172.21.15.60:0/3446863202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04080cf1e0 con 0x7f04080b3760 2023-11-18T20:42:59.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f03fdffb640 1 -- 172.21.15.60:0/3446863202 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040403eb20 con 0x7f04080b3760 2023-11-18T20:42:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.680+0000 7f040f361640 1 -- 172.21.15.60:0/3446863202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040812f5c0 con 0x7f04080b3760 2023-11-18T20:42:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.681+0000 7f040f361640 1 -- 172.21.15.60:0/3446863202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040813d0f0 con 0x7f04080b3760 2023-11-18T20:42:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.682+0000 7f03fdffb640 1 -- 172.21.15.60:0/3446863202 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f040403b050 con 0x7f04080b3760 2023-11-18T20:42:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.682+0000 7f03fdffb640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f03ec06ae40 0x7f03ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.682+0000 7f03fdffb640 1 -- 172.21.15.60:0/3446863202 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04040abc30 con 0x7f04080b3760 2023-11-18T20:42:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.682+0000 7f040c8fc640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f03ec06ae40 0x7f03ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.683+0000 7f040c8fc640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f03ec06ae40 0x7f03ec06d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f040813cb50 tx=0x7f03f8007c50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.687+0000 7f03fdffb640 1 -- 172.21.15.60:0/3446863202 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0404076e00 con 0x7f04080b3760 2023-11-18T20:42:59.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.863+0000 7f040f361640 1 -- 172.21.15.60:0/3446863202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0408067420 con 0x7f04080b3760 2023-11-18T20:42:59.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.863+0000 7f03fdffb640 1 -- 172.21.15.60:0/3446863202 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0404039020 con 0x7f04080b3760 2023-11-18T20:42:59.865 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:42:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.865+0000 7f040f361640 1 -- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f03ec06ae40 msgr2=0x7f03ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.865+0000 7f040f361640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f03ec06ae40 0x7f03ec06d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f040813cb50 tx=0x7f03f8007c50 comp rx=0 tx=0).stop 2023-11-18T20:42:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.865+0000 7f040f361640 1 -- 172.21.15.60:0/3446863202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04080b3760 msgr2=0x7f0408155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.865+0000 7f040f361640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04080b3760 0x7f0408155d70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0404009eb0 tx=0x7f0404004a40 comp rx=0 tx=0).stop 2023-11-18T20:42:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.865+0000 7f040f361640 1 -- 172.21.15.60:0/3446863202 shutdown_connections 2023-11-18T20:42:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.865+0000 7f040f361640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f03ec06ae40 0x7f03ec06d300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.865+0000 7f040f361640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0408143c50 0x7f040815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.865+0000 7f040f361640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04080b4440 0x7f040815de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.865+0000 7f040f361640 1 --2- 172.21.15.60:0/3446863202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04080b3760 0x7f0408155d70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.866+0000 7f040f361640 1 -- 172.21.15.60:0/3446863202 >> 172.21.15.60:0/3446863202 conn(0x7f04080b1d50 msgr2=0x7f04080b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:59.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.866+0000 7f040f361640 1 -- 172.21.15.60:0/3446863202 shutdown_connections 2023-11-18T20:42:59.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.866+0000 7f040f361640 1 -- 172.21.15.60:0/3446863202 wait complete. 2023-11-18T20:42:59.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:42:59.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.985+0000 7fed7fd89640 1 Processor -- start 2023-11-18T20:42:59.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.985+0000 7fed7fd89640 1 -- start start 2023-11-18T20:42:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.986+0000 7fed7fd89640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed780ceb70 0x7fed780cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.986+0000 7fed7fd89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780b4370 0x7fed780b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.986+0000 7fed7fd89640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed78071770 con 0x7fed78144980 2023-11-18T20:42:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.986+0000 7fed7fd89640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7815ced0 con 0x7fed780ceb70 2023-11-18T20:42:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.986+0000 7fed7fd89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed780b4ce0 con 0x7fed780b4370 2023-11-18T20:42:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.986+0000 7fed7d324640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed780ceb70 0x7fed780cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.986+0000 7fed7db25640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fed78144980 0x7fed7815c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56908/0 (socket says 172.21.15.60:56908) 2023-11-18T20:42:59.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.986+0000 7fed7db25640 1 -- 172.21.15.60:0/1143867349 learned_addr learned my addr 172.21.15.60:0/1143867349 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:42:59.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.986+0000 7fed7e326640 1 --2- 172.21.15.60:0/1143867349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780b4370 0x7fed780b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.986+0000 7fed7d324640 1 -- 172.21.15.60:0/1143867349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780b4370 msgr2=0x7fed780b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:42:59.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.987+0000 7fed7d324640 1 --2- 172.21.15.60:0/1143867349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780b4370 0x7fed780b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.987+0000 7fed7d324640 1 -- 172.21.15.60:0/1143867349 >> v1:172.21.15.60:6789/0 conn(0x7fed78144980 legacy=0x7fed7815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.987+0000 7fed7d324640 1 -- 172.21.15.60:0/1143867349 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed780ccdc0 con 0x7fed780ceb70 2023-11-18T20:42:59.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.987+0000 7fed7e326640 1 --2- 172.21.15.60:0/1143867349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780b4370 0x7fed780b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:42:59.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.987+0000 7fed7d324640 1 --2- 172.21.15.60:0/1143867349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed780ceb70 0x7fed780cc450 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed740097a0 tx=0x7fed7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52b6547d36289f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.987+0000 7fed7cb23640 1 -- 172.21.15.60:0/1143867349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74031830 con 0x7fed780ceb70 2023-11-18T20:42:59.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.987+0000 7fed7cb23640 1 -- 172.21.15.60:0/1143867349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed74031990 con 0x7fed780ceb70 2023-11-18T20:42:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.987+0000 7fed7cb23640 1 -- 172.21.15.60:0/1143867349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74031d20 con 0x7fed780ceb70 2023-11-18T20:42:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.988+0000 7fed7fd89640 1 -- 172.21.15.60:0/1143867349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed780ceb70 msgr2=0x7fed780cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.988+0000 7fed7fd89640 1 --2- 172.21.15.60:0/1143867349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed780ceb70 0x7fed780cc450 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed740097a0 tx=0x7fed7402b8d0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.988+0000 7fed7fd89640 1 -- 172.21.15.60:0/1143867349 shutdown_connections 2023-11-18T20:42:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.988+0000 7fed7fd89640 1 --2- 172.21.15.60:0/1143867349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780b4370 0x7fed780b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.988+0000 7fed7fd89640 1 --2- 172.21.15.60:0/1143867349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed780ceb70 0x7fed780cc450 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.988+0000 7fed7fd89640 1 -- 172.21.15.60:0/1143867349 >> 172.21.15.60:0/1143867349 conn(0x7fed7806a4d0 msgr2=0x7fed7806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:42:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.988+0000 7fed7fd89640 1 -- 172.21.15.60:0/1143867349 shutdown_connections 2023-11-18T20:42:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.988+0000 7fed7fd89640 1 -- 172.21.15.60:0/1143867349 wait complete. 2023-11-18T20:42:59.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.989+0000 7fed7fd89640 1 Processor -- start 2023-11-18T20:42:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.989+0000 7fed7fd89640 1 -- start start 2023-11-18T20:42:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7fd89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed780b4370 0x7fed781f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7fd89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780ceb70 0x7fed780cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7fd89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed78144980 0x7fed7813f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7fd89640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7813e360 con 0x7fed78144980 2023-11-18T20:42:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7fd89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781f73d0 con 0x7fed780b4370 2023-11-18T20:42:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7fd89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed780b3f90 con 0x7fed780ceb70 2023-11-18T20:42:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7e326640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed78144980 0x7fed7813f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7e326640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed78144980 0x7fed7813f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57888/0 (socket says 172.21.15.60:57888) 2023-11-18T20:42:59.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7e326640 1 -- 172.21.15.60:0/2124023720 learned_addr learned my addr 172.21.15.60:0/2124023720 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:42:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7d324640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780ceb70 0x7fed780cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7db25640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed780b4370 0x7fed781f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7e326640 1 -- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780ceb70 msgr2=0x7fed780cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7e326640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780ceb70 0x7fed780cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7e326640 1 -- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed780b4370 msgr2=0x7fed781f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:42:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7e326640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed780b4370 0x7fed781f6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:42:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.990+0000 7fed7e326640 1 -- 172.21.15.60:0/2124023720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed74009400 con 0x7fed78144980 2023-11-18T20:42:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.991+0000 7fed7e326640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed78144980 0x7fed7813f4c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fed70007ea0 tx=0x7fed70007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.991+0000 7fed6e7fc640 1 -- 172.21.15.60:0/2124023720 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70003b90 con 0x7fed78144980 2023-11-18T20:42:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.991+0000 7fed6e7fc640 1 -- 172.21.15.60:0/2124023720 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7000f040 con 0x7fed78144980 2023-11-18T20:42:59.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.991+0000 7fed7fd89640 1 -- 172.21.15.60:0/2124023720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed780b2300 con 0x7fed78144980 2023-11-18T20:42:59.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.991+0000 7fed6e7fc640 1 -- 172.21.15.60:0/2124023720 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7000de70 con 0x7fed78144980 2023-11-18T20:42:59.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.991+0000 7fed7fd89640 1 -- 172.21.15.60:0/2124023720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed781f6e10 con 0x7fed78144980 2023-11-18T20:42:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.992+0000 7fed7fd89640 1 -- 172.21.15.60:0/2124023720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed780b2160 con 0x7fed78144980 2023-11-18T20:42:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.992+0000 7fed6e7fc640 1 -- 172.21.15.60:0/2124023720 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed70004030 con 0x7fed78144980 2023-11-18T20:42:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.993+0000 7fed6e7fc640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed5c06ae40 0x7fed5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:42:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.993+0000 7fed6e7fc640 1 -- 172.21.15.60:0/2124023720 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed70089850 con 0x7fed78144980 2023-11-18T20:42:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.993+0000 7fed7db25640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed5c06ae40 0x7fed5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:42:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.993+0000 7fed7db25640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed5c06ae40 0x7fed5c06d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fed68005880 tx=0x7fed68009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:42:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:42:59.997+0000 7fed6e7fc640 1 -- 172.21.15.60:0/2124023720 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed780b2160 con 0x7fed78144980 2023-11-18T20:43:00.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.176+0000 7fed7fd89640 1 -- 172.21.15.60:0/2124023720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed780b2160 con 0x7fed78144980 2023-11-18T20:43:00.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.177+0000 7fed6e7fc640 1 -- 172.21.15.60:0/2124023720 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed780b2160 con 0x7fed78144980 2023-11-18T20:43:00.178 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:00.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 -- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed5c06ae40 msgr2=0x7fed5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed5c06ae40 0x7fed5c06d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fed68005880 tx=0x7fed68009040 comp rx=0 tx=0).stop 2023-11-18T20:43:00.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 -- 172.21.15.60:0/2124023720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed78144980 msgr2=0x7fed7813f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed78144980 0x7fed7813f4c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fed70007ea0 tx=0x7fed70007f90 comp rx=0 tx=0).stop 2023-11-18T20:43:00.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 -- 172.21.15.60:0/2124023720 shutdown_connections 2023-11-18T20:43:00.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed5c06ae40 0x7fed5c06d300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed78144980 0x7fed7813f4c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed780ceb70 0x7fed780cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 --2- 172.21.15.60:0/2124023720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed780b4370 0x7fed781f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 -- 172.21.15.60:0/2124023720 >> 172.21.15.60:0/2124023720 conn(0x7fed7806a4d0 msgr2=0x7fed7815b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:00.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 -- 172.21.15.60:0/2124023720 shutdown_connections 2023-11-18T20:43:00.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.179+0000 7fed7fd89640 1 -- 172.21.15.60:0/2124023720 wait complete. 2023-11-18T20:43:00.191 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:00.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.296+0000 7f944f6af640 1 Processor -- start 2023-11-18T20:43:00.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.296+0000 7f944f6af640 1 -- start start 2023-11-18T20:43:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944f6af640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9448144980 0x7f944815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944f6af640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f94480b4370 0x7f94480b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944f6af640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9448071770 con 0x7f94480ceb70 2023-11-18T20:43:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944f6af640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94480ccb60 con 0x7f9448144980 2023-11-18T20:43:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944f6af640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94480b4ce0 con 0x7f94480b4370 2023-11-18T20:43:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944d44b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9448144980 0x7f944815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944d44b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9448144980 0x7f944815c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57890/0 (socket says 172.21.15.60:57890) 2023-11-18T20:43:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944d44b640 1 -- 172.21.15.60:0/2909083227 learned_addr learned my addr 172.21.15.60:0/2909083227 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944dc4c640 1 --2- 172.21.15.60:0/2909083227 >> v2:172.21.15.100:3300/0 conn(0x7f94480b4370 0x7f94480b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944d44b640 1 -- 172.21.15.60:0/2909083227 >> v2:172.21.15.100:3300/0 conn(0x7f94480b4370 msgr2=0x7f94480b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944d44b640 1 --2- 172.21.15.60:0/2909083227 >> v2:172.21.15.100:3300/0 conn(0x7f94480b4370 0x7f94480b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944d44b640 1 -- 172.21.15.60:0/2909083227 >> v1:172.21.15.100:6789/0 conn(0x7f94480ceb70 legacy=0x7f94480cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.297+0000 7f944d44b640 1 -- 172.21.15.60:0/2909083227 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94480ccdc0 con 0x7f9448144980 2023-11-18T20:43:00.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.298+0000 7f944d44b640 1 --2- 172.21.15.60:0/2909083227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9448144980 0x7f944815c7c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94480716e0 tx=0x7f944402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60ae11830576281 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:00.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.298+0000 7f943ffff640 1 -- 172.21.15.60:0/2909083227 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444030860 con 0x7f9448144980 2023-11-18T20:43:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.298+0000 7f943ffff640 1 -- 172.21.15.60:0/2909083227 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94440309c0 con 0x7f9448144980 2023-11-18T20:43:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.298+0000 7f943ffff640 1 -- 172.21.15.60:0/2909083227 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444030d50 con 0x7f9448144980 2023-11-18T20:43:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.299+0000 7f944f6af640 1 -- 172.21.15.60:0/2909083227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9448144980 msgr2=0x7f944815c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.299+0000 7f944f6af640 1 --2- 172.21.15.60:0/2909083227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9448144980 0x7f944815c7c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94480716e0 tx=0x7f944402b9a0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.299+0000 7f944f6af640 1 -- 172.21.15.60:0/2909083227 shutdown_connections 2023-11-18T20:43:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.299+0000 7f944f6af640 1 --2- 172.21.15.60:0/2909083227 >> v2:172.21.15.100:3300/0 conn(0x7f94480b4370 0x7f94480b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.299+0000 7f944f6af640 1 --2- 172.21.15.60:0/2909083227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9448144980 0x7f944815c7c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.299+0000 7f944f6af640 1 -- 172.21.15.60:0/2909083227 >> 172.21.15.60:0/2909083227 conn(0x7f944806a4d0 msgr2=0x7f944806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.299+0000 7f944f6af640 1 -- 172.21.15.60:0/2909083227 shutdown_connections 2023-11-18T20:43:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.299+0000 7f944f6af640 1 -- 172.21.15.60:0/2909083227 wait complete. 2023-11-18T20:43:00.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.300+0000 7f944f6af640 1 Processor -- start 2023-11-18T20:43:00.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.300+0000 7f944f6af640 1 -- start start 2023-11-18T20:43:00.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944f6af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94480b4370 0x7f9448158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944f6af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94480ceb70 0x7f944813deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944f6af640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9448144980 0x7f94481f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944f6af640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944813e890 con 0x7f94480ceb70 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944f6af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94481f20a0 con 0x7f94480b4370 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944f6af640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944813e5b0 con 0x7f9448144980 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944cc4a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94480ceb70 0x7f944813deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944cc4a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94480ceb70 0x7f944813deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57894/0 (socket says 172.21.15.60:57894) 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944cc4a640 1 -- 172.21.15.60:0/2571611594 learned_addr learned my addr 172.21.15.60:0/2571611594 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944d44b640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94480b4370 0x7f9448158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944dc4c640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9448144980 0x7f94481f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944cc4a640 1 -- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9448144980 msgr2=0x7f94481f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944cc4a640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9448144980 0x7f94481f1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944cc4a640 1 -- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94480b4370 msgr2=0x7f9448158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944cc4a640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94480b4370 0x7f9448158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944cc4a640 1 -- 172.21.15.60:0/2571611594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94440094d0 con 0x7f94480ceb70 2023-11-18T20:43:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.301+0000 7f944cc4a640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94480ceb70 0x7f944813deb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f943800c450 tx=0x7f943800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.302+0000 7f943dffb640 1 -- 172.21.15.60:0/2571611594 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94380052d0 con 0x7f94480ceb70 2023-11-18T20:43:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.302+0000 7f943dffb640 1 -- 172.21.15.60:0/2571611594 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9438005430 con 0x7f94480ceb70 2023-11-18T20:43:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.302+0000 7f944f6af640 1 -- 172.21.15.60:0/2571611594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9448155e60 con 0x7f94480ceb70 2023-11-18T20:43:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.302+0000 7f943dffb640 1 -- 172.21.15.60:0/2571611594 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438012870 con 0x7f94480ceb70 2023-11-18T20:43:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.302+0000 7f944f6af640 1 -- 172.21.15.60:0/2571611594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9448156420 con 0x7f94480ceb70 2023-11-18T20:43:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.303+0000 7f944f6af640 1 -- 172.21.15.60:0/2571611594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9448155cc0 con 0x7f94480ceb70 2023-11-18T20:43:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.303+0000 7f943dffb640 1 -- 172.21.15.60:0/2571611594 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94380129d0 con 0x7f94480ceb70 2023-11-18T20:43:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.303+0000 7f943dffb640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f942c06ae40 0x7f942c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.304+0000 7f943dffb640 1 -- 172.21.15.60:0/2571611594 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9438088f80 con 0x7f94480ceb70 2023-11-18T20:43:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.304+0000 7f944d44b640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f942c06ae40 0x7f942c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.304+0000 7f944d44b640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f942c06ae40 0x7f942c06d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9444036000 tx=0x7f9444008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:00.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.308+0000 7f943dffb640 1 -- 172.21.15.60:0/2571611594 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94380bbe90 con 0x7f94480ceb70 2023-11-18T20:43:00.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.485+0000 7f944f6af640 1 -- 172.21.15.60:0/2571611594 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9448069400 con 0x7f94480ceb70 2023-11-18T20:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.486+0000 7f943dffb640 1 -- 172.21.15.60:0/2571611594 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f9448155cc0 con 0x7f94480ceb70 2023-11-18T20:43:00.488 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:00.488 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 -- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f942c06ae40 msgr2=0x7f942c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f942c06ae40 0x7f942c06d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9444036000 tx=0x7f9444008030 comp rx=0 tx=0).stop 2023-11-18T20:43:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 -- 172.21.15.60:0/2571611594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94480ceb70 msgr2=0x7f944813deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94480ceb70 0x7f944813deb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f943800c450 tx=0x7f943800c910 comp rx=0 tx=0).stop 2023-11-18T20:43:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 -- 172.21.15.60:0/2571611594 shutdown_connections 2023-11-18T20:43:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f942c06ae40 0x7f942c06d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9448144980 0x7f94481f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94480ceb70 0x7f944813deb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 --2- 172.21.15.60:0/2571611594 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94480b4370 0x7f9448158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 -- 172.21.15.60:0/2571611594 >> 172.21.15.60:0/2571611594 conn(0x7f944806a4d0 msgr2=0x7f944815ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:00.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 -- 172.21.15.60:0/2571611594 shutdown_connections 2023-11-18T20:43:00.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.488+0000 7f944f6af640 1 -- 172.21.15.60:0/2571611594 wait complete. 2023-11-18T20:43:00.500 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:00.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:00.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.608+0000 7fb8382d7640 1 Processor -- start 2023-11-18T20:43:00.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.608+0000 7fb8382d7640 1 -- start start 2023-11-18T20:43:00.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.609+0000 7fb8382d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb83015d330 0x7fb8300b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.609+0000 7fb8382d7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb8300d2910 0x7fb8300d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.609+0000 7fb8382d7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8300b2b10 con 0x7fb8300d3730 2023-11-18T20:43:00.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.609+0000 7fb8382d7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8300d3bc0 con 0x7fb83015d330 2023-11-18T20:43:00.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.609+0000 7fb8382d7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8300d1590 con 0x7fb8300d2910 2023-11-18T20:43:00.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.609+0000 7fb836073640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb83015d330 0x7fb8300b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.609+0000 7fb836874640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb8300d3730 0x7fb8300b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50024/0 (socket says 172.21.15.60:50024) 2023-11-18T20:43:00.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.609+0000 7fb836874640 1 -- 172.21.15.60:0/3106694642 learned_addr learned my addr 172.21.15.60:0/3106694642 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.610+0000 7fb835071640 1 -- 172.21.15.60:0/3106694642 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801907837 0 0) 0x7fb8300b2b10 con 0x7fb8300d3730 2023-11-18T20:43:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.610+0000 7fb835071640 1 -- 172.21.15.60:0/3106694642 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb818003520 con 0x7fb8300d3730 2023-11-18T20:43:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.610+0000 7fb835872640 1 --2- 172.21.15.60:0/3106694642 >> v2:172.21.15.100:3300/0 conn(0x7fb8300d2910 0x7fb8300d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.610+0000 7fb836073640 1 -- 172.21.15.60:0/3106694642 >> v2:172.21.15.100:3300/0 conn(0x7fb8300d2910 msgr2=0x7fb8300d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.610+0000 7fb836073640 1 --2- 172.21.15.60:0/3106694642 >> v2:172.21.15.100:3300/0 conn(0x7fb8300d2910 0x7fb8300d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.610+0000 7fb836073640 1 -- 172.21.15.60:0/3106694642 >> v1:172.21.15.60:6789/0 conn(0x7fb8300d3730 legacy=0x7fb8300b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.610+0000 7fb836073640 1 -- 172.21.15.60:0/3106694642 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300d1870 con 0x7fb83015d330 2023-11-18T20:43:00.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.610+0000 7fb836073640 1 --2- 172.21.15.60:0/3106694642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015d330 0x7fb8300b8b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb8300d1540 tx=0x7fb82802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b638d5cec1d982f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb835071640 1 -- 172.21.15.60:0/3106694642 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82802e070 con 0x7fb83015d330 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb835071640 1 -- 172.21.15.60:0/3106694642 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8280315a0 con 0x7fb83015d330 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb835071640 1 -- 172.21.15.60:0/3106694642 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828031930 con 0x7fb83015d330 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb8382d7640 1 -- 172.21.15.60:0/3106694642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015d330 msgr2=0x7fb8300b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb8382d7640 1 --2- 172.21.15.60:0/3106694642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015d330 0x7fb8300b8b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb8300d1540 tx=0x7fb82802b840 comp rx=0 tx=0).stop 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb8382d7640 1 -- 172.21.15.60:0/3106694642 shutdown_connections 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb8382d7640 1 --2- 172.21.15.60:0/3106694642 >> v2:172.21.15.100:3300/0 conn(0x7fb8300d2910 0x7fb8300d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb8382d7640 1 --2- 172.21.15.60:0/3106694642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015d330 0x7fb8300b8b80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb8382d7640 1 -- 172.21.15.60:0/3106694642 >> 172.21.15.60:0/3106694642 conn(0x7fb830072020 msgr2=0x7fb830072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb8382d7640 1 -- 172.21.15.60:0/3106694642 shutdown_connections 2023-11-18T20:43:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.611+0000 7fb8382d7640 1 -- 172.21.15.60:0/3106694642 wait complete. 2023-11-18T20:43:00.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.612+0000 7fb8382d7640 1 Processor -- start 2023-11-18T20:43:00.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.613+0000 7fb8382d7640 1 -- start start 2023-11-18T20:43:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.613+0000 7fb8382d7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300d2910 0x7fb830148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.613+0000 7fb8382d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300d3730 0x7fb8300cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.613+0000 7fb8382d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83015d330 0x7fb8300cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.613+0000 7fb8382d7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300cca10 con 0x7fb8300d3730 2023-11-18T20:43:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.613+0000 7fb8382d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300cccf0 con 0x7fb83015d330 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.613+0000 7fb8382d7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300ccfd0 con 0x7fb8300d2910 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.613+0000 7fb835872640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300d3730 0x7fb8300cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.613+0000 7fb835872640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300d3730 0x7fb8300cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42378/0 (socket says 172.21.15.60:42378) 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.613+0000 7fb835872640 1 -- 172.21.15.60:0/2998663439 learned_addr learned my addr 172.21.15.60:0/2998663439 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb836073640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300d2910 0x7fb830148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb836874640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83015d330 0x7fb8300cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb835872640 1 -- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300d2910 msgr2=0x7fb830148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb835872640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300d2910 0x7fb830148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb835872640 1 -- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83015d330 msgr2=0x7fb8300cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb835872640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83015d330 0x7fb8300cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb835872640 1 -- 172.21.15.60:0/2998663439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb828009400 con 0x7fb8300d3730 2023-11-18T20:43:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb836073640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300d2910 0x7fb830148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb836874640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83015d330 0x7fb8300cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb835872640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300d3730 0x7fb8300cbec0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb82400c7f0 tx=0x7fb82400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb82effd640 1 -- 172.21.15.60:0/2998663439 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824004220 con 0x7fb8300d3730 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb82effd640 1 -- 172.21.15.60:0/2998663439 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8240043a0 con 0x7fb8300d3730 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb8382d7640 1 -- 172.21.15.60:0/2998663439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8300cd2b0 con 0x7fb8300d3730 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb82effd640 1 -- 172.21.15.60:0/2998663439 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824007860 con 0x7fb8300d3730 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.614+0000 7fb8382d7640 1 -- 172.21.15.60:0/2998663439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83014c480 con 0x7fb8300d3730 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.615+0000 7fb8382d7640 1 -- 172.21.15.60:0/2998663439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb830069400 con 0x7fb8300d3730 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.616+0000 7fb82effd640 1 -- 172.21.15.60:0/2998663439 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb824007bd0 con 0x7fb8300d3730 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.616+0000 7fb82effd640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb81c06ae40 0x7fb81c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.616+0000 7fb82effd640 1 -- 172.21.15.60:0/2998663439 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8240895e0 con 0x7fb8300d3730 2023-11-18T20:43:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.616+0000 7fb836073640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb81c06ae40 0x7fb81c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.617+0000 7fb836073640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb81c06ae40 0x7fb81c06d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb828036000 tx=0x7fb82803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.620+0000 7fb82effd640 1 -- 172.21.15.60:0/2998663439 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb824054690 con 0x7fb8300d3730 2023-11-18T20:43:00.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.794+0000 7fb8382d7640 1 -- 172.21.15.60:0/2998663439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8300b38f0 con 0x7fb8300d3730 2023-11-18T20:43:00.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.794+0000 7fb82effd640 1 -- 172.21.15.60:0/2998663439 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb824057cb0 con 0x7fb8300d3730 2023-11-18T20:43:00.796 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 -- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb81c06ae40 msgr2=0x7fb81c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb81c06ae40 0x7fb81c06d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb828036000 tx=0x7fb82803d040 comp rx=0 tx=0).stop 2023-11-18T20:43:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 -- 172.21.15.60:0/2998663439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300d3730 msgr2=0x7fb8300cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300d3730 0x7fb8300cbec0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb82400c7f0 tx=0x7fb82400ccb0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 -- 172.21.15.60:0/2998663439 shutdown_connections 2023-11-18T20:43:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb81c06ae40 0x7fb81c06d300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83015d330 0x7fb8300cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300d3730 0x7fb8300cbec0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 --2- 172.21.15.60:0/2998663439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300d2910 0x7fb830148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 -- 172.21.15.60:0/2998663439 >> 172.21.15.60:0/2998663439 conn(0x7fb830072020 msgr2=0x7fb830072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:00.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 -- 172.21.15.60:0/2998663439 shutdown_connections 2023-11-18T20:43:00.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.796+0000 7fb8382d7640 1 -- 172.21.15.60:0/2998663439 wait complete. 2023-11-18T20:43:00.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:00.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.916+0000 7f2362f9e640 1 Processor -- start 2023-11-18T20:43:00.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.916+0000 7f2362f9e640 1 -- start start 2023-11-18T20:43:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f2362f9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f235c15d2d0 0x7f235c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f2362f9e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f235c0ce650 0x7f235c0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f2362f9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c1614e0 con 0x7f235c160f70 2023-11-18T20:43:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f2362f9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c1617c0 con 0x7f235c15d2d0 2023-11-18T20:43:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f2362f9e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c161aa0 con 0x7f235c0ce650 2023-11-18T20:43:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f2360d3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f235c15d2d0 0x7f235c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f235bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f235c160f70 0x7f235c0cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50026/0 (socket says 172.21.15.60:50026) 2023-11-18T20:43:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f235bfff640 1 -- 172.21.15.60:0/1178729583 learned_addr learned my addr 172.21.15.60:0/1178729583 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f236153b640 1 --2- 172.21.15.60:0/1178729583 >> v2:172.21.15.100:3300/0 conn(0x7f235c0ce650 0x7f235c0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f2360d3a640 1 -- 172.21.15.60:0/1178729583 >> v2:172.21.15.100:3300/0 conn(0x7f235c0ce650 msgr2=0x7f235c0d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f2360d3a640 1 --2- 172.21.15.60:0/1178729583 >> v2:172.21.15.100:3300/0 conn(0x7f235c0ce650 0x7f235c0d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f2360d3a640 1 -- 172.21.15.60:0/1178729583 >> v1:172.21.15.60:6789/0 conn(0x7f235c160f70 legacy=0x7f235c0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f2360d3a640 1 -- 172.21.15.60:0/1178729583 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c161d80 con 0x7f235c15d2d0 2023-11-18T20:43:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.917+0000 7f235b7fe640 1 -- 172.21.15.60:0/1178729583 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950269558 0 0) 0x7f235c1614e0 con 0x7f235c160f70 2023-11-18T20:43:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.918+0000 7f2360d3a640 1 --2- 172.21.15.60:0/1178729583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c15d2d0 0x7f235c0b8b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f235c0cf630 tx=0x7f235402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887053a36b472f42 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.918+0000 7f235b7fe640 1 -- 172.21.15.60:0/1178729583 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235402cc80 con 0x7f235c15d2d0 2023-11-18T20:43:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.918+0000 7f235b7fe640 1 -- 172.21.15.60:0/1178729583 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235402cde0 con 0x7f235c15d2d0 2023-11-18T20:43:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.918+0000 7f235b7fe640 1 -- 172.21.15.60:0/1178729583 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354034660 con 0x7f235c15d2d0 2023-11-18T20:43:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.918+0000 7f2362f9e640 1 -- 172.21.15.60:0/1178729583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c15d2d0 msgr2=0x7f235c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.918+0000 7f2362f9e640 1 --2- 172.21.15.60:0/1178729583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c15d2d0 0x7f235c0b8b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f235c0cf630 tx=0x7f235402bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.919+0000 7f2362f9e640 1 -- 172.21.15.60:0/1178729583 shutdown_connections 2023-11-18T20:43:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.919+0000 7f2362f9e640 1 --2- 172.21.15.60:0/1178729583 >> v2:172.21.15.100:3300/0 conn(0x7f235c0ce650 0x7f235c0d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.919+0000 7f2362f9e640 1 --2- 172.21.15.60:0/1178729583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c15d2d0 0x7f235c0b8b90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.919+0000 7f2362f9e640 1 -- 172.21.15.60:0/1178729583 >> 172.21.15.60:0/1178729583 conn(0x7f235c072230 msgr2=0x7f235c0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.919+0000 7f2362f9e640 1 -- 172.21.15.60:0/1178729583 shutdown_connections 2023-11-18T20:43:00.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.919+0000 7f2362f9e640 1 -- 172.21.15.60:0/1178729583 wait complete. 2023-11-18T20:43:00.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.920+0000 7f2362f9e640 1 Processor -- start 2023-11-18T20:43:00.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.920+0000 7f2362f9e640 1 -- start start 2023-11-18T20:43:00.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2362f9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c0ce650 0x7f235c14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2362f9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c15d2d0 0x7f235c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2362f9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c160f70 0x7f235c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2362f9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c159de0 con 0x7f235c0ce650 2023-11-18T20:43:00.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2362f9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c1591d0 con 0x7f235c160f70 2023-11-18T20:43:00.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2362f9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c1594b0 con 0x7f235c15d2d0 2023-11-18T20:43:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2360d3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c0ce650 0x7f235c14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2360d3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c0ce650 0x7f235c14e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42402/0 (socket says 172.21.15.60:42402) 2023-11-18T20:43:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2360d3a640 1 -- 172.21.15.60:0/1642948802 learned_addr learned my addr 172.21.15.60:0/1642948802 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f235bfff640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c15d2d0 0x7f235c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f236153b640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c160f70 0x7f235c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2360d3a640 1 -- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c15d2d0 msgr2=0x7f235c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2360d3a640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c15d2d0 0x7f235c0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2360d3a640 1 -- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c160f70 msgr2=0x7f235c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2360d3a640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c160f70 0x7f235c0b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:00.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2360d3a640 1 -- 172.21.15.60:0/1642948802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2354009a70 con 0x7f235c0ce650 2023-11-18T20:43:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f235bfff640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c15d2d0 0x7f235c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f236153b640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c160f70 0x7f235c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2360d3a640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c0ce650 0x7f235c14e650 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2354037040 tx=0x7f2354004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f23597fa640 1 -- 172.21.15.60:0/1642948802 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354040650 con 0x7f235c0ce650 2023-11-18T20:43:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f23597fa640 1 -- 172.21.15.60:0/1642948802 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23540407b0 con 0x7f235c0ce650 2023-11-18T20:43:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.921+0000 7f2362f9e640 1 -- 172.21.15.60:0/1642948802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c159790 con 0x7f235c0ce650 2023-11-18T20:43:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.922+0000 7f23597fa640 1 -- 172.21.15.60:0/1642948802 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354040b60 con 0x7f235c0ce650 2023-11-18T20:43:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.922+0000 7f2362f9e640 1 -- 172.21.15.60:0/1642948802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c158b70 con 0x7f235c0ce650 2023-11-18T20:43:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.923+0000 7f2362f9e640 1 -- 172.21.15.60:0/1642948802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c158d00 con 0x7f235c0ce650 2023-11-18T20:43:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.923+0000 7f23597fa640 1 -- 172.21.15.60:0/1642948802 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2354040cc0 con 0x7f235c0ce650 2023-11-18T20:43:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.923+0000 7f23597fa640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f233c06ae70 0x7f233c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.924+0000 7f23597fa640 1 -- 172.21.15.60:0/1642948802 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23540ac370 con 0x7f235c0ce650 2023-11-18T20:43:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.924+0000 7f235bfff640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f233c06ae70 0x7f233c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.924+0000 7f235bfff640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f233c06ae70 0x7f233c06d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f235c0d2b30 tx=0x7f2344009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:00.928+0000 7f23597fa640 1 -- 172.21.15.60:0/1642948802 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23540775d0 con 0x7f235c0ce650 2023-11-18T20:43:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.103+0000 7f2362f9e640 1 -- 172.21.15.60:0/1642948802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f235c069400 con 0x7f235c0ce650 2023-11-18T20:43:01.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.103+0000 7f23597fa640 1 -- 172.21.15.60:0/1642948802 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f235407abf0 con 0x7f235c0ce650 2023-11-18T20:43:01.105 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:01.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.105+0000 7f2362f9e640 1 -- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f233c06ae70 msgr2=0x7f233c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.105+0000 7f2362f9e640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f233c06ae70 0x7f233c06d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f235c0d2b30 tx=0x7f2344009040 comp rx=0 tx=0).stop 2023-11-18T20:43:01.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.105+0000 7f2362f9e640 1 -- 172.21.15.60:0/1642948802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c0ce650 msgr2=0x7f235c14e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.105+0000 7f2362f9e640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c0ce650 0x7f235c14e650 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2354037040 tx=0x7f2354004a40 comp rx=0 tx=0).stop 2023-11-18T20:43:01.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.105+0000 7f2362f9e640 1 -- 172.21.15.60:0/1642948802 shutdown_connections 2023-11-18T20:43:01.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.105+0000 7f2362f9e640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f233c06ae70 0x7f233c06d330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.105+0000 7f2362f9e640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c160f70 0x7f235c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.105+0000 7f2362f9e640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c15d2d0 0x7f235c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.105+0000 7f2362f9e640 1 --2- 172.21.15.60:0/1642948802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c0ce650 0x7f235c14e650 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.105+0000 7f2362f9e640 1 -- 172.21.15.60:0/1642948802 >> 172.21.15.60:0/1642948802 conn(0x7f235c072230 msgr2=0x7f235c0d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:01.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.106+0000 7f2362f9e640 1 -- 172.21.15.60:0/1642948802 shutdown_connections 2023-11-18T20:43:01.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.106+0000 7f2362f9e640 1 -- 172.21.15.60:0/1642948802 wait complete. 2023-11-18T20:43:01.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:01.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.226+0000 7f98f4701640 1 Processor -- start 2023-11-18T20:43:01.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.226+0000 7f98f4701640 1 -- start start 2023-11-18T20:43:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f4701640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec143490 0x7f98ec0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f4701640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f98ec15e730 0x7f98ec0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f4701640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98ec15a4e0 0x7f98ec15a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f4701640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ec15b520 con 0x7f98ec15a4e0 2023-11-18T20:43:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f4701640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ec143870 con 0x7f98ec15e730 2023-11-18T20:43:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f4701640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec15e2a0 con 0x7f98ec143490 2023-11-18T20:43:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f2c9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98ec15a4e0 0x7f98ec15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f2c9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98ec15a4e0 0x7f98ec15a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42404/0 (socket says 172.21.15.60:42404) 2023-11-18T20:43:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f2c9e640 1 -- 172.21.15.60:0/2596384133 learned_addr learned my addr 172.21.15.60:0/2596384133 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:01.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f1c9c640 1 --2- 172.21.15.60:0/2596384133 >> v2:172.21.15.100:3300/0 conn(0x7f98ec15e730 0x7f98ec0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.227+0000 7f98f249d640 1 --2- 172.21.15.60:0/2596384133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec143490 0x7f98ec0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.228+0000 7f98f2c9e640 1 -- 172.21.15.60:0/2596384133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec143490 msgr2=0x7f98ec0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:01.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.228+0000 7f98f2c9e640 1 --2- 172.21.15.60:0/2596384133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec143490 0x7f98ec0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.228+0000 7f98f2c9e640 1 -- 172.21.15.60:0/2596384133 >> v2:172.21.15.100:3300/0 conn(0x7f98ec15e730 msgr2=0x7f98ec0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:01.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.228+0000 7f98f2c9e640 1 --2- 172.21.15.60:0/2596384133 >> v2:172.21.15.100:3300/0 conn(0x7f98ec15e730 0x7f98ec0b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.228+0000 7f98f2c9e640 1 -- 172.21.15.60:0/2596384133 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec15b240 con 0x7f98ec15a4e0 2023-11-18T20:43:01.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.228+0000 7f98f1c9c640 1 --2- 172.21.15.60:0/2596384133 >> v2:172.21.15.100:3300/0 conn(0x7f98ec15e730 0x7f98ec0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:01.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.228+0000 7f98f2c9e640 1 --2- 172.21.15.60:0/2596384133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec15a4e0 0x7f98ec15a970 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f98ec06e680 tx=0x7f98dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e466c7349da92f18 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f149b640 1 -- 172.21.15.60:0/2596384133 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc02ca70 con 0x7f98ec15a4e0 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f149b640 1 -- 172.21.15.60:0/2596384133 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc02cbd0 con 0x7f98ec15a4e0 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f149b640 1 -- 172.21.15.60:0/2596384133 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0343f0 con 0x7f98ec15a4e0 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f4701640 1 -- 172.21.15.60:0/2596384133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec15a4e0 msgr2=0x7f98ec15a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f4701640 1 --2- 172.21.15.60:0/2596384133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec15a4e0 0x7f98ec15a970 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f98ec06e680 tx=0x7f98dc02bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f4701640 1 -- 172.21.15.60:0/2596384133 shutdown_connections 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f4701640 1 --2- 172.21.15.60:0/2596384133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec15a4e0 0x7f98ec15a970 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f4701640 1 --2- 172.21.15.60:0/2596384133 >> v2:172.21.15.100:3300/0 conn(0x7f98ec15e730 0x7f98ec0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f4701640 1 --2- 172.21.15.60:0/2596384133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec143490 0x7f98ec0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f4701640 1 -- 172.21.15.60:0/2596384133 >> 172.21.15.60:0/2596384133 conn(0x7f98ec06a4d0 msgr2=0x7f98ec06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f4701640 1 -- 172.21.15.60:0/2596384133 shutdown_connections 2023-11-18T20:43:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.229+0000 7f98f4701640 1 -- 172.21.15.60:0/2596384133 wait complete. 2023-11-18T20:43:01.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.230+0000 7f98f4701640 1 Processor -- start 2023-11-18T20:43:01.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.230+0000 7f98f4701640 1 -- start start 2023-11-18T20:43:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f4701640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec143490 0x7f98ec13b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f4701640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec15a4e0 0x7f98ec070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f4701640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec15e730 0x7f98ec0708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f4701640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec15b660 con 0x7f98ec143490 2023-11-18T20:43:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f4701640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec15c0e0 con 0x7f98ec15e730 2023-11-18T20:43:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f4701640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec0ac8e0 con 0x7f98ec15a4e0 2023-11-18T20:43:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f249d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec143490 0x7f98ec13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f249d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec143490 0x7f98ec13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42420/0 (socket says 172.21.15.60:42420) 2023-11-18T20:43:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f249d640 1 -- 172.21.15.60:0/2246006900 learned_addr learned my addr 172.21.15.60:0/2246006900 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f2c9e640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec15e730 0x7f98ec0708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f1c9c640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec15a4e0 0x7f98ec070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f249d640 1 -- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec15a4e0 msgr2=0x7f98ec070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f249d640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec15a4e0 0x7f98ec070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f249d640 1 -- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec15e730 msgr2=0x7f98ec0708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f249d640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec15e730 0x7f98ec0708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.231+0000 7f98f249d640 1 -- 172.21.15.60:0/2246006900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc009b30 con 0x7f98ec143490 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.232+0000 7f98f249d640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec143490 0x7f98ec13b160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f98e400d800 tx=0x7f98e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.232+0000 7f98eaffd640 1 -- 172.21.15.60:0/2246006900 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e400b6c0 con 0x7f98ec143490 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.232+0000 7f98eaffd640 1 -- 172.21.15.60:0/2246006900 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e400b820 con 0x7f98ec143490 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.232+0000 7f98f4701640 1 -- 172.21.15.60:0/2246006900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ec15b840 con 0x7f98ec143490 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.232+0000 7f98eaffd640 1 -- 172.21.15.60:0/2246006900 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e400bbb0 con 0x7f98ec143490 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.232+0000 7f98f4701640 1 -- 172.21.15.60:0/2246006900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ec15be00 con 0x7f98ec143490 2023-11-18T20:43:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.233+0000 7f98f4701640 1 -- 172.21.15.60:0/2246006900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ec069400 con 0x7f98ec143490 2023-11-18T20:43:01.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.233+0000 7f98eaffd640 1 -- 172.21.15.60:0/2246006900 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98e400bd10 con 0x7f98ec143490 2023-11-18T20:43:01.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.234+0000 7f98eaffd640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98d806ae40 0x7f98d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.234+0000 7f98eaffd640 1 -- 172.21.15.60:0/2246006900 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98e4089b20 con 0x7f98ec143490 2023-11-18T20:43:01.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.234+0000 7f98f1c9c640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98d806ae40 0x7f98d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.234+0000 7f98f1c9c640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98d806ae40 0x7f98d806d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98ec13cc60 tx=0x7f98e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:01.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.238+0000 7f98eaffd640 1 -- 172.21.15.60:0/2246006900 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98e4054c60 con 0x7f98ec143490 2023-11-18T20:43:01.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.412+0000 7f98f4701640 1 -- 172.21.15.60:0/2246006900 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98ec0b2820 con 0x7f98ec143490 2023-11-18T20:43:01.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.412+0000 7f98eaffd640 1 -- 172.21.15.60:0/2246006900 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98e4058280 con 0x7f98ec143490 2023-11-18T20:43:01.415 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 -- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98d806ae40 msgr2=0x7f98d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98d806ae40 0x7f98d806d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98ec13cc60 tx=0x7f98e0008040 comp rx=0 tx=0).stop 2023-11-18T20:43:01.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 -- 172.21.15.60:0/2246006900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec143490 msgr2=0x7f98ec13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec143490 0x7f98ec13b160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f98e400d800 tx=0x7f98e400dcc0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 -- 172.21.15.60:0/2246006900 shutdown_connections 2023-11-18T20:43:01.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98d806ae40 0x7f98d806d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec15e730 0x7f98ec0708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec15a4e0 0x7f98ec070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 --2- 172.21.15.60:0/2246006900 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec143490 0x7f98ec13b160 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 -- 172.21.15.60:0/2246006900 >> 172.21.15.60:0/2246006900 conn(0x7f98ec06a4d0 msgr2=0x7f98ec0cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 -- 172.21.15.60:0/2246006900 shutdown_connections 2023-11-18T20:43:01.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.414+0000 7f98f4701640 1 -- 172.21.15.60:0/2246006900 wait complete. 2023-11-18T20:43:01.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:01.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.531+0000 7fa7dd056640 1 Processor -- start 2023-11-18T20:43:01.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.532+0000 7fa7dd056640 1 -- start start 2023-11-18T20:43:01.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.532+0000 7fa7dd056640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7d815b130 0x7fa7d80d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.532+0000 7fa7dd056640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa7d815f760 0x7fa7d80cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.532+0000 7fa7dd056640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80cf150 0x7fa7d806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.532+0000 7fa7dd056640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d815fd20 con 0x7fa7d815b130 2023-11-18T20:43:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.532+0000 7fa7dd056640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d80d0730 con 0x7fa7d815f760 2023-11-18T20:43:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.532+0000 7fa7dd056640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80cf5a0 con 0x7fa7d80cf150 2023-11-18T20:43:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.533+0000 7fa7d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7d815b130 0x7fa7d80d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.533+0000 7fa7d6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa7d815b130 0x7fa7d80d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42430/0 (socket says 172.21.15.60:42430) 2023-11-18T20:43:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.533+0000 7fa7d6d9d640 1 -- 172.21.15.60:0/2828838719 learned_addr learned my addr 172.21.15.60:0/2828838719 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.533+0000 7fa7d759e640 1 --2- 172.21.15.60:0/2828838719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80cf150 0x7fa7d806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.533+0000 7fa7cffff640 1 --2- 172.21.15.60:0/2828838719 >> v2:172.21.15.100:3300/0 conn(0x7fa7d815f760 0x7fa7d80cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.533+0000 7fa7d6d9d640 1 -- 172.21.15.60:0/2828838719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80cf150 msgr2=0x7fa7d806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.533+0000 7fa7d6d9d640 1 --2- 172.21.15.60:0/2828838719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80cf150 0x7fa7d806ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.533+0000 7fa7d6d9d640 1 -- 172.21.15.60:0/2828838719 >> v2:172.21.15.100:3300/0 conn(0x7fa7d815f760 msgr2=0x7fa7d80cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.533+0000 7fa7d6d9d640 1 --2- 172.21.15.60:0/2828838719 >> v2:172.21.15.100:3300/0 conn(0x7fa7d815f760 0x7fa7d80cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.533+0000 7fa7d6d9d640 1 -- 172.21.15.60:0/2828838719 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d80b3f10 con 0x7fa7d815b130 2023-11-18T20:43:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.534+0000 7fa7d6d9d640 1 --2- 172.21.15.60:0/2828838719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815b130 0x7fa7d80d01f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80b71b0 tx=0x7fa7c802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffea4720dc389aa server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.534+0000 7fa7d659c640 1 -- 172.21.15.60:0/2828838719 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80317c0 con 0x7fa7d815b130 2023-11-18T20:43:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.534+0000 7fa7d659c640 1 -- 172.21.15.60:0/2828838719 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c8031920 con 0x7fa7d815b130 2023-11-18T20:43:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.534+0000 7fa7d659c640 1 -- 172.21.15.60:0/2828838719 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8031cb0 con 0x7fa7d815b130 2023-11-18T20:43:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.535+0000 7fa7dd056640 1 -- 172.21.15.60:0/2828838719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815b130 msgr2=0x7fa7d80d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.535+0000 7fa7dd056640 1 --2- 172.21.15.60:0/2828838719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815b130 0x7fa7d80d01f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80b71b0 tx=0x7fa7c802b860 comp rx=0 tx=0).stop 2023-11-18T20:43:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.535+0000 7fa7dd056640 1 -- 172.21.15.60:0/2828838719 shutdown_connections 2023-11-18T20:43:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.535+0000 7fa7dd056640 1 --2- 172.21.15.60:0/2828838719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80cf150 0x7fa7d806ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.535+0000 7fa7dd056640 1 --2- 172.21.15.60:0/2828838719 >> v2:172.21.15.100:3300/0 conn(0x7fa7d815f760 0x7fa7d80cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.535+0000 7fa7dd056640 1 --2- 172.21.15.60:0/2828838719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815b130 0x7fa7d80d01f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.535+0000 7fa7dd056640 1 -- 172.21.15.60:0/2828838719 >> 172.21.15.60:0/2828838719 conn(0x7fa7d80b2c10 msgr2=0x7fa7d80b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.535+0000 7fa7dd056640 1 -- 172.21.15.60:0/2828838719 shutdown_connections 2023-11-18T20:43:01.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.535+0000 7fa7dd056640 1 -- 172.21.15.60:0/2828838719 wait complete. 2023-11-18T20:43:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.536+0000 7fa7dd056640 1 Processor -- start 2023-11-18T20:43:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.536+0000 7fa7dd056640 1 -- start start 2023-11-18T20:43:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7dd056640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d80cf150 0x7fa7d81f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7dd056640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d815b130 0x7fa7d8155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7dd056640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815f760 0x7fa7d815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7dd056640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d81f15f0 con 0x7fa7d815f760 2023-11-18T20:43:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7dd056640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d81561d0 con 0x7fa7d80cf150 2023-11-18T20:43:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7dd056640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80c8320 con 0x7fa7d815b130 2023-11-18T20:43:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815f760 0x7fa7d815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815f760 0x7fa7d815cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42432/0 (socket says 172.21.15.60:42432) 2023-11-18T20:43:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d759e640 1 -- 172.21.15.60:0/2949998794 learned_addr learned my addr 172.21.15.60:0/2949998794 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7cffff640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d815b130 0x7fa7d8155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d6d9d640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d80cf150 0x7fa7d81f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d759e640 1 -- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d815b130 msgr2=0x7fa7d8155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d759e640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d815b130 0x7fa7d8155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d759e640 1 -- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d80cf150 msgr2=0x7fa7d81f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d759e640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d80cf150 0x7fa7d81f1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d759e640 1 -- 172.21.15.60:0/2949998794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c80094d0 con 0x7fa7d815f760 2023-11-18T20:43:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d6d9d640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d80cf150 0x7fa7d81f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.537+0000 7fa7d759e640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815f760 0x7fa7d815cc60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007e60 tx=0x7fa7d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.538+0000 7fa7cf7fe640 1 -- 172.21.15.60:0/2949998794 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0014620 con 0x7fa7d815f760 2023-11-18T20:43:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.538+0000 7fa7cf7fe640 1 -- 172.21.15.60:0/2949998794 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d000ee90 con 0x7fa7d815f760 2023-11-18T20:43:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.538+0000 7fa7cf7fe640 1 -- 172.21.15.60:0/2949998794 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00149e0 con 0x7fa7d815f760 2023-11-18T20:43:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.538+0000 7fa7dd056640 1 -- 172.21.15.60:0/2949998794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d815d5d0 con 0x7fa7d815f760 2023-11-18T20:43:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.538+0000 7fa7dd056640 1 -- 172.21.15.60:0/2949998794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d81f1210 con 0x7fa7d815f760 2023-11-18T20:43:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.539+0000 7fa7dd056640 1 -- 172.21.15.60:0/2949998794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d8067fc0 con 0x7fa7d815f760 2023-11-18T20:43:01.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.539+0000 7fa7cf7fe640 1 -- 172.21.15.60:0/2949998794 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7d0014b40 con 0x7fa7d815f760 2023-11-18T20:43:01.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.539+0000 7fa7cf7fe640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7b806ae40 0x7fa7b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.540+0000 7fa7cf7fe640 1 -- 172.21.15.60:0/2949998794 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7d0089f70 con 0x7fa7d815f760 2023-11-18T20:43:01.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.540+0000 7fa7d6d9d640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7b806ae40 0x7fa7b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.540+0000 7fa7d6d9d640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7b806ae40 0x7fa7b806d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8036000 tx=0x7fa7c803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:01.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.544+0000 7fa7cf7fe640 1 -- 172.21.15.60:0/2949998794 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7d00bce90 con 0x7fa7d815f760 2023-11-18T20:43:01.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.724+0000 7fa7dd056640 1 -- 172.21.15.60:0/2949998794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7d81f1730 con 0x7fa7d815f760 2023-11-18T20:43:01.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.725+0000 7fa7cf7fe640 1 -- 172.21.15.60:0/2949998794 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fa7d81f1730 con 0x7fa7d815f760 2023-11-18T20:43:01.727 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:01.727 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.726+0000 7fa7dd056640 1 -- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7b806ae40 msgr2=0x7fa7b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.726+0000 7fa7dd056640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7b806ae40 0x7fa7b806d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8036000 tx=0x7fa7c803d040 comp rx=0 tx=0).stop 2023-11-18T20:43:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.726+0000 7fa7dd056640 1 -- 172.21.15.60:0/2949998794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815f760 msgr2=0x7fa7d815cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.726+0000 7fa7dd056640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815f760 0x7fa7d815cc60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007e60 tx=0x7fa7d000e4b0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.727+0000 7fa7dd056640 1 -- 172.21.15.60:0/2949998794 shutdown_connections 2023-11-18T20:43:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.727+0000 7fa7dd056640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7b806ae40 0x7fa7b806d300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.727+0000 7fa7dd056640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d815f760 0x7fa7d815cc60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.727+0000 7fa7dd056640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d815b130 0x7fa7d8155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.727+0000 7fa7dd056640 1 --2- 172.21.15.60:0/2949998794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d80cf150 0x7fa7d81f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.727+0000 7fa7dd056640 1 -- 172.21.15.60:0/2949998794 >> 172.21.15.60:0/2949998794 conn(0x7fa7d80b2c10 msgr2=0x7fa7d815ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.727+0000 7fa7dd056640 1 -- 172.21.15.60:0/2949998794 shutdown_connections 2023-11-18T20:43:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.727+0000 7fa7dd056640 1 -- 172.21.15.60:0/2949998794 wait complete. 2023-11-18T20:43:01.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:01.740 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.847+0000 7f3a465ee640 1 Processor -- start 2023-11-18T20:43:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.847+0000 7f3a465ee640 1 -- start start 2023-11-18T20:43:01.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.848+0000 7f3a465ee640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a400cf320 0x7f3a400d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.848+0000 7f3a465ee640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a4015f760 0x7f3a400cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.848+0000 7f3a465ee640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a400b3f10 con 0x7f3a4015b130 2023-11-18T20:43:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.848+0000 7f3a465ee640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4015fd20 con 0x7f3a400cf320 2023-11-18T20:43:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.848+0000 7f3a465ee640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a400d0730 con 0x7f3a4015f760 2023-11-18T20:43:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.848+0000 7f3a3f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a400cf320 0x7f3a400d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.848+0000 7f3a3f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a400cf320 0x7f3a400d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42438/0 (socket says 172.21.15.60:42438) 2023-11-18T20:43:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.848+0000 7f3a3f7fe640 1 -- 172.21.15.60:0/2920732335 learned_addr learned my addr 172.21.15.60:0/2920732335 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.849+0000 7f3a44b8b640 1 --2- 172.21.15.60:0/2920732335 >> v2:172.21.15.100:3300/0 conn(0x7f3a4015f760 0x7f3a400cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.849+0000 7f3a3f7fe640 1 -- 172.21.15.60:0/2920732335 >> v2:172.21.15.100:3300/0 conn(0x7f3a4015f760 msgr2=0x7f3a400cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.849+0000 7f3a3f7fe640 1 --2- 172.21.15.60:0/2920732335 >> v2:172.21.15.100:3300/0 conn(0x7f3a4015f760 0x7f3a400cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.849+0000 7f3a3f7fe640 1 -- 172.21.15.60:0/2920732335 >> v1:172.21.15.100:6789/0 conn(0x7f3a4015b130 legacy=0x7f3a400cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.849+0000 7f3a3f7fe640 1 -- 172.21.15.60:0/2920732335 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a400d0a50 con 0x7f3a400cf320 2023-11-18T20:43:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.849+0000 7f3a3f7fe640 1 --2- 172.21.15.60:0/2920732335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400cf320 0x7f3a400d01f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3a4006eb70 tx=0x7f3a3002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3be1b86bd27396 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a3effd640 1 -- 172.21.15.60:0/2920732335 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a30030860 con 0x7f3a400cf320 2023-11-18T20:43:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a3effd640 1 -- 172.21.15.60:0/2920732335 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a300309c0 con 0x7f3a400cf320 2023-11-18T20:43:01.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a3effd640 1 -- 172.21.15.60:0/2920732335 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a30030d50 con 0x7f3a400cf320 2023-11-18T20:43:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a465ee640 1 -- 172.21.15.60:0/2920732335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400cf320 msgr2=0x7f3a400d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a465ee640 1 --2- 172.21.15.60:0/2920732335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400cf320 0x7f3a400d01f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3a4006eb70 tx=0x7f3a3002b9a0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a465ee640 1 -- 172.21.15.60:0/2920732335 shutdown_connections 2023-11-18T20:43:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a465ee640 1 --2- 172.21.15.60:0/2920732335 >> v2:172.21.15.100:3300/0 conn(0x7f3a4015f760 0x7f3a400cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a465ee640 1 --2- 172.21.15.60:0/2920732335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a400cf320 0x7f3a400d01f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a465ee640 1 -- 172.21.15.60:0/2920732335 >> 172.21.15.60:0/2920732335 conn(0x7f3a400b2c10 msgr2=0x7f3a400b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a465ee640 1 -- 172.21.15.60:0/2920732335 shutdown_connections 2023-11-18T20:43:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.850+0000 7f3a465ee640 1 -- 172.21.15.60:0/2920732335 wait complete. 2023-11-18T20:43:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.851+0000 7f3a465ee640 1 Processor -- start 2023-11-18T20:43:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a465ee640 1 -- start start 2023-11-18T20:43:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a465ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a400cf320 0x7f3a401f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a465ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a4015b130 0x7f3a401f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a465ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4015f760 0x7f3a401f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a465ee640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a401f5ec0 con 0x7f3a4015b130 2023-11-18T20:43:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a465ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a400ca770 con 0x7f3a4015f760 2023-11-18T20:43:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a465ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a400caa50 con 0x7f3a400cf320 2023-11-18T20:43:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a3f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a4015b130 0x7f3a401f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a3f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a4015b130 0x7f3a401f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42448/0 (socket says 172.21.15.60:42448) 2023-11-18T20:43:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a3f7fe640 1 -- 172.21.15.60:0/3648675415 learned_addr learned my addr 172.21.15.60:0/3648675415 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.852+0000 7f3a44b8b640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4015f760 0x7f3a401f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a3ffff640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a400cf320 0x7f3a401f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a3f7fe640 1 -- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a400cf320 msgr2=0x7f3a401f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a3f7fe640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a400cf320 0x7f3a401f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a3f7fe640 1 -- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4015f760 msgr2=0x7f3a401f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a3f7fe640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4015f760 0x7f3a401f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a3f7fe640 1 -- 172.21.15.60:0/3648675415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a300094d0 con 0x7f3a4015b130 2023-11-18T20:43:01.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a44b8b640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4015f760 0x7f3a401f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a3f7fe640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a4015b130 0x7f3a401f60d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a30000c00 tx=0x7f3a30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a3cff9640 1 -- 172.21.15.60:0/3648675415 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a30039c90 con 0x7f3a4015b130 2023-11-18T20:43:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a3cff9640 1 -- 172.21.15.60:0/3648675415 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a30039df0 con 0x7f3a4015b130 2023-11-18T20:43:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a465ee640 1 -- 172.21.15.60:0/3648675415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a400cad30 con 0x7f3a4015b130 2023-11-18T20:43:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a3cff9640 1 -- 172.21.15.60:0/3648675415 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a30043640 con 0x7f3a4015b130 2023-11-18T20:43:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.853+0000 7f3a465ee640 1 -- 172.21.15.60:0/3648675415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a400b4130 con 0x7f3a4015b130 2023-11-18T20:43:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.854+0000 7f3a465ee640 1 -- 172.21.15.60:0/3648675415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a40067fc0 con 0x7f3a4015b130 2023-11-18T20:43:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.855+0000 7f3a3cff9640 1 -- 172.21.15.60:0/3648675415 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a300437a0 con 0x7f3a4015b130 2023-11-18T20:43:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.855+0000 7f3a3cff9640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3a2c06ae40 0x7f3a2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.855+0000 7f3a3cff9640 1 -- 172.21.15.60:0/3648675415 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a3003e070 con 0x7f3a4015b130 2023-11-18T20:43:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.855+0000 7f3a3ffff640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3a2c06ae40 0x7f3a2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.856+0000 7f3a3ffff640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3a2c06ae40 0x7f3a2c06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a34000f60 tx=0x7f3a34009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:01.859+0000 7f3a3cff9640 1 -- 172.21.15.60:0/3648675415 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a30077790 con 0x7f3a4015b130 2023-11-18T20:43:02.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.036+0000 7f3a465ee640 1 -- 172.21.15.60:0/3648675415 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a400b4320 con 0x7f3a4015b130 2023-11-18T20:43:02.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.037+0000 7f3a3cff9640 1 -- 172.21.15.60:0/3648675415 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a3007adb0 con 0x7f3a4015b130 2023-11-18T20:43:02.039 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:02.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 -- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3a2c06ae40 msgr2=0x7f3a2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3a2c06ae40 0x7f3a2c06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a34000f60 tx=0x7f3a34009040 comp rx=0 tx=0).stop 2023-11-18T20:43:02.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 -- 172.21.15.60:0/3648675415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a4015b130 msgr2=0x7f3a401f60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a4015b130 0x7f3a401f60d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a30000c00 tx=0x7f3a30004730 comp rx=0 tx=0).stop 2023-11-18T20:43:02.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 -- 172.21.15.60:0/3648675415 shutdown_connections 2023-11-18T20:43:02.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3a2c06ae40 0x7f3a2c06d300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4015f760 0x7f3a401f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a4015b130 0x7f3a401f60d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 --2- 172.21.15.60:0/3648675415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a400cf320 0x7f3a401f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 -- 172.21.15.60:0/3648675415 >> 172.21.15.60:0/3648675415 conn(0x7f3a400b2c10 msgr2=0x7f3a400cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 -- 172.21.15.60:0/3648675415 shutdown_connections 2023-11-18T20:43:02.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.039+0000 7f3a465ee640 1 -- 172.21.15.60:0/3648675415 wait complete. 2023-11-18T20:43:02.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:02.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.160+0000 7fc1219c0640 1 Processor -- start 2023-11-18T20:43:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.160+0000 7fc1219c0640 1 -- start start 2023-11-18T20:43:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.161+0000 7fc1219c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c06a4d0 0x7fc11c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.161+0000 7fc1219c0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc11c072130 0x7fc11c15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.161+0000 7fc1219c0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c15f3a0 con 0x7fc11c1448f0 2023-11-18T20:43:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.161+0000 7fc1219c0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c0714c0 con 0x7fc11c072130 2023-11-18T20:43:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.161+0000 7fc1219c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c15f680 con 0x7fc11c06a4d0 2023-11-18T20:43:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.161+0000 7fc11affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc11c1448f0 0x7fc11c15ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50036/0 (socket says 172.21.15.60:50036) 2023-11-18T20:43:02.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.161+0000 7fc11affd640 1 -- 172.21.15.60:0/3888279224 learned_addr learned my addr 172.21.15.60:0/3888279224 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.161+0000 7fc11b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc11c072130 0x7fc11c15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.162+0000 7fc11a7fc640 1 --2- 172.21.15.60:0/3888279224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c06a4d0 0x7fc11c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.162+0000 7fc119ffb640 1 -- 172.21.15.60:0/3888279224 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157721249 0 0) 0x7fc11c15f3a0 con 0x7fc11c1448f0 2023-11-18T20:43:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.162+0000 7fc119ffb640 1 -- 172.21.15.60:0/3888279224 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc108003520 con 0x7fc11c1448f0 2023-11-18T20:43:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.162+0000 7fc11b7fe640 1 -- 172.21.15.60:0/3888279224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c06a4d0 msgr2=0x7fc11c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.162+0000 7fc11b7fe640 1 --2- 172.21.15.60:0/3888279224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c06a4d0 0x7fc11c071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.162+0000 7fc11b7fe640 1 -- 172.21.15.60:0/3888279224 >> v1:172.21.15.60:6789/0 conn(0x7fc11c1448f0 legacy=0x7fc11c15ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.162+0000 7fc11b7fe640 1 -- 172.21.15.60:0/3888279224 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c15f960 con 0x7fc11c072130 2023-11-18T20:43:02.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc11b7fe640 1 --2- 172.21.15.60:0/3888279224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c072130 0x7fc11c15b5d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009d90 tx=0x7fc10c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445a2e53e7c76b58 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc119ffb640 1 -- 172.21.15.60:0/3888279224 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c02cbc0 con 0x7fc11c072130 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc119ffb640 1 -- 172.21.15.60:0/3888279224 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c02cd20 con 0x7fc11c072130 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc119ffb640 1 -- 172.21.15.60:0/3888279224 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c034660 con 0x7fc11c072130 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc1219c0640 1 -- 172.21.15.60:0/3888279224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c072130 msgr2=0x7fc11c15b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc1219c0640 1 --2- 172.21.15.60:0/3888279224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c072130 0x7fc11c15b5d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009d90 tx=0x7fc10c02be10 comp rx=0 tx=0).stop 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc1219c0640 1 -- 172.21.15.60:0/3888279224 shutdown_connections 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc1219c0640 1 --2- 172.21.15.60:0/3888279224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c072130 0x7fc11c15b5d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc1219c0640 1 --2- 172.21.15.60:0/3888279224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c06a4d0 0x7fc11c071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc1219c0640 1 -- 172.21.15.60:0/3888279224 >> 172.21.15.60:0/3888279224 conn(0x7fc11c0b1de0 msgr2=0x7fc11c0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc1219c0640 1 -- 172.21.15.60:0/3888279224 shutdown_connections 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.163+0000 7fc1219c0640 1 -- 172.21.15.60:0/3888279224 wait complete. 2023-11-18T20:43:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.164+0000 7fc1219c0640 1 Processor -- start 2023-11-18T20:43:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.165+0000 7fc1219c0640 1 -- start start 2023-11-18T20:43:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.165+0000 7fc1219c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c06a4d0 0x7fc11c15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.165+0000 7fc1219c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c072130 0x7fc11c15e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.165+0000 7fc1219c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c1448f0 0x7fc11c15d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.165+0000 7fc1219c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c141910 con 0x7fc11c1448f0 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.165+0000 7fc1219c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c141bf0 con 0x7fc11c06a4d0 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.165+0000 7fc1219c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c15ca20 con 0x7fc11c072130 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c1448f0 0x7fc11c15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c1448f0 0x7fc11c15d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42460/0 (socket says 172.21.15.60:42460) 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11b7fe640 1 -- 172.21.15.60:0/1075232052 learned_addr learned my addr 172.21.15.60:0/1075232052 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11a7fc640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c072130 0x7fc11c15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11affd640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c06a4d0 0x7fc11c15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11b7fe640 1 -- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c072130 msgr2=0x7fc11c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11b7fe640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c072130 0x7fc11c15e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11b7fe640 1 -- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c06a4d0 msgr2=0x7fc11c15a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11b7fe640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c06a4d0 0x7fc11c15a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11b7fe640 1 -- 172.21.15.60:0/1075232052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c009a70 con 0x7fc11c1448f0 2023-11-18T20:43:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11a7fc640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c072130 0x7fc11c15e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11affd640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c06a4d0 0x7fc11c15a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc11b7fe640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c1448f0 0x7fc11c15d220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009d90 tx=0x7fc10c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc113fff640 1 -- 172.21.15.60:0/1075232052 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c040650 con 0x7fc11c1448f0 2023-11-18T20:43:02.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc113fff640 1 -- 172.21.15.60:0/1075232052 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c02ce90 con 0x7fc11c1448f0 2023-11-18T20:43:02.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc1219c0640 1 -- 172.21.15.60:0/1075232052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c15cd00 con 0x7fc11c1448f0 2023-11-18T20:43:02.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc113fff640 1 -- 172.21.15.60:0/1075232052 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0409f0 con 0x7fc11c1448f0 2023-11-18T20:43:02.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.166+0000 7fc1219c0640 1 -- 172.21.15.60:0/1075232052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c15d760 con 0x7fc11c1448f0 2023-11-18T20:43:02.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.167+0000 7fc1219c0640 1 -- 172.21.15.60:0/1075232052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c067420 con 0x7fc11c1448f0 2023-11-18T20:43:02.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.168+0000 7fc113fff640 1 -- 172.21.15.60:0/1075232052 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc10c040b50 con 0x7fc11c1448f0 2023-11-18T20:43:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.168+0000 7fc113fff640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0fc06ae40 0x7fc0fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.168+0000 7fc113fff640 1 -- 172.21.15.60:0/1075232052 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc10c0ac220 con 0x7fc11c1448f0 2023-11-18T20:43:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.168+0000 7fc11affd640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0fc06ae40 0x7fc0fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.169+0000 7fc11affd640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0fc06ae40 0x7fc0fc06d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc114007fd0 tx=0x7fc114009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.173+0000 7fc113fff640 1 -- 172.21.15.60:0/1075232052 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc10c0772d0 con 0x7fc11c1448f0 2023-11-18T20:43:02.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.347+0000 7fc1219c0640 1 -- 172.21.15.60:0/1075232052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc11c06fe80 con 0x7fc11c1448f0 2023-11-18T20:43:02.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.348+0000 7fc113fff640 1 -- 172.21.15.60:0/1075232052 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc10c009450 con 0x7fc11c1448f0 2023-11-18T20:43:02.350 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 -- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0fc06ae40 msgr2=0x7fc0fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0fc06ae40 0x7fc0fc06d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc114007fd0 tx=0x7fc114009040 comp rx=0 tx=0).stop 2023-11-18T20:43:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 -- 172.21.15.60:0/1075232052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c1448f0 msgr2=0x7fc11c15d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c1448f0 0x7fc11c15d220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009d90 tx=0x7fc10c004a40 comp rx=0 tx=0).stop 2023-11-18T20:43:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 -- 172.21.15.60:0/1075232052 shutdown_connections 2023-11-18T20:43:02.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0fc06ae40 0x7fc0fc06d300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c1448f0 0x7fc11c15d220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c072130 0x7fc11c15e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 --2- 172.21.15.60:0/1075232052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c06a4d0 0x7fc11c15a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 -- 172.21.15.60:0/1075232052 >> 172.21.15.60:0/1075232052 conn(0x7fc11c0b1de0 msgr2=0x7fc11c0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 -- 172.21.15.60:0/1075232052 shutdown_connections 2023-11-18T20:43:02.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.350+0000 7fc1219c0640 1 -- 172.21.15.60:0/1075232052 wait complete. 2023-11-18T20:43:02.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.470+0000 7f69c63e7640 1 Processor -- start 2023-11-18T20:43:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.470+0000 7f69c63e7640 1 -- start start 2023-11-18T20:43:02.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69c63e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69c006ea50 0x7f69c006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69c63e7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69c0160fb0 0x7f69c0161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69c63e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69c00b3ff0 0x7f69c00b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69c63e7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c0161d20 con 0x7f69c006ea50 2023-11-18T20:43:02.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69c63e7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c006f370 con 0x7f69c0160fb0 2023-11-18T20:43:02.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69c63e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c015e760 con 0x7f69c00b3ff0 2023-11-18T20:43:02.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69bffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69c006ea50 0x7f69c006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69bffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69c006ea50 0x7f69c006ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42466/0 (socket says 172.21.15.60:42466) 2023-11-18T20:43:02.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69bffff640 1 -- 172.21.15.60:0/415219740 learned_addr learned my addr 172.21.15.60:0/415219740 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:02.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69c4984640 1 --2- 172.21.15.60:0/415219740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69c00b3ff0 0x7f69c00b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69bf7fe640 1 --2- 172.21.15.60:0/415219740 >> v2:172.21.15.100:3300/0 conn(0x7f69c0160fb0 0x7f69c0161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69bffff640 1 -- 172.21.15.60:0/415219740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69c00b3ff0 msgr2=0x7f69c00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:02.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69bffff640 1 --2- 172.21.15.60:0/415219740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69c00b3ff0 0x7f69c00b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69bffff640 1 -- 172.21.15.60:0/415219740 >> v2:172.21.15.100:3300/0 conn(0x7f69c0160fb0 msgr2=0x7f69c0161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:02.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.471+0000 7f69bffff640 1 --2- 172.21.15.60:0/415219740 >> v2:172.21.15.100:3300/0 conn(0x7f69c0160fb0 0x7f69c0161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.472+0000 7f69bffff640 1 -- 172.21.15.60:0/415219740 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c015d9a0 con 0x7f69c006ea50 2023-11-18T20:43:02.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.472+0000 7f69bffff640 1 --2- 172.21.15.60:0/415219740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 0x7f69c006ee30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69c00b5b30 tx=0x7f69b402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0c871838c20e10 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:02.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.472+0000 7f69beffd640 1 -- 172.21.15.60:0/415219740 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b40317e0 con 0x7f69c006ea50 2023-11-18T20:43:02.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.472+0000 7f69beffd640 1 -- 172.21.15.60:0/415219740 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b4031940 con 0x7f69c006ea50 2023-11-18T20:43:02.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.472+0000 7f69beffd640 1 -- 172.21.15.60:0/415219740 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4031cd0 con 0x7f69c006ea50 2023-11-18T20:43:02.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.473+0000 7f69c63e7640 1 -- 172.21.15.60:0/415219740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 msgr2=0x7f69c006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.473+0000 7f69c63e7640 1 --2- 172.21.15.60:0/415219740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 0x7f69c006ee30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69c00b5b30 tx=0x7f69b402b880 comp rx=0 tx=0).stop 2023-11-18T20:43:02.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.473+0000 7f69c63e7640 1 -- 172.21.15.60:0/415219740 shutdown_connections 2023-11-18T20:43:02.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.473+0000 7f69c63e7640 1 --2- 172.21.15.60:0/415219740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69c00b3ff0 0x7f69c00b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.473+0000 7f69c63e7640 1 --2- 172.21.15.60:0/415219740 >> v2:172.21.15.100:3300/0 conn(0x7f69c0160fb0 0x7f69c0161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.473+0000 7f69c63e7640 1 --2- 172.21.15.60:0/415219740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 0x7f69c006ee30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.473+0000 7f69c63e7640 1 -- 172.21.15.60:0/415219740 >> 172.21.15.60:0/415219740 conn(0x7f69c006a6b0 msgr2=0x7f69c006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:02.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.473+0000 7f69c63e7640 1 -- 172.21.15.60:0/415219740 shutdown_connections 2023-11-18T20:43:02.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.473+0000 7f69c63e7640 1 -- 172.21.15.60:0/415219740 wait complete. 2023-11-18T20:43:02.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.474+0000 7f69c63e7640 1 Processor -- start 2023-11-18T20:43:02.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.474+0000 7f69c63e7640 1 -- start start 2023-11-18T20:43:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69c63e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 0x7f69c0150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69c63e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c00b3ff0 0x7f69c01516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69c63e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69c0160fb0 0x7f69c0151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69c63e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c01552f0 con 0x7f69c006ea50 2023-11-18T20:43:02.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69c63e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c01514f0 con 0x7f69c00b3ff0 2023-11-18T20:43:02.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69c63e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c0154600 con 0x7f69c0160fb0 2023-11-18T20:43:02.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69bffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 0x7f69c0150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69bffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 0x7f69c0150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42474/0 (socket says 172.21.15.60:42474) 2023-11-18T20:43:02.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69bffff640 1 -- 172.21.15.60:0/1436307093 learned_addr learned my addr 172.21.15.60:0/1436307093 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69bf7fe640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c00b3ff0 0x7f69c01516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69c4984640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69c0160fb0 0x7f69c0151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69bffff640 1 -- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69c0160fb0 msgr2=0x7f69c0151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69bffff640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69c0160fb0 0x7f69c0151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69bffff640 1 -- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c00b3ff0 msgr2=0x7f69c01516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69bffff640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c00b3ff0 0x7f69c01516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.475+0000 7f69bffff640 1 -- 172.21.15.60:0/1436307093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40094d0 con 0x7f69c006ea50 2023-11-18T20:43:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.476+0000 7f69bffff640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 0x7f69c0150fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f69b4000c00 tx=0x7f69b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:02.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.476+0000 7f69bcff9640 1 -- 172.21.15.60:0/1436307093 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4032880 con 0x7f69c006ea50 2023-11-18T20:43:02.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.476+0000 7f69bcff9640 1 -- 172.21.15.60:0/1436307093 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b40329e0 con 0x7f69c006ea50 2023-11-18T20:43:02.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.476+0000 7f69c63e7640 1 -- 172.21.15.60:0/1436307093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c01548e0 con 0x7f69c006ea50 2023-11-18T20:43:02.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.476+0000 7f69bcff9640 1 -- 172.21.15.60:0/1436307093 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4032d90 con 0x7f69c006ea50 2023-11-18T20:43:02.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.476+0000 7f69c63e7640 1 -- 172.21.15.60:0/1436307093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c0154f10 con 0x7f69c006ea50 2023-11-18T20:43:02.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.477+0000 7f69c63e7640 1 -- 172.21.15.60:0/1436307093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c0155430 con 0x7f69c006ea50 2023-11-18T20:43:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.477+0000 7f69bcff9640 1 -- 172.21.15.60:0/1436307093 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69b404e450 con 0x7f69c006ea50 2023-11-18T20:43:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.478+0000 7f69bcff9640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69ac06ae40 0x7f69ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.478+0000 7f69bcff9640 1 -- 172.21.15.60:0/1436307093 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69b40ab470 con 0x7f69c006ea50 2023-11-18T20:43:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.478+0000 7f69bf7fe640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69ac06ae40 0x7f69ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.478+0000 7f69bf7fe640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69ac06ae40 0x7f69ac06d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69c01f62a0 tx=0x7f69b0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.482+0000 7f69bcff9640 1 -- 172.21.15.60:0/1436307093 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69c0155430 con 0x7f69c006ea50 2023-11-18T20:43:02.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.658+0000 7f69c63e7640 1 -- 172.21.15.60:0/1436307093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69c0155430 con 0x7f69c006ea50 2023-11-18T20:43:02.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.659+0000 7f69bcff9640 1 -- 172.21.15.60:0/1436307093 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f69c0155430 con 0x7f69c006ea50 2023-11-18T20:43:02.661 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.660+0000 7f69c63e7640 1 -- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69ac06ae40 msgr2=0x7f69ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.660+0000 7f69c63e7640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69ac06ae40 0x7f69ac06d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69c01f62a0 tx=0x7f69b0009190 comp rx=0 tx=0).stop 2023-11-18T20:43:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.660+0000 7f69c63e7640 1 -- 172.21.15.60:0/1436307093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 msgr2=0x7f69c0150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.660+0000 7f69c63e7640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 0x7f69c0150fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f69b4000c00 tx=0x7f69b40046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.660+0000 7f69c63e7640 1 -- 172.21.15.60:0/1436307093 shutdown_connections 2023-11-18T20:43:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.660+0000 7f69c63e7640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69ac06ae40 0x7f69ac06d300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.661+0000 7f69c63e7640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69c0160fb0 0x7f69c0151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.661+0000 7f69c63e7640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69c00b3ff0 0x7f69c01516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.661+0000 7f69c63e7640 1 --2- 172.21.15.60:0/1436307093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69c006ea50 0x7f69c0150fb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.661+0000 7f69c63e7640 1 -- 172.21.15.60:0/1436307093 >> 172.21.15.60:0/1436307093 conn(0x7f69c006a6b0 msgr2=0x7f69c006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:02.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.661+0000 7f69c63e7640 1 -- 172.21.15.60:0/1436307093 shutdown_connections 2023-11-18T20:43:02.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.661+0000 7f69c63e7640 1 -- 172.21.15.60:0/1436307093 wait complete. 2023-11-18T20:43:02.673 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:02.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.778+0000 7f919250f640 1 Processor -- start 2023-11-18T20:43:02.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.778+0000 7f919250f640 1 -- start start 2023-11-18T20:43:02.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f919250f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c13d080 0x7f918c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f919250f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f918c06b4e0 0x7f918c06b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f919250f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f918c0b3d20 0x7f918c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f919250f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918c13d460 con 0x7f918c0b3d20 2023-11-18T20:43:02.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f919250f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918c06be80 con 0x7f918c06b4e0 2023-11-18T20:43:02.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f919250f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918c0ccc40 con 0x7f918c13d080 2023-11-18T20:43:02.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f9190aac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f918c0b3d20 0x7f918c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f9190aac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f918c0b3d20 0x7f918c0b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42476/0 (socket says 172.21.15.60:42476) 2023-11-18T20:43:02.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f9190aac640 1 -- 172.21.15.60:0/2109939155 learned_addr learned my addr 172.21.15.60:0/2109939155 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:02.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f918bfff640 1 --2- 172.21.15.60:0/2109939155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c13d080 0x7f918c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f918b7fe640 1 --2- 172.21.15.60:0/2109939155 >> v2:172.21.15.100:3300/0 conn(0x7f918c06b4e0 0x7f918c06b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f9190aac640 1 -- 172.21.15.60:0/2109939155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c13d080 msgr2=0x7f918c15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:02.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f9190aac640 1 --2- 172.21.15.60:0/2109939155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c13d080 0x7f918c15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f9190aac640 1 -- 172.21.15.60:0/2109939155 >> v2:172.21.15.100:3300/0 conn(0x7f918c06b4e0 msgr2=0x7f918c06b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:02.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f9190aac640 1 --2- 172.21.15.60:0/2109939155 >> v2:172.21.15.100:3300/0 conn(0x7f918c06b4e0 0x7f918c06b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.779+0000 7f9190aac640 1 -- 172.21.15.60:0/2109939155 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c15b530 con 0x7f918c0b3d20 2023-11-18T20:43:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.780+0000 7f9190aac640 1 --2- 172.21.15.60:0/2109939155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 0x7f918c0b4170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f918c0b2860 tx=0x7f918402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fafab8cad9236eb server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.780+0000 7f918affd640 1 -- 172.21.15.60:0/2109939155 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918402ca70 con 0x7f918c0b3d20 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.780+0000 7f918affd640 1 -- 172.21.15.60:0/2109939155 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918402cbd0 con 0x7f918c0b3d20 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.780+0000 7f918affd640 1 -- 172.21.15.60:0/2109939155 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91840343f0 con 0x7f918c0b3d20 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.781+0000 7f919250f640 1 -- 172.21.15.60:0/2109939155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 msgr2=0x7f918c0b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.781+0000 7f919250f640 1 --2- 172.21.15.60:0/2109939155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 0x7f918c0b4170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f918c0b2860 tx=0x7f918402bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.781+0000 7f919250f640 1 -- 172.21.15.60:0/2109939155 shutdown_connections 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.781+0000 7f919250f640 1 --2- 172.21.15.60:0/2109939155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 0x7f918c0b4170 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.781+0000 7f919250f640 1 --2- 172.21.15.60:0/2109939155 >> v2:172.21.15.100:3300/0 conn(0x7f918c06b4e0 0x7f918c06b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.781+0000 7f919250f640 1 --2- 172.21.15.60:0/2109939155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c13d080 0x7f918c15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.781+0000 7f919250f640 1 -- 172.21.15.60:0/2109939155 >> 172.21.15.60:0/2109939155 conn(0x7f918c070e70 msgr2=0x7f918c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.781+0000 7f919250f640 1 -- 172.21.15.60:0/2109939155 shutdown_connections 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.781+0000 7f919250f640 1 -- 172.21.15.60:0/2109939155 wait complete. 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.782+0000 7f919250f640 1 Processor -- start 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.782+0000 7f919250f640 1 -- start start 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f919250f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c06b4e0 0x7f918c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f919250f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 0x7f918c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f919250f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c13d080 0x7f918c155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f919250f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1558d0 con 0x7f918c0b3d20 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f919250f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918c141df0 con 0x7f918c13d080 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f919250f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918c1411e0 con 0x7f918c06b4e0 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 0x7f918c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 0x7f918c154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42480/0 (socket says 172.21.15.60:42480) 2023-11-18T20:43:02.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918b7fe640 1 -- 172.21.15.60:0/1378165475 learned_addr learned my addr 172.21.15.60:0/1378165475 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f9190aac640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c13d080 0x7f918c155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918bfff640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c06b4e0 0x7f918c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918b7fe640 1 -- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c06b4e0 msgr2=0x7f918c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918b7fe640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c06b4e0 0x7f918c0cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918b7fe640 1 -- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c13d080 msgr2=0x7f918c155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918b7fe640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c13d080 0x7f918c155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918b7fe640 1 -- 172.21.15.60:0/1378165475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9184009b30 con 0x7f918c0b3d20 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f9190aac640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c13d080 0x7f918c155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918bfff640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c06b4e0 0x7f918c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.783+0000 7f918b7fe640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 0x7f918c154e50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f917c00e890 tx=0x7f917c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.784+0000 7f9188ff9640 1 -- 172.21.15.60:0/1378165475 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c00c940 con 0x7f918c0b3d20 2023-11-18T20:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.784+0000 7f9188ff9640 1 -- 172.21.15.60:0/1378165475 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c00caa0 con 0x7f918c0b3d20 2023-11-18T20:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.784+0000 7f919250f640 1 -- 172.21.15.60:0/1378165475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918c1414c0 con 0x7f918c0b3d20 2023-11-18T20:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.784+0000 7f9188ff9640 1 -- 172.21.15.60:0/1378165475 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c00ce30 con 0x7f918c0b3d20 2023-11-18T20:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.784+0000 7f919250f640 1 -- 172.21.15.60:0/1378165475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918c141a80 con 0x7f918c0b3d20 2023-11-18T20:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.785+0000 7f919250f640 1 -- 172.21.15.60:0/1378165475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918c0ccdb0 con 0x7f918c0b3d20 2023-11-18T20:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.785+0000 7f9188ff9640 1 -- 172.21.15.60:0/1378165475 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f917c013470 con 0x7f918c0b3d20 2023-11-18T20:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.786+0000 7f9188ff9640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f917806ae40 0x7f917806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.786+0000 7f9188ff9640 1 -- 172.21.15.60:0/1378165475 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f917c0897e0 con 0x7f918c0b3d20 2023-11-18T20:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.786+0000 7f918bfff640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f917806ae40 0x7f917806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.787+0000 7f918bfff640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f917806ae40 0x7f917806d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9180009a20 tx=0x7f9180008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.790+0000 7f9188ff9640 1 -- 172.21.15.60:0/1378165475 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f917c054880 con 0x7f918c0b3d20 2023-11-18T20:43:02.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.966+0000 7f919250f640 1 -- 172.21.15.60:0/1378165475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f918c141f60 con 0x7f918c0b3d20 2023-11-18T20:43:02.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.967+0000 7f9188ff9640 1 -- 172.21.15.60:0/1378165475 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f917c057ea0 con 0x7f918c0b3d20 2023-11-18T20:43:02.969 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:02.969 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:02.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 -- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f917806ae40 msgr2=0x7f917806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f917806ae40 0x7f917806d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9180009a20 tx=0x7f9180008040 comp rx=0 tx=0).stop 2023-11-18T20:43:02.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 -- 172.21.15.60:0/1378165475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 msgr2=0x7f918c154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:02.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 0x7f918c154e50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f917c00e890 tx=0x7f917c00ed50 comp rx=0 tx=0).stop 2023-11-18T20:43:02.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 -- 172.21.15.60:0/1378165475 shutdown_connections 2023-11-18T20:43:02.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f917806ae40 0x7f917806d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f918c13d080 0x7f918c155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f918c0b3d20 0x7f918c154e50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 --2- 172.21.15.60:0/1378165475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f918c06b4e0 0x7f918c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:02.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 -- 172.21.15.60:0/1378165475 >> 172.21.15.60:0/1378165475 conn(0x7f918c070e70 msgr2=0x7f918c15a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:02.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 -- 172.21.15.60:0/1378165475 shutdown_connections 2023-11-18T20:43:02.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:02.969+0000 7f919250f640 1 -- 172.21.15.60:0/1378165475 wait complete. 2023-11-18T20:43:02.982 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:02.982 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:03.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.091+0000 7f41267f5640 1 Processor -- start 2023-11-18T20:43:03.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.091+0000 7f41267f5640 1 -- start start 2023-11-18T20:43:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.091+0000 7f41267f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f412006abe0 0x7f41201618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.091+0000 7f41267f5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f41201409d0 0x7f412006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f41267f5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41201406f0 con 0x7f4120140310 2023-11-18T20:43:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f41267f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4120161e20 con 0x7f412006abe0 2023-11-18T20:43:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f41267f5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4120143960 con 0x7f41201409d0 2023-11-18T20:43:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f411f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f412006abe0 0x7f41201618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f411ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4120140310 0x7f412006a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50050/0 (socket says 172.21.15.60:50050) 2023-11-18T20:43:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f411f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f412006abe0 0x7f41201618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42492/0 (socket says 172.21.15.60:42492) 2023-11-18T20:43:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f411ffff640 1 -- 172.21.15.60:0/349908373 learned_addr learned my addr 172.21.15.60:0/349908373 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f4124d92640 1 --2- 172.21.15.60:0/349908373 >> v2:172.21.15.100:3300/0 conn(0x7f41201409d0 0x7f412006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f411f7fe640 1 -- 172.21.15.60:0/349908373 >> v2:172.21.15.100:3300/0 conn(0x7f41201409d0 msgr2=0x7f412006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f411f7fe640 1 --2- 172.21.15.60:0/349908373 >> v2:172.21.15.100:3300/0 conn(0x7f41201409d0 0x7f412006efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f411f7fe640 1 -- 172.21.15.60:0/349908373 >> v1:172.21.15.60:6789/0 conn(0x7f4120140310 legacy=0x7f412006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f411f7fe640 1 -- 172.21.15.60:0/349908373 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120142de0 con 0x7f412006abe0 2023-11-18T20:43:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.092+0000 7f411effd640 1 -- 172.21.15.60:0/349908373 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257321488 0 0) 0x7f41201406f0 con 0x7f4120140310 2023-11-18T20:43:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.093+0000 7f411f7fe640 1 --2- 172.21.15.60:0/349908373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f412006abe0 0x7f41201618e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4120154960 tx=0x7f411402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e755302b76e970 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.093+0000 7f411effd640 1 -- 172.21.15.60:0/349908373 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411402e070 con 0x7f412006abe0 2023-11-18T20:43:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.093+0000 7f411effd640 1 -- 172.21.15.60:0/349908373 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41140315a0 con 0x7f412006abe0 2023-11-18T20:43:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.093+0000 7f411effd640 1 -- 172.21.15.60:0/349908373 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114031930 con 0x7f412006abe0 2023-11-18T20:43:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.093+0000 7f41267f5640 1 -- 172.21.15.60:0/349908373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f412006abe0 msgr2=0x7f41201618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.093+0000 7f41267f5640 1 --2- 172.21.15.60:0/349908373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f412006abe0 0x7f41201618e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4120154960 tx=0x7f411402b840 comp rx=0 tx=0).stop 2023-11-18T20:43:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.093+0000 7f41267f5640 1 -- 172.21.15.60:0/349908373 shutdown_connections 2023-11-18T20:43:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.093+0000 7f41267f5640 1 --2- 172.21.15.60:0/349908373 >> v2:172.21.15.100:3300/0 conn(0x7f41201409d0 0x7f412006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.093+0000 7f41267f5640 1 --2- 172.21.15.60:0/349908373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f412006abe0 0x7f41201618e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.093+0000 7f41267f5640 1 -- 172.21.15.60:0/349908373 >> 172.21.15.60:0/349908373 conn(0x7f41200b1e50 msgr2=0x7f41200b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.094+0000 7f41267f5640 1 -- 172.21.15.60:0/349908373 shutdown_connections 2023-11-18T20:43:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.094+0000 7f41267f5640 1 -- 172.21.15.60:0/349908373 wait complete. 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.095+0000 7f41267f5640 1 Processor -- start 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.095+0000 7f41267f5640 1 -- start start 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f41267f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412006abe0 0x7f4120145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f41267f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4120140310 0x7f412013dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f41267f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41201409d0 0x7f41201f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f41267f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4120145b60 con 0x7f4120140310 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f41267f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412013f400 con 0x7f412006abe0 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f41267f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412013ee80 con 0x7f41201409d0 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4120140310 0x7f412013dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4120140310 0x7f412013dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42496/0 (socket says 172.21.15.60:42496) 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411f7fe640 1 -- 172.21.15.60:0/255458140 learned_addr learned my addr 172.21.15.60:0/255458140 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f4124d92640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41201409d0 0x7f41201f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411ffff640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412006abe0 0x7f4120145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411f7fe640 1 -- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41201409d0 msgr2=0x7f41201f2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411f7fe640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41201409d0 0x7f41201f2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411f7fe640 1 -- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412006abe0 msgr2=0x7f4120145620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411f7fe640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412006abe0 0x7f4120145620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411f7fe640 1 -- 172.21.15.60:0/255458140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114009400 con 0x7f4120140310 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411f7fe640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4120140310 0x7f412013dd10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f411402be50 tx=0x7f41140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411cff9640 1 -- 172.21.15.60:0/255458140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114032840 con 0x7f4120140310 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.096+0000 7f411cff9640 1 -- 172.21.15.60:0/255458140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41140329a0 con 0x7f4120140310 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.097+0000 7f41267f5640 1 -- 172.21.15.60:0/255458140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412013e250 con 0x7f4120140310 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.097+0000 7f411cff9640 1 -- 172.21.15.60:0/255458140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114032d50 con 0x7f4120140310 2023-11-18T20:43:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.097+0000 7f41267f5640 1 -- 172.21.15.60:0/255458140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41200b3050 con 0x7f4120140310 2023-11-18T20:43:03.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.098+0000 7f41267f5640 1 -- 172.21.15.60:0/255458140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4120069400 con 0x7f4120140310 2023-11-18T20:43:03.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.098+0000 7f411cff9640 1 -- 172.21.15.60:0/255458140 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f41140453f0 con 0x7f4120140310 2023-11-18T20:43:03.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.098+0000 7f411cff9640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f410c06ae70 0x7f410c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.098+0000 7f411cff9640 1 -- 172.21.15.60:0/255458140 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41140ab6d0 con 0x7f4120140310 2023-11-18T20:43:03.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.099+0000 7f411ffff640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f410c06ae70 0x7f410c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.099+0000 7f411ffff640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f410c06ae70 0x7f410c06d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4110000f60 tx=0x7f4110007b50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:03.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.103+0000 7f411cff9640 1 -- 172.21.15.60:0/255458140 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41140768a0 con 0x7f4120140310 2023-11-18T20:43:03.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.276+0000 7f41267f5640 1 -- 172.21.15.60:0/255458140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f412006ea60 con 0x7f4120140310 2023-11-18T20:43:03.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.277+0000 7f411cff9640 1 -- 172.21.15.60:0/255458140 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4114079ec0 con 0x7f4120140310 2023-11-18T20:43:03.279 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.278+0000 7f41267f5640 1 -- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f410c06ae70 msgr2=0x7f410c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.278+0000 7f41267f5640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f410c06ae70 0x7f410c06d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4110000f60 tx=0x7f4110007b50 comp rx=0 tx=0).stop 2023-11-18T20:43:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.278+0000 7f41267f5640 1 -- 172.21.15.60:0/255458140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4120140310 msgr2=0x7f412013dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.278+0000 7f41267f5640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4120140310 0x7f412013dd10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f411402be50 tx=0x7f41140046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.278+0000 7f41267f5640 1 -- 172.21.15.60:0/255458140 shutdown_connections 2023-11-18T20:43:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.278+0000 7f41267f5640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f410c06ae70 0x7f410c06d330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.279+0000 7f41267f5640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41201409d0 0x7f41201f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.279+0000 7f41267f5640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4120140310 0x7f412013dd10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.279+0000 7f41267f5640 1 --2- 172.21.15.60:0/255458140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f412006abe0 0x7f4120145620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.279+0000 7f41267f5640 1 -- 172.21.15.60:0/255458140 >> 172.21.15.60:0/255458140 conn(0x7f41200b1e50 msgr2=0x7f412006ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.279+0000 7f41267f5640 1 -- 172.21.15.60:0/255458140 shutdown_connections 2023-11-18T20:43:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.279+0000 7f41267f5640 1 -- 172.21.15.60:0/255458140 wait complete. 2023-11-18T20:43:03.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.398+0000 7f79df9ce640 1 Processor -- start 2023-11-18T20:43:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.398+0000 7f79df9ce640 1 -- start start 2023-11-18T20:43:03.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79df9ce640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79d8150560 0x7f79d80b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.401 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79df9ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f79d80b3390 0x7f79d815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79df9ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d8150940 0x7f79d815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79df9ce640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80b49f0 con 0x7f79d8150560 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79df9ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80b3de0 con 0x7f79d80b3390 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79df9ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80b40c0 con 0x7f79d8150940 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79dd76a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79d8150560 0x7f79d80b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79dd76a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79d8150560 0x7f79d80b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42504/0 (socket says 172.21.15.60:42504) 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79dd76a640 1 -- 172.21.15.60:0/2154736768 learned_addr learned my addr 172.21.15.60:0/2154736768 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79ddf6b640 1 --2- 172.21.15.60:0/2154736768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d8150940 0x7f79d815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79dcf69640 1 --2- 172.21.15.60:0/2154736768 >> v2:172.21.15.100:3300/0 conn(0x7f79d80b3390 0x7f79d815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79dd76a640 1 -- 172.21.15.60:0/2154736768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d8150940 msgr2=0x7f79d815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79dd76a640 1 --2- 172.21.15.60:0/2154736768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d8150940 0x7f79d815d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79dd76a640 1 -- 172.21.15.60:0/2154736768 >> v2:172.21.15.100:3300/0 conn(0x7f79d80b3390 msgr2=0x7f79d815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:03.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79dd76a640 1 --2- 172.21.15.60:0/2154736768 >> v2:172.21.15.100:3300/0 conn(0x7f79d80b3390 0x7f79d815d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.399+0000 7f79dd76a640 1 -- 172.21.15.60:0/2154736768 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80b43a0 con 0x7f79d8150560 2023-11-18T20:43:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.400+0000 7f79dd76a640 1 --2- 172.21.15.60:0/2154736768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8150560 0x7f79d80b2e50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f79d8072810 tx=0x7f79d402db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e125da5af3b126bd server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.400+0000 7f79cffff640 1 -- 172.21.15.60:0/2154736768 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d402f8e0 con 0x7f79d8150560 2023-11-18T20:43:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.400+0000 7f79cffff640 1 -- 172.21.15.60:0/2154736768 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d402fa40 con 0x7f79d8150560 2023-11-18T20:43:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.400+0000 7f79cffff640 1 -- 172.21.15.60:0/2154736768 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d402fdd0 con 0x7f79d8150560 2023-11-18T20:43:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.400+0000 7f79df9ce640 1 -- 172.21.15.60:0/2154736768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8150560 msgr2=0x7f79d80b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.400+0000 7f79df9ce640 1 --2- 172.21.15.60:0/2154736768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8150560 0x7f79d80b2e50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f79d8072810 tx=0x7f79d402db20 comp rx=0 tx=0).stop 2023-11-18T20:43:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.401+0000 7f79df9ce640 1 -- 172.21.15.60:0/2154736768 shutdown_connections 2023-11-18T20:43:03.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.401+0000 7f79df9ce640 1 --2- 172.21.15.60:0/2154736768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d8150940 0x7f79d815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.401+0000 7f79df9ce640 1 --2- 172.21.15.60:0/2154736768 >> v2:172.21.15.100:3300/0 conn(0x7f79d80b3390 0x7f79d815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.401+0000 7f79df9ce640 1 --2- 172.21.15.60:0/2154736768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8150560 0x7f79d80b2e50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.401+0000 7f79df9ce640 1 -- 172.21.15.60:0/2154736768 >> 172.21.15.60:0/2154736768 conn(0x7f79d806a4d0 msgr2=0x7f79d806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.401+0000 7f79df9ce640 1 -- 172.21.15.60:0/2154736768 shutdown_connections 2023-11-18T20:43:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.401+0000 7f79df9ce640 1 -- 172.21.15.60:0/2154736768 wait complete. 2023-11-18T20:43:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.402+0000 7f79df9ce640 1 Processor -- start 2023-11-18T20:43:03.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.402+0000 7f79df9ce640 1 -- start start 2023-11-18T20:43:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79df9ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d80b3390 0x7f79d8147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79df9ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d8150560 0x7f79d8159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79df9ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8159b70 0x7f79d80ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79df9ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8147eb0 con 0x7f79d8159b70 2023-11-18T20:43:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79df9ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80ac8e0 con 0x7f79d8150560 2023-11-18T20:43:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79df9ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80acbc0 con 0x7f79d80b3390 2023-11-18T20:43:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79ddf6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8159b70 0x7f79d80ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79ddf6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8159b70 0x7f79d80ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42520/0 (socket says 172.21.15.60:42520) 2023-11-18T20:43:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79ddf6b640 1 -- 172.21.15.60:0/636713077 learned_addr learned my addr 172.21.15.60:0/636713077 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79dcf69640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d8150560 0x7f79d8159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79dd76a640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d80b3390 0x7f79d8147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79ddf6b640 1 -- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d80b3390 msgr2=0x7f79d8147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79ddf6b640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d80b3390 0x7f79d8147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79ddf6b640 1 -- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d8150560 msgr2=0x7f79d8159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79ddf6b640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d8150560 0x7f79d8159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79ddf6b640 1 -- 172.21.15.60:0/636713077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d40094d0 con 0x7f79d8159b70 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79ddf6b640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8159b70 0x7f79d80ad980 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f79d000cc80 tx=0x7f79d000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79cdffb640 1 -- 172.21.15.60:0/636713077 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d000ce30 con 0x7f79d8159b70 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79cdffb640 1 -- 172.21.15.60:0/636713077 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0003990 con 0x7f79d8159b70 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79df9ce640 1 -- 172.21.15.60:0/636713077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d80acea0 con 0x7f79d8159b70 2023-11-18T20:43:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.403+0000 7f79cdffb640 1 -- 172.21.15.60:0/636713077 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0003d80 con 0x7f79d8159b70 2023-11-18T20:43:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.404+0000 7f79df9ce640 1 -- 172.21.15.60:0/636713077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d80ad460 con 0x7f79d8159b70 2023-11-18T20:43:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.405+0000 7f79df9ce640 1 -- 172.21.15.60:0/636713077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d80b4e40 con 0x7f79d8159b70 2023-11-18T20:43:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.405+0000 7f79cdffb640 1 -- 172.21.15.60:0/636713077 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79d0003ee0 con 0x7f79d8159b70 2023-11-18T20:43:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.405+0000 7f79cdffb640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79bc06ae40 0x7f79bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.405+0000 7f79cdffb640 1 -- 172.21.15.60:0/636713077 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79d0089b30 con 0x7f79d8159b70 2023-11-18T20:43:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.406+0000 7f79dd76a640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79bc06ae40 0x7f79bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.406+0000 7f79dd76a640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79bc06ae40 0x7f79bc06d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f79d4038000 tx=0x7f79d4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:03.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.409+0000 7f79cdffb640 1 -- 172.21.15.60:0/636713077 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79d0054d20 con 0x7f79d8159b70 2023-11-18T20:43:03.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.585+0000 7f79df9ce640 1 -- 172.21.15.60:0/636713077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79d8069400 con 0x7f79d8159b70 2023-11-18T20:43:03.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.586+0000 7f79cdffb640 1 -- 172.21.15.60:0/636713077 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79d001d020 con 0x7f79d8159b70 2023-11-18T20:43:03.588 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:03.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 -- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79bc06ae40 msgr2=0x7f79bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79bc06ae40 0x7f79bc06d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f79d4038000 tx=0x7f79d4008030 comp rx=0 tx=0).stop 2023-11-18T20:43:03.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 -- 172.21.15.60:0/636713077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8159b70 msgr2=0x7f79d80ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8159b70 0x7f79d80ad980 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f79d000cc80 tx=0x7f79d000a5c0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 -- 172.21.15.60:0/636713077 shutdown_connections 2023-11-18T20:43:03.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79bc06ae40 0x7f79bc06d300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d8159b70 0x7f79d80ad980 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d8150560 0x7f79d8159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 --2- 172.21.15.60:0/636713077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d80b3390 0x7f79d8147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 -- 172.21.15.60:0/636713077 >> 172.21.15.60:0/636713077 conn(0x7f79d806a4d0 msgr2=0x7f79d8071150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:03.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 -- 172.21.15.60:0/636713077 shutdown_connections 2023-11-18T20:43:03.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.588+0000 7f79df9ce640 1 -- 172.21.15.60:0/636713077 wait complete. 2023-11-18T20:43:03.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:03.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.708+0000 7f67b67c0640 1 Processor -- start 2023-11-18T20:43:03.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.708+0000 7f67b67c0640 1 -- start start 2023-11-18T20:43:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67b67c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b0143490 0x7f67b00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67b67c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67b015abf0 0x7f67b00b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67b67c0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b015e2a0 con 0x7f67b015e730 2023-11-18T20:43:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67b67c0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b015b520 con 0x7f67b015abf0 2023-11-18T20:43:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67b67c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0143870 con 0x7f67b0143490 2023-11-18T20:43:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67affff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b0143490 0x7f67b00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67b4d5d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67b015abf0 0x7f67b00b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67af7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f67b015e730 0x7f67b015a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:39018/0 (socket says 172.21.15.60:39018) 2023-11-18T20:43:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67b4d5d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67b015abf0 0x7f67b00b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33856/0 (socket says 172.21.15.60:33856) 2023-11-18T20:43:03.712 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67affff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b0143490 0x7f67b00b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43806/0 (socket says 172.21.15.60:43806) 2023-11-18T20:43:03.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.709+0000 7f67b4d5d640 1 -- 172.21.15.60:0/2974270814 learned_addr learned my addr 172.21.15.60:0/2974270814 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:03.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.710+0000 7f67b4d5d640 1 -- 172.21.15.60:0/2974270814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b0143490 msgr2=0x7f67b00b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:03.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.710+0000 7f67b4d5d640 1 --2- 172.21.15.60:0/2974270814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b0143490 0x7f67b00b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.710+0000 7f67b4d5d640 1 -- 172.21.15.60:0/2974270814 >> v1:172.21.15.100:6789/0 conn(0x7f67b015e730 legacy=0x7f67b015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.710+0000 7f67b4d5d640 1 -- 172.21.15.60:0/2974270814 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b015b240 con 0x7f67b015abf0 2023-11-18T20:43:03.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.710+0000 7f67affff640 1 --2- 172.21.15.60:0/2974270814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b0143490 0x7f67b00b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:03.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.710+0000 7f67b4d5d640 1 --2- 172.21.15.60:0/2974270814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b015abf0 0x7f67b00b1230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009e50 tx=0x7f67a802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e9278ce2a3e725 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:03.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67aeffd640 1 -- 172.21.15.60:0/2974270814 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a802cb00 con 0x7f67b015abf0 2023-11-18T20:43:03.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67aeffd640 1 -- 172.21.15.60:0/2974270814 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a802cc60 con 0x7f67b015abf0 2023-11-18T20:43:03.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67aeffd640 1 -- 172.21.15.60:0/2974270814 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a80343f0 con 0x7f67b015abf0 2023-11-18T20:43:03.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67b67c0640 1 -- 172.21.15.60:0/2974270814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b015abf0 msgr2=0x7f67b00b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67b67c0640 1 --2- 172.21.15.60:0/2974270814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b015abf0 0x7f67b00b1230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009e50 tx=0x7f67a802bf60 comp rx=0 tx=0).stop 2023-11-18T20:43:03.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67b67c0640 1 -- 172.21.15.60:0/2974270814 shutdown_connections 2023-11-18T20:43:03.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67b67c0640 1 --2- 172.21.15.60:0/2974270814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b015abf0 0x7f67b00b1230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67b67c0640 1 --2- 172.21.15.60:0/2974270814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b0143490 0x7f67b00b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67b67c0640 1 -- 172.21.15.60:0/2974270814 >> 172.21.15.60:0/2974270814 conn(0x7f67b006a4d0 msgr2=0x7f67b006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:03.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67b67c0640 1 -- 172.21.15.60:0/2974270814 shutdown_connections 2023-11-18T20:43:03.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.711+0000 7f67b67c0640 1 -- 172.21.15.60:0/2974270814 wait complete. 2023-11-18T20:43:03.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.712+0000 7f67b67c0640 1 Processor -- start 2023-11-18T20:43:03.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.713+0000 7f67b67c0640 1 -- start start 2023-11-18T20:43:03.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.713+0000 7f67b67c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b0143490 0x7f67b0158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.713+0000 7f67b67c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b015abf0 0x7f67b0158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.713+0000 7f67b67c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b015e730 0x7f67b0140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.713+0000 7f67b67c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b013b8c0 con 0x7f67b0143490 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.713+0000 7f67b67c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01416e0 con 0x7f67b015abf0 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.713+0000 7f67b67c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01419c0 con 0x7f67b015e730 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.713+0000 7f67affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b0143490 0x7f67b0158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b0143490 0x7f67b0158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42522/0 (socket says 172.21.15.60:42522) 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67affff640 1 -- 172.21.15.60:0/3571074538 learned_addr learned my addr 172.21.15.60:0/3571074538 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67b4d5d640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b015e730 0x7f67b0140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67af7fe640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b015abf0 0x7f67b0158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67affff640 1 -- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b015e730 msgr2=0x7f67b0140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67affff640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b015e730 0x7f67b0140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67affff640 1 -- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b015abf0 msgr2=0x7f67b0158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67affff640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b015abf0 0x7f67b0158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67affff640 1 -- 172.21.15.60:0/3571074538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a0009400 con 0x7f67b0143490 2023-11-18T20:43:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67af7fe640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b015abf0 0x7f67b0158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67b4d5d640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b015e730 0x7f67b0140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67affff640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b0143490 0x7f67b0158590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f67a000ebe0 tx=0x7f67a000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67acff9640 1 -- 172.21.15.60:0/3571074538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0009760 con 0x7f67b0143490 2023-11-18T20:43:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67acff9640 1 -- 172.21.15.60:0/3571074538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a000cc30 con 0x7f67b0143490 2023-11-18T20:43:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67b67c0640 1 -- 172.21.15.60:0/3571074538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a8009b30 con 0x7f67b0143490 2023-11-18T20:43:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67acff9640 1 -- 172.21.15.60:0/3571074538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a000eee0 con 0x7f67b0143490 2023-11-18T20:43:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.714+0000 7f67b67c0640 1 -- 172.21.15.60:0/3571074538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b013bb10 con 0x7f67b0143490 2023-11-18T20:43:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.715+0000 7f67b67c0640 1 -- 172.21.15.60:0/3571074538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b0141ee0 con 0x7f67b0143490 2023-11-18T20:43:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.716+0000 7f67acff9640 1 -- 172.21.15.60:0/3571074538 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f67a0013470 con 0x7f67b0143490 2023-11-18T20:43:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.716+0000 7f67acff9640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f679c06ae70 0x7f679c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.716+0000 7f67acff9640 1 -- 172.21.15.60:0/3571074538 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67a00898a0 con 0x7f67b0143490 2023-11-18T20:43:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.717+0000 7f67af7fe640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f679c06ae70 0x7f679c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.717+0000 7f67af7fe640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f679c06ae70 0x7f679c06d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67a4007fd0 tx=0x7f67a4007d40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:03.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.721+0000 7f67acff9640 1 -- 172.21.15.60:0/3571074538 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67a0054950 con 0x7f67b0143490 2023-11-18T20:43:03.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.895+0000 7f67b67c0640 1 -- 172.21.15.60:0/3571074538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67b0069400 con 0x7f67b0143490 2023-11-18T20:43:03.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.895+0000 7f67acff9640 1 -- 172.21.15.60:0/3571074538 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f67a0057f70 con 0x7f67b0143490 2023-11-18T20:43:03.897 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 -- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f679c06ae70 msgr2=0x7f679c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f679c06ae70 0x7f679c06d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67a4007fd0 tx=0x7f67a4007d40 comp rx=0 tx=0).stop 2023-11-18T20:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 -- 172.21.15.60:0/3571074538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b0143490 msgr2=0x7f67b0158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b0143490 0x7f67b0158590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f67a000ebe0 tx=0x7f67a000c520 comp rx=0 tx=0).stop 2023-11-18T20:43:03.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 -- 172.21.15.60:0/3571074538 shutdown_connections 2023-11-18T20:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f679c06ae70 0x7f679c06d330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b015e730 0x7f67b0140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b015abf0 0x7f67b0158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 --2- 172.21.15.60:0/3571074538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b0143490 0x7f67b0158590 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 -- 172.21.15.60:0/3571074538 >> 172.21.15.60:0/3571074538 conn(0x7f67b006a4d0 msgr2=0x7f67b00b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 -- 172.21.15.60:0/3571074538 shutdown_connections 2023-11-18T20:43:03.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:03.897+0000 7f67b67c0640 1 -- 172.21.15.60:0/3571074538 wait complete. 2023-11-18T20:43:03.934 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2023-11-18T20:43:03.934 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-11-18T20:43:03.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.039+0000 7f255d5b0640 1 Processor -- start 2023-11-18T20:43:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.039+0000 7f255d5b0640 1 -- start start 2023-11-18T20:43:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f255d5b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2558160fb0 0x7f2558161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f255d5b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25580d0ed0 0x7f25580d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f255d5b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2558161d20 con 0x7f255806ea50 2023-11-18T20:43:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f255d5b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255815e760 con 0x7f25580d0ed0 2023-11-18T20:43:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f255d5b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255815d9a0 con 0x7f2558160fb0 2023-11-18T20:43:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f25577fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25580d0ed0 0x7f25580d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f2556ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f255806ea50 0x7f255806ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50066/0 (socket says 172.21.15.60:50066) 2023-11-18T20:43:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f2556ffd640 1 -- 172.21.15.60:0/850652037 learned_addr learned my addr 172.21.15.60:0/850652037 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f25567fc640 1 --2- 172.21.15.60:0/850652037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2558160fb0 0x7f2558161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f2555ffb640 1 -- 172.21.15.60:0/850652037 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335331332 0 0) 0x7f2558161d20 con 0x7f255806ea50 2023-11-18T20:43:04.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f2555ffb640 1 -- 172.21.15.60:0/850652037 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2544003520 con 0x7f255806ea50 2023-11-18T20:43:04.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f25577fe640 1 -- 172.21.15.60:0/850652037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2558160fb0 msgr2=0x7f2558161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:04.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f25577fe640 1 --2- 172.21.15.60:0/850652037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2558160fb0 0x7f2558161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.040+0000 7f25577fe640 1 -- 172.21.15.60:0/850652037 >> v1:172.21.15.60:6789/0 conn(0x7f255806ea50 legacy=0x7f255806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.041+0000 7f25577fe640 1 -- 172.21.15.60:0/850652037 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255815dc80 con 0x7f25580d0ed0 2023-11-18T20:43:04.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.041+0000 7f25577fe640 1 --2- 172.21.15.60:0/850652037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25580d0ed0 0x7f25580d1360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2548009d90 tx=0x7f254802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdb3e0ca8241a34 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.041+0000 7f2555ffb640 1 -- 172.21.15.60:0/850652037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254802cc50 con 0x7f25580d0ed0 2023-11-18T20:43:04.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.041+0000 7f2555ffb640 1 -- 172.21.15.60:0/850652037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254802cdb0 con 0x7f25580d0ed0 2023-11-18T20:43:04.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.041+0000 7f2555ffb640 1 -- 172.21.15.60:0/850652037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548034660 con 0x7f25580d0ed0 2023-11-18T20:43:04.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.042+0000 7f255d5b0640 1 -- 172.21.15.60:0/850652037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25580d0ed0 msgr2=0x7f25580d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.042+0000 7f255d5b0640 1 --2- 172.21.15.60:0/850652037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25580d0ed0 0x7f25580d1360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2548009d90 tx=0x7f254802bea0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.042+0000 7f255d5b0640 1 -- 172.21.15.60:0/850652037 shutdown_connections 2023-11-18T20:43:04.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.042+0000 7f255d5b0640 1 --2- 172.21.15.60:0/850652037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25580d0ed0 0x7f25580d1360 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.042+0000 7f255d5b0640 1 --2- 172.21.15.60:0/850652037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2558160fb0 0x7f2558161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.042+0000 7f255d5b0640 1 -- 172.21.15.60:0/850652037 >> 172.21.15.60:0/850652037 conn(0x7f255806a6b0 msgr2=0x7f255806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:04.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.042+0000 7f255d5b0640 1 -- 172.21.15.60:0/850652037 shutdown_connections 2023-11-18T20:43:04.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.042+0000 7f255d5b0640 1 -- 172.21.15.60:0/850652037 wait complete. 2023-11-18T20:43:04.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.043+0000 7f255d5b0640 1 Processor -- start 2023-11-18T20:43:04.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.043+0000 7f255d5b0640 1 -- start start 2023-11-18T20:43:04.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f255d5b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f255806ea50 0x7f2558157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f255d5b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25580d0ed0 0x7f255815a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f255d5b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2558160fb0 0x7f255806c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f255d5b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2558135eb0 con 0x7f255806ea50 2023-11-18T20:43:04.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f255d5b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255813fb80 con 0x7f25580d0ed0 2023-11-18T20:43:04.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f255d5b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2558159240 con 0x7f2558160fb0 2023-11-18T20:43:04.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f2556ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f255806ea50 0x7f2558157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f2556ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f255806ea50 0x7f2558157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42536/0 (socket says 172.21.15.60:42536) 2023-11-18T20:43:04.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f2556ffd640 1 -- 172.21.15.60:0/1487773293 learned_addr learned my addr 172.21.15.60:0/1487773293 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:04.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f2556ffd640 1 -- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2558160fb0 msgr2=0x7f255806c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f2556ffd640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2558160fb0 0x7f255806c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f2556ffd640 1 -- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25580d0ed0 msgr2=0x7f255815a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f2556ffd640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25580d0ed0 0x7f255815a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.044+0000 7f2556ffd640 1 -- 172.21.15.60:0/1487773293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548009a70 con 0x7f255806ea50 2023-11-18T20:43:04.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.045+0000 7f2556ffd640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f255806ea50 0x7f2558157090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f255000c440 tx=0x7f255000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.045+0000 7f254f7fe640 1 -- 172.21.15.60:0/1487773293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550005ce0 con 0x7f255806ea50 2023-11-18T20:43:04.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.045+0000 7f254f7fe640 1 -- 172.21.15.60:0/1487773293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2550005e40 con 0x7f255806ea50 2023-11-18T20:43:04.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.045+0000 7f255d5b0640 1 -- 172.21.15.60:0/1487773293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2558159520 con 0x7f255806ea50 2023-11-18T20:43:04.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.045+0000 7f254f7fe640 1 -- 172.21.15.60:0/1487773293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25500072b0 con 0x7f255806ea50 2023-11-18T20:43:04.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.045+0000 7f255d5b0640 1 -- 172.21.15.60:0/1487773293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2558159ae0 con 0x7f255806ea50 2023-11-18T20:43:04.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.046+0000 7f255d5b0640 1 -- 172.21.15.60:0/1487773293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2558069400 con 0x7f255806ea50 2023-11-18T20:43:04.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.046+0000 7f254f7fe640 1 -- 172.21.15.60:0/1487773293 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2550007410 con 0x7f255806ea50 2023-11-18T20:43:04.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.047+0000 7f254f7fe640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f253806ae40 0x7f253806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.047+0000 7f254f7fe640 1 -- 172.21.15.60:0/1487773293 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2550088b80 con 0x7f255806ea50 2023-11-18T20:43:04.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.047+0000 7f25567fc640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f253806ae40 0x7f253806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.047+0000 7f25567fc640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f253806ae40 0x7f253806d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f255815a700 tx=0x7f2540008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.051+0000 7f254f7fe640 1 -- 172.21.15.60:0/1487773293 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2550053cf0 con 0x7f255806ea50 2023-11-18T20:43:04.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.228+0000 7f255d5b0640 1 -- 172.21.15.60:0/1487773293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25580cc030 con 0x7f255806ea50 2023-11-18T20:43:04.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.229+0000 7f254f7fe640 1 -- 172.21.15.60:0/1487773293 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f2550017020 con 0x7f255806ea50 2023-11-18T20:43:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:04.231 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 -- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f253806ae40 msgr2=0x7f253806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f253806ae40 0x7f253806d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f255815a700 tx=0x7f2540008040 comp rx=0 tx=0).stop 2023-11-18T20:43:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 -- 172.21.15.60:0/1487773293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f255806ea50 msgr2=0x7f2558157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f255806ea50 0x7f2558157090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f255000c440 tx=0x7f255000c900 comp rx=0 tx=0).stop 2023-11-18T20:43:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 -- 172.21.15.60:0/1487773293 shutdown_connections 2023-11-18T20:43:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f253806ae40 0x7f253806d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2558160fb0 0x7f255806c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25580d0ed0 0x7f255815a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 --2- 172.21.15.60:0/1487773293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f255806ea50 0x7f2558157090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 -- 172.21.15.60:0/1487773293 >> 172.21.15.60:0/1487773293 conn(0x7f255806a6b0 msgr2=0x7f25580b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 -- 172.21.15.60:0/1487773293 shutdown_connections 2023-11-18T20:43:04.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.231+0000 7f255d5b0640 1 -- 172.21.15.60:0/1487773293 wait complete. 2023-11-18T20:43:04.244 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:04.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:04.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.352+0000 7f2525741640 1 Processor -- start 2023-11-18T20:43:04.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.352+0000 7f2525741640 1 -- start start 2023-11-18T20:43:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f2525741640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f252015d330 0x7f25200b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f2525741640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25200d2910 0x7f25200d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f2525741640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25200b2b10 con 0x7f25200d2d90 2023-11-18T20:43:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f2525741640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25200d1590 con 0x7f25200d2910 2023-11-18T20:43:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f2525741640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25200d1870 con 0x7f252015d330 2023-11-18T20:43:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f251e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25200d2910 0x7f25200d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f251f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f25200d2d90 0x7f25200b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50068/0 (socket says 172.21.15.60:50068) 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f251f7fe640 1 -- 172.21.15.60:0/2864918257 learned_addr learned my addr 172.21.15.60:0/2864918257 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f251effd640 1 --2- 172.21.15.60:0/2864918257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f252015d330 0x7f25200b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f251e7fc640 1 -- 172.21.15.60:0/2864918257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f252015d330 msgr2=0x7f25200b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f251e7fc640 1 --2- 172.21.15.60:0/2864918257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f252015d330 0x7f25200b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f251e7fc640 1 -- 172.21.15.60:0/2864918257 >> v1:172.21.15.60:6789/0 conn(0x7f25200d2d90 legacy=0x7f25200b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f251dffb640 1 -- 172.21.15.60:0/2864918257 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681080235 0 0) 0x7f25200b2b10 con 0x7f25200d2d90 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.353+0000 7f251e7fc640 1 -- 172.21.15.60:0/2864918257 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25200d1b50 con 0x7f25200d2910 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.354+0000 7f251e7fc640 1 --2- 172.21.15.60:0/2864918257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25200d2910 0x7f25200d31f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f25140097a0 tx=0x7f251402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a484dcd38b81985 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.354+0000 7f251dffb640 1 -- 172.21.15.60:0/2864918257 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251402e070 con 0x7f25200d2910 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.354+0000 7f251dffb640 1 -- 172.21.15.60:0/2864918257 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25140315a0 con 0x7f25200d2910 2023-11-18T20:43:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.354+0000 7f251dffb640 1 -- 172.21.15.60:0/2864918257 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514031930 con 0x7f25200d2910 2023-11-18T20:43:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.355+0000 7f2525741640 1 -- 172.21.15.60:0/2864918257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25200d2910 msgr2=0x7f25200d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.355+0000 7f2525741640 1 --2- 172.21.15.60:0/2864918257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25200d2910 0x7f25200d31f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f25140097a0 tx=0x7f251402b840 comp rx=0 tx=0).stop 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.355+0000 7f2525741640 1 -- 172.21.15.60:0/2864918257 shutdown_connections 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.355+0000 7f2525741640 1 --2- 172.21.15.60:0/2864918257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25200d2910 0x7f25200d31f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.355+0000 7f2525741640 1 --2- 172.21.15.60:0/2864918257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f252015d330 0x7f25200b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.355+0000 7f2525741640 1 -- 172.21.15.60:0/2864918257 >> 172.21.15.60:0/2864918257 conn(0x7f2520072020 msgr2=0x7f2520072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.355+0000 7f2525741640 1 -- 172.21.15.60:0/2864918257 shutdown_connections 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.355+0000 7f2525741640 1 -- 172.21.15.60:0/2864918257 wait complete. 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.356+0000 7f2525741640 1 Processor -- start 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.356+0000 7f2525741640 1 -- start start 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f2525741640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25200d2910 0x7f2520149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f2525741640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25200d2d90 0x7f2520149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f2525741640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252015d330 0x7f252015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f2525741640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25200cab40 con 0x7f252015d330 2023-11-18T20:43:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f2525741640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252015f380 con 0x7f25200d2910 2023-11-18T20:43:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f2525741640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25200ca330 con 0x7f25200d2d90 2023-11-18T20:43:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252015d330 0x7f252015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252015d330 0x7f252015f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42542/0 (socket says 172.21.15.60:42542) 2023-11-18T20:43:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251f7fe640 1 -- 172.21.15.60:0/1236207863 learned_addr learned my addr 172.21.15.60:0/1236207863 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251effd640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25200d2910 0x7f2520149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251e7fc640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25200d2d90 0x7f2520149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251f7fe640 1 -- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25200d2d90 msgr2=0x7f2520149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251f7fe640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25200d2d90 0x7f2520149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251f7fe640 1 -- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25200d2910 msgr2=0x7f2520149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251f7fe640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25200d2910 0x7f2520149c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251f7fe640 1 -- 172.21.15.60:0/1236207863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514009400 con 0x7f252015d330 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.357+0000 7f251e7fc640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25200d2d90 0x7f2520149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.358+0000 7f251f7fe640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252015d330 0x7f252015f7d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f251800c430 tx=0x7f251800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.358+0000 7f25077fe640 1 -- 172.21.15.60:0/1236207863 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180074e0 con 0x7f252015d330 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.358+0000 7f25077fe640 1 -- 172.21.15.60:0/1236207863 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2518007660 con 0x7f252015d330 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.358+0000 7f2525741640 1 -- 172.21.15.60:0/1236207863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25200cd4e0 con 0x7f252015d330 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.358+0000 7f25077fe640 1 -- 172.21.15.60:0/1236207863 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518004a20 con 0x7f252015d330 2023-11-18T20:43:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.358+0000 7f2525741640 1 -- 172.21.15.60:0/1236207863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25200ca860 con 0x7f252015d330 2023-11-18T20:43:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.359+0000 7f2525741640 1 -- 172.21.15.60:0/1236207863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252015f4f0 con 0x7f252015d330 2023-11-18T20:43:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.359+0000 7f25077fe640 1 -- 172.21.15.60:0/1236207863 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2518004b80 con 0x7f252015d330 2023-11-18T20:43:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.360+0000 7f25077fe640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f250c06ae40 0x7f250c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.360+0000 7f25077fe640 1 -- 172.21.15.60:0/1236207863 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2518089550 con 0x7f252015d330 2023-11-18T20:43:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.360+0000 7f251effd640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f250c06ae40 0x7f250c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.360+0000 7f251effd640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f250c06ae40 0x7f250c06d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2510009a60 tx=0x7f2510003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.364+0000 7f25077fe640 1 -- 172.21.15.60:0/1236207863 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f252015f4f0 con 0x7f252015d330 2023-11-18T20:43:04.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.538+0000 7f2525741640 1 -- 172.21.15.60:0/1236207863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f252015f4f0 con 0x7f252015d330 2023-11-18T20:43:04.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.539+0000 7f25077fe640 1 -- 172.21.15.60:0/1236207863 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f252015f4f0 con 0x7f252015d330 2023-11-18T20:43:04.541 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:04.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 -- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f250c06ae40 msgr2=0x7f250c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f250c06ae40 0x7f250c06d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2510009a60 tx=0x7f2510003990 comp rx=0 tx=0).stop 2023-11-18T20:43:04.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 -- 172.21.15.60:0/1236207863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252015d330 msgr2=0x7f252015f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252015d330 0x7f252015f7d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f251800c430 tx=0x7f251800c8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 -- 172.21.15.60:0/1236207863 shutdown_connections 2023-11-18T20:43:04.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f250c06ae40 0x7f250c06d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f252015d330 0x7f252015f7d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25200d2d90 0x7f2520149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 --2- 172.21.15.60:0/1236207863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25200d2910 0x7f2520149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 -- 172.21.15.60:0/1236207863 >> 172.21.15.60:0/1236207863 conn(0x7f2520072020 msgr2=0x7f25200d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:04.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 -- 172.21.15.60:0/1236207863 shutdown_connections 2023-11-18T20:43:04.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.541+0000 7f2525741640 1 -- 172.21.15.60:0/1236207863 wait complete. 2023-11-18T20:43:04.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:04.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.660+0000 7f7f04f33640 1 Processor -- start 2023-11-18T20:43:04.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.661+0000 7f7f04f33640 1 -- start start 2023-11-18T20:43:04.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.661+0000 7f7f04f33640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f0015d330 0x7f7f000b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.661+0000 7f7f04f33640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7f000d3900 0x7f7f000b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.661+0000 7f7f04f33640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f000b2b10 con 0x7f7f000d2910 2023-11-18T20:43:04.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.661+0000 7f7f04f33640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f000d1590 con 0x7f7f0015d330 2023-11-18T20:43:04.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.661+0000 7f7f04f33640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f000d1870 con 0x7f7f000d3900 2023-11-18T20:43:04.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.661+0000 7f7efe59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f0015d330 0x7f7f000b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.661+0000 7f7ef7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7f000d2910 0x7f7f000d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50082/0 (socket says 172.21.15.60:50082) 2023-11-18T20:43:04.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.661+0000 7f7ef7fff640 1 -- 172.21.15.60:0/3459437239 learned_addr learned my addr 172.21.15.60:0/3459437239 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:04.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.662+0000 7f7efed9d640 1 --2- 172.21.15.60:0/3459437239 >> v2:172.21.15.100:3300/0 conn(0x7f7f000d3900 0x7f7f000b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.662+0000 7f7efe59c640 1 -- 172.21.15.60:0/3459437239 >> v2:172.21.15.100:3300/0 conn(0x7f7f000d3900 msgr2=0x7f7f000b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:04.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.662+0000 7f7efe59c640 1 --2- 172.21.15.60:0/3459437239 >> v2:172.21.15.100:3300/0 conn(0x7f7f000d3900 0x7f7f000b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.662+0000 7f7efe59c640 1 -- 172.21.15.60:0/3459437239 >> v1:172.21.15.60:6789/0 conn(0x7f7f000d2910 legacy=0x7f7f000d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.662+0000 7f7efe59c640 1 -- 172.21.15.60:0/3459437239 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f000d1b50 con 0x7f7f0015d330 2023-11-18T20:43:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7efe59c640 1 --2- 172.21.15.60:0/3459437239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f0015d330 0x7f7f000b8b80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f0006c500 tx=0x7f7ef002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744b8df8c77f3483 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7efdd9b640 1 -- 172.21.15.60:0/3459437239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef002e070 con 0x7f7f0015d330 2023-11-18T20:43:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7efdd9b640 1 -- 172.21.15.60:0/3459437239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef00315a0 con 0x7f7f0015d330 2023-11-18T20:43:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7efdd9b640 1 -- 172.21.15.60:0/3459437239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0031930 con 0x7f7f0015d330 2023-11-18T20:43:04.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7f04f33640 1 -- 172.21.15.60:0/3459437239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f0015d330 msgr2=0x7f7f000b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7f04f33640 1 --2- 172.21.15.60:0/3459437239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f0015d330 0x7f7f000b8b80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f0006c500 tx=0x7f7ef002b840 comp rx=0 tx=0).stop 2023-11-18T20:43:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7f04f33640 1 -- 172.21.15.60:0/3459437239 shutdown_connections 2023-11-18T20:43:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7f04f33640 1 --2- 172.21.15.60:0/3459437239 >> v2:172.21.15.100:3300/0 conn(0x7f7f000d3900 0x7f7f000b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7f04f33640 1 --2- 172.21.15.60:0/3459437239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f0015d330 0x7f7f000b8b80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7f04f33640 1 -- 172.21.15.60:0/3459437239 >> 172.21.15.60:0/3459437239 conn(0x7f7f00072020 msgr2=0x7f7f00072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7f04f33640 1 -- 172.21.15.60:0/3459437239 shutdown_connections 2023-11-18T20:43:04.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.663+0000 7f7f04f33640 1 -- 172.21.15.60:0/3459437239 wait complete. 2023-11-18T20:43:04.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7f04f33640 1 Processor -- start 2023-11-18T20:43:04.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7f04f33640 1 -- start start 2023-11-18T20:43:04.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7f04f33640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f000d2910 0x7f7f001f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7f04f33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f000d3900 0x7f7f0014b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7f04f33640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f0015d330 0x7f7f000cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7f04f33640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000ca330 con 0x7f7f000d2910 2023-11-18T20:43:04.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7f04f33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000c7bb0 con 0x7f7f000d3900 2023-11-18T20:43:04.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7f04f33640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000ca580 con 0x7f7f0015d330 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7efe59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f000d2910 0x7f7f001f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7efe59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f000d2910 0x7f7f001f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42550/0 (socket says 172.21.15.60:42550) 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.665+0000 7f7efe59c640 1 -- 172.21.15.60:0/3632990966 learned_addr learned my addr 172.21.15.60:0/3632990966 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7efe59c640 1 -- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f0015d330 msgr2=0x7f7f000cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7ef7fff640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f000d3900 0x7f7f0014b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7efe59c640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f0015d330 0x7f7f000cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7efe59c640 1 -- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f000d3900 msgr2=0x7f7f0014b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7efe59c640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f000d3900 0x7f7f0014b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7efe59c640 1 -- 172.21.15.60:0/3632990966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0009400 con 0x7f7f000d2910 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7efe59c640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f000d2910 0x7f7f001f3f40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009890 tx=0x7f7ef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7ef6ffd640 1 -- 172.21.15.60:0/3632990966 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0032900 con 0x7f7f000d2910 2023-11-18T20:43:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7ef6ffd640 1 -- 172.21.15.60:0/3632990966 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef0032a60 con 0x7f7f000d2910 2023-11-18T20:43:04.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7ef6ffd640 1 -- 172.21.15.60:0/3632990966 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0032e10 con 0x7f7f000d2910 2023-11-18T20:43:04.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7f04f33640 1 -- 172.21.15.60:0/3632990966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f000ca860 con 0x7f7f000d2910 2023-11-18T20:43:04.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.666+0000 7f7f04f33640 1 -- 172.21.15.60:0/3632990966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0014bb50 con 0x7f7f000d2910 2023-11-18T20:43:04.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.667+0000 7f7f04f33640 1 -- 172.21.15.60:0/3632990966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f00066da0 con 0x7f7f000d2910 2023-11-18T20:43:04.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.667+0000 7f7ef6ffd640 1 -- 172.21.15.60:0/3632990966 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ef003b3f0 con 0x7f7f000d2910 2023-11-18T20:43:04.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.668+0000 7f7ef6ffd640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7ee006ae40 0x7f7ee006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.668+0000 7f7ef6ffd640 1 -- 172.21.15.60:0/3632990966 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ef00ab810 con 0x7f7f000d2910 2023-11-18T20:43:04.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.668+0000 7f7ef7fff640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7ee006ae40 0x7f7ee006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.669+0000 7f7ef7fff640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7ee006ae40 0x7f7ee006d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f000c9ca0 tx=0x7f7ee8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.673+0000 7f7ef6ffd640 1 -- 172.21.15.60:0/3632990966 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7ef0076840 con 0x7f7f000d2910 2023-11-18T20:43:04.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.848+0000 7f7f04f33640 1 -- 172.21.15.60:0/3632990966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f00069400 con 0x7f7f000d2910 2023-11-18T20:43:04.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.849+0000 7f7ef6ffd640 1 -- 172.21.15.60:0/3632990966 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ef0079e60 con 0x7f7f000d2910 2023-11-18T20:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.850+0000 7f7f04f33640 1 -- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7ee006ae40 msgr2=0x7f7ee006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.850+0000 7f7f04f33640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7ee006ae40 0x7f7ee006d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f000c9ca0 tx=0x7f7ee8009040 comp rx=0 tx=0).stop 2023-11-18T20:43:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.850+0000 7f7f04f33640 1 -- 172.21.15.60:0/3632990966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f000d2910 msgr2=0x7f7f001f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.850+0000 7f7f04f33640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f000d2910 0x7f7f001f3f40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009890 tx=0x7f7ef00046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.850+0000 7f7f04f33640 1 -- 172.21.15.60:0/3632990966 shutdown_connections 2023-11-18T20:43:04.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.850+0000 7f7f04f33640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7ee006ae40 0x7f7ee006d300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.850+0000 7f7f04f33640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f0015d330 0x7f7f000cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.850+0000 7f7f04f33640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f000d3900 0x7f7f0014b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.850+0000 7f7f04f33640 1 --2- 172.21.15.60:0/3632990966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f000d2910 0x7f7f001f3f40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.850+0000 7f7f04f33640 1 -- 172.21.15.60:0/3632990966 >> 172.21.15.60:0/3632990966 conn(0x7f7f00072020 msgr2=0x7f7f00072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:04.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.851+0000 7f7f04f33640 1 -- 172.21.15.60:0/3632990966 shutdown_connections 2023-11-18T20:43:04.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.851+0000 7f7f04f33640 1 -- 172.21.15.60:0/3632990966 wait complete. 2023-11-18T20:43:04.863 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:04.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.971+0000 7f7b2c6c9640 1 Processor -- start 2023-11-18T20:43:04.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.971+0000 7f7b2c6c9640 1 -- start start 2023-11-18T20:43:04.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.972+0000 7f7b2c6c9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b240b1400 0x7f7b2415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.972+0000 7f7b2c6c9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2415e2a0 con 0x7f7b24143490 2023-11-18T20:43:04.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.972+0000 7f7b2c6c9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2415b240 con 0x7f7b2415e730 2023-11-18T20:43:04.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.972+0000 7f7b2c6c9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2415b520 con 0x7f7b240b1400 2023-11-18T20:43:04.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.972+0000 7f7b2a465640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7b24143490 0x7f7b240b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50092/0 (socket says 172.21.15.60:50092) 2023-11-18T20:43:04.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.972+0000 7f7b2a465640 1 -- 172.21.15.60:0/603762208 learned_addr learned my addr 172.21.15.60:0/603762208 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:04.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.972+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264979395 0 0) 0x7f7b2415e2a0 con 0x7f7b24143490 2023-11-18T20:43:04.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.972+0000 7f7b29c64640 1 --2- 172.21.15.60:0/603762208 >> v2:172.21.15.100:3300/0 conn(0x7f7b240b1400 0x7f7b2415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.972+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b0c003520 con 0x7f7b24143490 2023-11-18T20:43:04.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1213024405 0 0) 0x7f7b0c003520 con 0x7f7b24143490 2023-11-18T20:43:04.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 >> v2:172.21.15.100:3300/0 conn(0x7f7b240b1400 msgr2=0x7f7b2415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:04.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 --2- 172.21.15.60:0/603762208 >> v2:172.21.15.100:3300/0 conn(0x7f7b240b1400 0x7f7b2415a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 >> v1:172.21.15.100:6789/0 conn(0x7f7b2415e730 legacy=0x7f7b2415aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2415dc50 con 0x7f7b24143490 2023-11-18T20:43:04.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f7b1c001670 con 0x7f7b24143490 2023-11-18T20:43:04.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 >> v1:172.21.15.60:6789/0 conn(0x7f7b24143490 legacy=0x7f7b240b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0c003970 0x7f7b0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b0c005180 0x7f7b0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2415b520 con 0x7f7b0c0047c0 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c00c020 con 0x7f7b0c005180 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c00c1e0 con 0x7f7b0c003970 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29c64640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b2ac66640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b0c005180 0x7f7b0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b2a465640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0c003970 0x7f7b0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29c64640 1 -- 172.21.15.60:0/603762208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0c003970 msgr2=0x7f7b0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29c64640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b0c003970 0x7f7b0c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29c64640 1 -- 172.21.15.60:0/603762208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b0c005180 msgr2=0x7f7b0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:04.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29c64640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b0c005180 0x7f7b0c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b29c64640 1 -- 172.21.15.60:0/603762208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c003520 con 0x7f7b0c0047c0 2023-11-18T20:43:04.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.973+0000 7f7b2ac66640 1 -- 172.21.15.60:0/603762208 reap_dead start 2023-11-18T20:43:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.974+0000 7f7b29c64640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b0c004c40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009ac0 tx=0x7f7b0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26775a3031e34e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.974+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1802c6b0 con 0x7f7b0c0047c0 2023-11-18T20:43:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.974+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1802c810 con 0x7f7b0c0047c0 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.974+0000 7f7b29463640 1 -- 172.21.15.60:0/603762208 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1802cba0 con 0x7f7b0c0047c0 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.975+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/603762208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 msgr2=0x7f7b0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.975+0000 7f7b2c6c9640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b0c004c40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009ac0 tx=0x7f7b0c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.975+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/603762208 shutdown_connections 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.975+0000 7f7b2c6c9640 1 --2- 172.21.15.60:0/603762208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b0c004c40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.975+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/603762208 >> 172.21.15.60:0/603762208 conn(0x7f7b2406a4d0 msgr2=0x7f7b2406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.975+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/603762208 shutdown_connections 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.975+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/603762208 wait complete. 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.976+0000 7f7b2c6c9640 1 Processor -- start 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.976+0000 7f7b2c6c9640 1 -- start start 2023-11-18T20:43:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2c6c9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b24141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2c6c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b240ad6a0 0x7f7b240adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2c6c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b24134350 0x7f7b241347d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2c6c9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240a84d0 con 0x7f7b0c0047c0 2023-11-18T20:43:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2c6c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b24134d10 con 0x7f7b240ad6a0 2023-11-18T20:43:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2c6c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240ae1d0 con 0x7f7b24134350 2023-11-18T20:43:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2a465640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b24141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2a465640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b24141430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42566/0 (socket says 172.21.15.60:42566) 2023-11-18T20:43:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2a465640 1 -- 172.21.15.60:0/271764403 learned_addr learned my addr 172.21.15.60:0/271764403 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:04.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2ac66640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b24134350 0x7f7b241347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b29c64640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b240ad6a0 0x7f7b240adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2a465640 1 -- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b24134350 msgr2=0x7f7b241347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2a465640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b24134350 0x7f7b241347d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2a465640 1 -- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b240ad6a0 msgr2=0x7f7b240adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2a465640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b240ad6a0 0x7f7b240adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2a465640 1 -- 172.21.15.60:0/271764403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b240a8720 con 0x7f7b0c0047c0 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b2a465640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b24141430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c007290 tx=0x7f7b1c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.977+0000 7f7b22ffd640 1 -- 172.21.15.60:0/271764403 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c001500 con 0x7f7b0c0047c0 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.978+0000 7f7b22ffd640 1 -- 172.21.15.60:0/271764403 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c00f9c0 con 0x7f7b0c0047c0 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.978+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/271764403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b240a8a00 con 0x7f7b0c0047c0 2023-11-18T20:43:04.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.978+0000 7f7b22ffd640 1 -- 172.21.15.60:0/271764403 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c00fd50 con 0x7f7b0c0047c0 2023-11-18T20:43:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.978+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/271764403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2406d120 con 0x7f7b0c0047c0 2023-11-18T20:43:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.979+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/271764403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b240b4380 con 0x7f7b0c0047c0 2023-11-18T20:43:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.979+0000 7f7b22ffd640 1 -- 172.21.15.60:0/271764403 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b1c01d460 con 0x7f7b0c0047c0 2023-11-18T20:43:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.979+0000 7f7b22ffd640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b1006ae40 0x7f7b1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.980+0000 7f7b22ffd640 1 -- 172.21.15.60:0/271764403 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b1c0895e0 con 0x7f7b0c0047c0 2023-11-18T20:43:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.980+0000 7f7b29c64640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b1006ae40 0x7f7b1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:04.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.980+0000 7f7b29c64640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b1006ae40 0x7f7b1006d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b18036000 tx=0x7f7b18030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:04.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:04.984+0000 7f7b22ffd640 1 -- 172.21.15.60:0/271764403 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b1c0547d0 con 0x7f7b0c0047c0 2023-11-18T20:43:05.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.159+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/271764403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b240a8f20 con 0x7f7b0c0047c0 2023-11-18T20:43:05.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.159+0000 7f7b22ffd640 1 -- 172.21.15.60:0/271764403 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b240a8f20 con 0x7f7b0c0047c0 2023-11-18T20:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.161+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b1006ae40 msgr2=0x7f7b1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.161+0000 7f7b2c6c9640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b1006ae40 0x7f7b1006d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b18036000 tx=0x7f7b18030000 comp rx=0 tx=0).stop 2023-11-18T20:43:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.161+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/271764403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 msgr2=0x7f7b24141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.161+0000 7f7b2c6c9640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b24141430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c007290 tx=0x7f7b1c007750 comp rx=0 tx=0).stop 2023-11-18T20:43:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.161+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/271764403 shutdown_connections 2023-11-18T20:43:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.161+0000 7f7b2c6c9640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b1006ae40 0x7f7b1006d300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:05.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.161+0000 7f7b2c6c9640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b24134350 0x7f7b241347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.161+0000 7f7b2c6c9640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b240ad6a0 0x7f7b240adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.161+0000 7f7b2c6c9640 1 --2- 172.21.15.60:0/271764403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0c0047c0 0x7f7b24141430 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.161+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/271764403 >> 172.21.15.60:0/271764403 conn(0x7f7b2406a4d0 msgr2=0x7f7b240ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.162+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/271764403 shutdown_connections 2023-11-18T20:43:05.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.162+0000 7f7b2c6c9640 1 -- 172.21.15.60:0/271764403 wait complete. 2023-11-18T20:43:05.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-11-18T20:43:05.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.279+0000 7f6a83568640 1 Processor -- start 2023-11-18T20:43:05.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.279+0000 7f6a83568640 1 -- start start 2023-11-18T20:43:05.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.280+0000 7f6a83568640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a7c15d2b0 0x7f6a7c071c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:05.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.280+0000 7f6a83568640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6a7c072840 0x7f6a7c0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:05.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.280+0000 7f6a83568640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0ae910 con 0x7f6a7c0ae480 2023-11-18T20:43:05.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.280+0000 7f6a83568640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b2200 con 0x7f6a7c15d2b0 2023-11-18T20:43:05.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.280+0000 7f6a83568640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0b24e0 con 0x7f6a7c072840 2023-11-18T20:43:05.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.280+0000 7f6a81304640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a7c15d2b0 0x7f6a7c071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:05.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.280+0000 7f6a81304640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a7c15d2b0 0x7f6a7c071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42570/0 (socket says 172.21.15.60:42570) 2023-11-18T20:43:05.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.280+0000 7f6a81304640 1 -- 172.21.15.60:0/2266151105 learned_addr learned my addr 172.21.15.60:0/2266151105 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:05.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.280+0000 7f6a80b03640 1 --1- 172.21.15.60:0/2266151105 >> v1:172.21.15.60:6789/0 conn(0x7f6a7c0ae480 0x7f6a7c06d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50104/0 (socket says 172.21.15.60:50104) 2023-11-18T20:43:05.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.280+0000 7f6a81b05640 1 --2- 172.21.15.60:0/2266151105 >> v2:172.21.15.100:3300/0 conn(0x7f6a7c072840 0x7f6a7c0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:05.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.281+0000 7f6a81304640 1 -- 172.21.15.60:0/2266151105 >> v2:172.21.15.100:3300/0 conn(0x7f6a7c072840 msgr2=0x7f6a7c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:05.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.281+0000 7f6a81304640 1 --2- 172.21.15.60:0/2266151105 >> v2:172.21.15.100:3300/0 conn(0x7f6a7c072840 0x7f6a7c0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:05.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.281+0000 7f6a81304640 1 -- 172.21.15.60:0/2266151105 >> v1:172.21.15.60:6789/0 conn(0x7f6a7c0ae480 legacy=0x7f6a7c06d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:05.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.281+0000 7f6a7bfff640 1 -- 172.21.15.60:0/2266151105 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372444904 0 0) 0x7f6a7c0ae910 con 0x7f6a7c0ae480 2023-11-18T20:43:05.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.281+0000 7f6a81304640 1 -- 172.21.15.60:0/2266151105 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0b40a0 con 0x7f6a7c15d2b0 2023-11-18T20:43:05.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.281+0000 7f6a81304640 1 --2- 172.21.15.60:0/2266151105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c15d2b0 0x7f6a7c071c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0cfcf0 tx=0x7f6a7402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2afb53ef0c0225 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:05.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.281+0000 7f6a7bfff640 1 -- 172.21.15.60:0/2266151105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74031870 con 0x7f6a7c15d2b0 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.281+0000 7f6a7bfff640 1 -- 172.21.15.60:0/2266151105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a740319d0 con 0x7f6a7c15d2b0 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.281+0000 7f6a7bfff640 1 -- 172.21.15.60:0/2266151105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74031d60 con 0x7f6a7c15d2b0 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.282+0000 7f6a83568640 1 -- 172.21.15.60:0/2266151105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c15d2b0 msgr2=0x7f6a7c071c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.282+0000 7f6a83568640 1 --2- 172.21.15.60:0/2266151105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c15d2b0 0x7f6a7c071c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0cfcf0 tx=0x7f6a7402b910 comp rx=0 tx=0).stop 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.282+0000 7f6a83568640 1 -- 172.21.15.60:0/2266151105 shutdown_connections 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.282+0000 7f6a83568640 1 --2- 172.21.15.60:0/2266151105 >> v2:172.21.15.100:3300/0 conn(0x7f6a7c072840 0x7f6a7c0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.282+0000 7f6a83568640 1 --2- 172.21.15.60:0/2266151105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c15d2b0 0x7f6a7c071c80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.282+0000 7f6a83568640 1 -- 172.21.15.60:0/2266151105 >> 172.21.15.60:0/2266151105 conn(0x7f6a7c0b4720 msgr2=0x7f6a7c0b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.282+0000 7f6a83568640 1 -- 172.21.15.60:0/2266151105 shutdown_connections 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.282+0000 7f6a83568640 1 -- 172.21.15.60:0/2266151105 wait complete. 2023-11-18T20:43:05.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.283+0000 7f6a83568640 1 Processor -- start 2023-11-18T20:43:05.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.283+0000 7f6a83568640 1 -- start start 2023-11-18T20:43:05.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a83568640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c072840 0x7f6a7c099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:05.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a83568640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a7c15d2b0 0x7f6a7c144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:05.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a83568640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a7c1f3480 0x7f6a7c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:05.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a83568640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c158c90 con 0x7f6a7c072840 2023-11-18T20:43:05.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a83568640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c159af0 con 0x7f6a7c15d2b0 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a83568640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c099e60 con 0x7f6a7c1f3480 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a81304640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c072840 0x7f6a7c099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a81304640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c072840 0x7f6a7c099920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42572/0 (socket says 172.21.15.60:42572) 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a81304640 1 -- 172.21.15.60:0/649664579 learned_addr learned my addr 172.21.15.60:0/649664579 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a80b03640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a7c15d2b0 0x7f6a7c144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a81b05640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a7c1f3480 0x7f6a7c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a81304640 1 -- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a7c1f3480 msgr2=0x7f6a7c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a81304640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a7c1f3480 0x7f6a7c0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a81304640 1 -- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a7c15d2b0 msgr2=0x7f6a7c144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a81304640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a7c15d2b0 0x7f6a7c144ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.284+0000 7f6a81304640 1 -- 172.21.15.60:0/649664579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a740094d0 con 0x7f6a7c072840 2023-11-18T20:43:05.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.285+0000 7f6a81304640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c072840 0x7f6a7c099920 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a740090b0 tx=0x7f6a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:05.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.285+0000 7f6a79ffb640 1 -- 172.21.15.60:0/649664579 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74004230 con 0x7f6a7c072840 2023-11-18T20:43:05.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.285+0000 7f6a79ffb640 1 -- 172.21.15.60:0/649664579 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a740043b0 con 0x7f6a7c072840 2023-11-18T20:43:05.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.285+0000 7f6a83568640 1 -- 172.21.15.60:0/649664579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c0b2af0 con 0x7f6a7c072840 2023-11-18T20:43:05.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.285+0000 7f6a79ffb640 1 -- 172.21.15.60:0/649664579 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74004520 con 0x7f6a7c072840 2023-11-18T20:43:05.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.285+0000 7f6a83568640 1 -- 172.21.15.60:0/649664579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c1591c0 con 0x7f6a7c072840 2023-11-18T20:43:05.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.286+0000 7f6a83568640 1 -- 172.21.15.60:0/649664579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c069400 con 0x7f6a7c072840 2023-11-18T20:43:05.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.287+0000 7f6a79ffb640 1 -- 172.21.15.60:0/649664579 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a74042070 con 0x7f6a7c072840 2023-11-18T20:43:05.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.287+0000 7f6a79ffb640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6a6806ae40 0x7f6a6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:05.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.287+0000 7f6a79ffb640 1 -- 172.21.15.60:0/649664579 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a7402d040 con 0x7f6a7c072840 2023-11-18T20:43:05.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.287+0000 7f6a80b03640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6a6806ae40 0x7f6a6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:05.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.288+0000 7f6a80b03640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6a6806ae40 0x7f6a6806d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c156490 tx=0x7f6a70007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:05.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.291+0000 7f6a79ffb640 1 -- 172.21.15.60:0/649664579 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a740767b0 con 0x7f6a7c072840 2023-11-18T20:43:05.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:05.542+0000 7f6a83568640 1 -- 172.21.15.60:0/649664579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f6a7c06b870 con 0x7f6a7c072840 2023-11-18T20:43:06.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.188+0000 7f6a79ffb640 1 -- 172.21.15.60:0/649664579 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v50) v1 ==== 150+0+0 (secure 0 0 0) 0x7f6a740350d0 con 0x7f6a7c072840 2023-11-18T20:43:06.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 -- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6a6806ae40 msgr2=0x7f6a6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6a6806ae40 0x7f6a6806d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c156490 tx=0x7f6a70007d50 comp rx=0 tx=0).stop 2023-11-18T20:43:06.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 -- 172.21.15.60:0/649664579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c072840 msgr2=0x7f6a7c099920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c072840 0x7f6a7c099920 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a740090b0 tx=0x7f6a740046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 -- 172.21.15.60:0/649664579 shutdown_connections 2023-11-18T20:43:06.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6a6806ae40 0x7f6a6806d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a7c1f3480 0x7f6a7c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a7c15d2b0 0x7f6a7c144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 --2- 172.21.15.60:0/649664579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a7c072840 0x7f6a7c099920 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 -- 172.21.15.60:0/649664579 >> 172.21.15.60:0/649664579 conn(0x7f6a7c0b4720 msgr2=0x7f6a7c15df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:06.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 -- 172.21.15.60:0/649664579 shutdown_connections 2023-11-18T20:43:06.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.191+0000 7f6a83568640 1 -- 172.21.15.60:0/649664579 wait complete. 2023-11-18T20:43:06.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-18T20:43:06.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.320+0000 7fc9d4301640 1 Processor -- start 2023-11-18T20:43:06.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.320+0000 7fc9d4301640 1 -- start start 2023-11-18T20:43:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.321+0000 7fc9d4301640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc9cc06ea50 0x7fc9cc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.321+0000 7fc9d4301640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc161d20 con 0x7fc9cc160fb0 2023-11-18T20:43:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.321+0000 7fc9d4301640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc06f370 con 0x7fc9cc0d0ed0 2023-11-18T20:43:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.321+0000 7fc9d4301640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9cc15e760 con 0x7fc9cc06ea50 2023-11-18T20:43:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.321+0000 7fc9d189c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc9cc160fb0 0x7fc9cc161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50114/0 (socket says 172.21.15.60:50114) 2023-11-18T20:43:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.321+0000 7fc9d189c640 1 -- 172.21.15.60:0/2623395952 learned_addr learned my addr 172.21.15.60:0/2623395952 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d209d640 1 --2- 172.21.15.60:0/2623395952 >> v2:172.21.15.100:3300/0 conn(0x7fc9cc06ea50 0x7fc9cc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431512769 0 0) 0x7fc9cc161d20 con 0x7fc9cc160fb0 2023-11-18T20:43:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b8003520 con 0x7fc9cc160fb0 2023-11-18T20:43:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3236268410 0 0) 0x7fc9b8003520 con 0x7fc9cc160fb0 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 >> v2:172.21.15.100:3300/0 conn(0x7fc9cc06ea50 msgr2=0x7fc9cc06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 --2- 172.21.15.60:0/2623395952 >> v2:172.21.15.100:3300/0 conn(0x7fc9cc06ea50 0x7fc9cc06ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 >> v1:172.21.15.100:6789/0 conn(0x7fc9cc0d0ed0 legacy=0x7fc9cc0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc15d9a0 con 0x7fc9cc160fb0 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fc9bc001670 con 0x7fc9cc160fb0 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d209d640 1 --2- 172.21.15.60:0/2623395952 >> v2:172.21.15.100:3300/0 conn(0x7fc9cc06ea50 0x7fc9cc06ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 >> v1:172.21.15.60:6789/0 conn(0x7fc9cc160fb0 legacy=0x7fc9cc161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b8003970 0x7fc9b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b80047c0 0x7fc9b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9b8005180 0x7fc9b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc15e760 con 0x7fc9b80047c0 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b800c120 con 0x7fc9b8003970 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b800c2e0 con 0x7fc9b8005180 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d189c640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b80047c0 0x7fc9b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d209d640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b8003970 0x7fc9b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d289e640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9b8005180 0x7fc9b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d189c640 1 -- 172.21.15.60:0/2623395952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9b8005180 msgr2=0x7fc9b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d189c640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9b8005180 0x7fc9b800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d189c640 1 -- 172.21.15.60:0/2623395952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b8003970 msgr2=0x7fc9b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.322+0000 7fc9d189c640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b8003970 0x7fc9b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.323+0000 7fc9d189c640 1 -- 172.21.15.60:0/2623395952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b8003520 con 0x7fc9b80047c0 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.323+0000 7fc9d289e640 1 -- 172.21.15.60:0/2623395952 reap_dead start 2023-11-18T20:43:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.323+0000 7fc9d189c640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b80047c0 0x7fc9b8004c40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00c240 tx=0x7fc9b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fb79b635fc5810 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.323+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc005ad0 con 0x7fc9b80047c0 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.323+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9bc005c30 con 0x7fc9b80047c0 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.323+0000 7fc9d109b640 1 -- 172.21.15.60:0/2623395952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc02cab0 con 0x7fc9b80047c0 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.324+0000 7fc9d4301640 1 -- 172.21.15.60:0/2623395952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b80047c0 msgr2=0x7fc9b8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.324+0000 7fc9d4301640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b80047c0 0x7fc9b8004c40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00c240 tx=0x7fc9b80057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.324+0000 7fc9d4301640 1 -- 172.21.15.60:0/2623395952 shutdown_connections 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.324+0000 7fc9d4301640 1 --2- 172.21.15.60:0/2623395952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b80047c0 0x7fc9b8004c40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.324+0000 7fc9d4301640 1 -- 172.21.15.60:0/2623395952 >> 172.21.15.60:0/2623395952 conn(0x7fc9cc06a6b0 msgr2=0x7fc9cc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.324+0000 7fc9d4301640 1 -- 172.21.15.60:0/2623395952 shutdown_connections 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.324+0000 7fc9d4301640 1 -- 172.21.15.60:0/2623395952 wait complete. 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.325+0000 7fc9d4301640 1 Processor -- start 2023-11-18T20:43:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.325+0000 7fc9d4301640 1 -- start start 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d4301640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b80047c0 0x7fc9cc135940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d4301640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9cc0cb240 0x7fc9cc15a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d4301640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc15aac0 0x7fc9cc0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d4301640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc15af40 con 0x7fc9cc15aac0 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d4301640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc135e80 con 0x7fc9b80047c0 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d4301640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc13fb70 con 0x7fc9cc0cb240 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d289e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc15aac0 0x7fc9cc0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d289e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc15aac0 0x7fc9cc0b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42576/0 (socket says 172.21.15.60:42576) 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d289e640 1 -- 172.21.15.60:0/113184 learned_addr learned my addr 172.21.15.60:0/113184 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d209d640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b80047c0 0x7fc9cc135940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d189c640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9cc0cb240 0x7fc9cc15a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d289e640 1 -- 172.21.15.60:0/113184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9cc0cb240 msgr2=0x7fc9cc15a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d289e640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9cc0cb240 0x7fc9cc15a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d289e640 1 -- 172.21.15.60:0/113184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b80047c0 msgr2=0x7fc9cc135940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d289e640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b80047c0 0x7fc9cc135940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.326+0000 7fc9d289e640 1 -- 172.21.15.60:0/113184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc15b140 con 0x7fc9cc15aac0 2023-11-18T20:43:06.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.327+0000 7fc9d289e640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc15aac0 0x7fc9cc0b1d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc1407c0 tx=0x7fc9c800fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:06.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.327+0000 7fc9c2ffd640 1 -- 172.21.15.60:0/113184 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c800fe60 con 0x7fc9cc15aac0 2023-11-18T20:43:06.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.327+0000 7fc9c2ffd640 1 -- 172.21.15.60:0/113184 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c8006d60 con 0x7fc9cc15aac0 2023-11-18T20:43:06.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.327+0000 7fc9d4301640 1 -- 172.21.15.60:0/113184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc0ca480 con 0x7fc9cc15aac0 2023-11-18T20:43:06.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.327+0000 7fc9c2ffd640 1 -- 172.21.15.60:0/113184 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8003bb0 con 0x7fc9cc15aac0 2023-11-18T20:43:06.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.327+0000 7fc9d4301640 1 -- 172.21.15.60:0/113184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc0caa40 con 0x7fc9cc15aac0 2023-11-18T20:43:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.328+0000 7fc9d4301640 1 -- 172.21.15.60:0/113184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc069400 con 0x7fc9cc15aac0 2023-11-18T20:43:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.328+0000 7fc9c2ffd640 1 -- 172.21.15.60:0/113184 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9c8003d10 con 0x7fc9cc15aac0 2023-11-18T20:43:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.329+0000 7fc9c2ffd640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9b006ae40 0x7fc9b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.329+0000 7fc9c2ffd640 1 -- 172.21.15.60:0/113184 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9c8088cf0 con 0x7fc9cc15aac0 2023-11-18T20:43:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.329+0000 7fc9d209d640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9b006ae40 0x7fc9b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.329+0000 7fc9d209d640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9b006ae40 0x7fc9b006d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc050020 tx=0x7fc9c4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.333+0000 7fc9c2ffd640 1 -- 172.21.15.60:0/113184 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9c808d070 con 0x7fc9cc15aac0 2023-11-18T20:43:06.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.511+0000 7fc9d4301640 1 -- 172.21.15.60:0/113184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc9cc070d30 con 0x7fc9cc15aac0 2023-11-18T20:43:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.514+0000 7fc9c2ffd640 1 -- 172.21.15.60:0/113184 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc9c801a0c0 con 0x7fc9cc15aac0 2023-11-18T20:43:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.516+0000 7fc9d4301640 1 -- 172.21.15.60:0/113184 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9b006ae40 msgr2=0x7fc9b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.516+0000 7fc9d4301640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9b006ae40 0x7fc9b006d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc050020 tx=0x7fc9c4008040 comp rx=0 tx=0).stop 2023-11-18T20:43:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.516+0000 7fc9d4301640 1 -- 172.21.15.60:0/113184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc15aac0 msgr2=0x7fc9cc0b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.516+0000 7fc9d4301640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc15aac0 0x7fc9cc0b1d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc1407c0 tx=0x7fc9c800fc70 comp rx=0 tx=0).stop 2023-11-18T20:43:06.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.517+0000 7fc9d4301640 1 -- 172.21.15.60:0/113184 shutdown_connections 2023-11-18T20:43:06.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.517+0000 7fc9d4301640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc9b006ae40 0x7fc9b006d300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.517+0000 7fc9d4301640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9cc15aac0 0x7fc9cc0b1d70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.517+0000 7fc9d4301640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9cc0cb240 0x7fc9cc15a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.517+0000 7fc9d4301640 1 --2- 172.21.15.60:0/113184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9b80047c0 0x7fc9cc135940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.517+0000 7fc9d4301640 1 -- 172.21.15.60:0/113184 >> 172.21.15.60:0/113184 conn(0x7fc9cc06a6b0 msgr2=0x7fc9cc0b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.517+0000 7fc9d4301640 1 -- 172.21.15.60:0/113184 shutdown_connections 2023-11-18T20:43:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.517+0000 7fc9d4301640 1 -- 172.21.15.60:0/113184 wait complete. 2023-11-18T20:43:06.530 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-18T20:43:06.530 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-18T20:43:06.576 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-18T20:43:06.631 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-18T20:43:06.631 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:43:06.631 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:06.647 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:43:06.647 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:06.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:06.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.846+0000 7f69d2ae7640 1 Processor -- start 2023-11-18T20:43:06.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.846+0000 7f69d2ae7640 1 -- start start 2023-11-18T20:43:06.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.847+0000 7f69d2ae7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc15d330 0x7f69cc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.847+0000 7f69d2ae7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69cc0d2d90 0x7f69cc0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.847+0000 7f69d2ae7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69cc0b2b10 con 0x7f69cc0d2910 2023-11-18T20:43:06.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.847+0000 7f69d2ae7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69cc0d1590 con 0x7f69cc0d2d90 2023-11-18T20:43:06.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.847+0000 7f69d2ae7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0d1870 con 0x7f69cc15d330 2023-11-18T20:43:06.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.848+0000 7f69d1084640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69cc0d2d90 0x7f69cc0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.848+0000 7f69d0883640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc15d330 0x7f69cc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.848+0000 7f69cbfff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f69cc0d2910 0x7f69cc0d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:39040/0 (socket says 172.21.15.60:39040) 2023-11-18T20:43:06.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.848+0000 7f69d1084640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69cc0d2d90 0x7f69cc0b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33978/0 (socket says 172.21.15.60:33978) 2023-11-18T20:43:06.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.848+0000 7f69cbfff640 1 -- 172.21.15.60:0/3713351259 learned_addr learned my addr 172.21.15.60:0/3713351259 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:06.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.848+0000 7f69d1084640 1 -- 172.21.15.60:0/3713351259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc15d330 msgr2=0x7f69cc0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:06.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.848+0000 7f69d1084640 1 --2- 172.21.15.60:0/3713351259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc15d330 0x7f69cc0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.848+0000 7f69d1084640 1 -- 172.21.15.60:0/3713351259 >> v1:172.21.15.100:6789/0 conn(0x7f69cc0d2910 legacy=0x7f69cc0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.848+0000 7f69d1084640 1 -- 172.21.15.60:0/3713351259 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc0d1b50 con 0x7f69cc0d2d90 2023-11-18T20:43:06.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.848+0000 7f69d0883640 1 --2- 172.21.15.60:0/3713351259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc15d330 0x7f69cc0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:06.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.849+0000 7f69d1084640 1 --2- 172.21.15.60:0/3713351259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc0d2d90 0x7f69cc0b6390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0097a0 tx=0x7f69bc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4c9dd436eb26b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:06.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.849+0000 7f69cb7fe640 1 -- 172.21.15.60:0/3713351259 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc02e070 con 0x7f69cc0d2d90 2023-11-18T20:43:06.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.849+0000 7f69cb7fe640 1 -- 172.21.15.60:0/3713351259 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69bc0315a0 con 0x7f69cc0d2d90 2023-11-18T20:43:06.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.849+0000 7f69cb7fe640 1 -- 172.21.15.60:0/3713351259 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc031930 con 0x7f69cc0d2d90 2023-11-18T20:43:06.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.849+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3713351259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc0d2d90 msgr2=0x7f69cc0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.849+0000 7f69d2ae7640 1 --2- 172.21.15.60:0/3713351259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc0d2d90 0x7f69cc0b6390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0097a0 tx=0x7f69bc02b840 comp rx=0 tx=0).stop 2023-11-18T20:43:06.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.850+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3713351259 shutdown_connections 2023-11-18T20:43:06.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.850+0000 7f69d2ae7640 1 --2- 172.21.15.60:0/3713351259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc0d2d90 0x7f69cc0b6390 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.850+0000 7f69d2ae7640 1 --2- 172.21.15.60:0/3713351259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc15d330 0x7f69cc0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.850+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3713351259 >> 172.21.15.60:0/3713351259 conn(0x7f69cc072020 msgr2=0x7f69cc072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.850+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3713351259 shutdown_connections 2023-11-18T20:43:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.850+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3713351259 wait complete. 2023-11-18T20:43:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.851+0000 7f69d2ae7640 1 Processor -- start 2023-11-18T20:43:06.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.851+0000 7f69d2ae7640 1 -- start start 2023-11-18T20:43:06.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d2ae7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc0d2910 0x7f69cc15fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d2ae7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc0d2d90 0x7f69cc0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d2ae7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc15d330 0x7f69cc0c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d2ae7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0c6ba0 con 0x7f69cc0d2910 2023-11-18T20:43:06.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d2ae7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc1550b0 con 0x7f69cc15d330 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d2ae7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc1604e0 con 0x7f69cc0d2d90 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d0883640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc0d2910 0x7f69cc15fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d0883640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc0d2910 0x7f69cc15fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42588/0 (socket says 172.21.15.60:42588) 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d0883640 1 -- 172.21.15.60:0/3721365842 learned_addr learned my addr 172.21.15.60:0/3721365842 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69cbfff640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc0d2d90 0x7f69cc0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d1084640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc15d330 0x7f69cc0c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d0883640 1 -- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc0d2d90 msgr2=0x7f69cc0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d0883640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc0d2d90 0x7f69cc0ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d0883640 1 -- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc15d330 msgr2=0x7f69cc0c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d0883640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc15d330 0x7f69cc0c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d0883640 1 -- 172.21.15.60:0/3721365842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c4009a70 con 0x7f69cc0d2910 2023-11-18T20:43:06.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d0883640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc0d2910 0x7f69cc15fe50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009f60 tx=0x7f69c4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:06.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69c97fa640 1 -- 172.21.15.60:0/3721365842 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4014680 con 0x7f69cc0d2910 2023-11-18T20:43:06.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69c97fa640 1 -- 172.21.15.60:0/3721365842 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c400f040 con 0x7f69cc0d2910 2023-11-18T20:43:06.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.852+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3721365842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69bc009400 con 0x7f69cc0d2910 2023-11-18T20:43:06.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.853+0000 7f69c97fa640 1 -- 172.21.15.60:0/3721365842 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4014a00 con 0x7f69cc0d2910 2023-11-18T20:43:06.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.853+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3721365842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc0c6df0 con 0x7f69cc0d2910 2023-11-18T20:43:06.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.853+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3721365842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69cc069400 con 0x7f69cc0d2910 2023-11-18T20:43:06.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.854+0000 7f69c97fa640 1 -- 172.21.15.60:0/3721365842 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69c4014b60 con 0x7f69cc0d2910 2023-11-18T20:43:06.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.854+0000 7f69c97fa640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69b806ae40 0x7f69b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:06.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.854+0000 7f69c97fa640 1 -- 172.21.15.60:0/3721365842 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69c408a550 con 0x7f69cc0d2910 2023-11-18T20:43:06.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.855+0000 7f69cbfff640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69b806ae40 0x7f69b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:06.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.855+0000 7f69cbfff640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69b806ae40 0x7f69b806d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0c66f0 tx=0x7f69c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:06.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:06.859+0000 7f69c97fa640 1 -- 172.21.15.60:0/3721365842 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69c40556c0 con 0x7f69cc0d2910 2023-11-18T20:43:07.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.038+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3721365842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69cc0ca870 con 0x7f69cc0d2910 2023-11-18T20:43:07.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.039+0000 7f69c97fa640 1 -- 172.21.15.60:0/3721365842 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f69c4058ce0 con 0x7f69cc0d2910 2023-11-18T20:43:07.041 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:07.042 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:07.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69b806ae40 msgr2=0x7f69b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69b806ae40 0x7f69b806d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0c66f0 tx=0x7f69c0009040 comp rx=0 tx=0).stop 2023-11-18T20:43:07.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3721365842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc0d2910 msgr2=0x7f69cc15fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc0d2910 0x7f69cc15fe50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009f60 tx=0x7f69c4009f90 comp rx=0 tx=0).stop 2023-11-18T20:43:07.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3721365842 shutdown_connections 2023-11-18T20:43:07.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f69b806ae40 0x7f69b806d300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc15d330 0x7f69cc0c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc0d2d90 0x7f69cc0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 --2- 172.21.15.60:0/3721365842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc0d2910 0x7f69cc15fe50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3721365842 >> 172.21.15.60:0/3721365842 conn(0x7f69cc072020 msgr2=0x7f69cc0b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:07.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3721365842 shutdown_connections 2023-11-18T20:43:07.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.041+0000 7f69d2ae7640 1 -- 172.21.15.60:0/3721365842 wait complete. 2023-11-18T20:43:07.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:07.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:07.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.161+0000 7ff9f058a640 1 Processor -- start 2023-11-18T20:43:07.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.161+0000 7ff9f058a640 1 -- start start 2023-11-18T20:43:07.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.162+0000 7ff9f058a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80d3900 0x7ff9e80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.162+0000 7ff9f058a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e80b2b10 con 0x7ff9e80d2910 2023-11-18T20:43:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.162+0000 7ff9f058a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e80d1590 con 0x7ff9e815d330 2023-11-18T20:43:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.162+0000 7ff9f058a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80d1870 con 0x7ff9e80d3900 2023-11-18T20:43:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9edb25640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff9e80d2910 0x7ff9e80d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50126/0 (socket says 172.21.15.60:50126) 2023-11-18T20:43:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9edb25640 1 -- 172.21.15.60:0/1336975901 learned_addr learned my addr 172.21.15.60:0/1336975901 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9eeb27640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80d3900 0x7ff9e80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117477018 0 0) 0x7ff9e80b2b10 con 0x7ff9e80d2910 2023-11-18T20:43:07.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d0003520 con 0x7ff9e80d2910 2023-11-18T20:43:07.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705554710 0 0) 0x7ff9e80d1590 con 0x7ff9e815d330 2023-11-18T20:43:07.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e80b2b10 con 0x7ff9e815d330 2023-11-18T20:43:07.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1615578413 0 0) 0x7ff9d0003520 con 0x7ff9e80d2910 2023-11-18T20:43:07.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80d3900 msgr2=0x7ff9e80b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:07.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80d3900 0x7ff9e80b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 >> v1:172.21.15.100:6789/0 conn(0x7ff9e815d330 legacy=0x7ff9e80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e80d1b50 con 0x7ff9e80d2910 2023-11-18T20:43:07.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7ff9dc001670 con 0x7ff9e80d2910 2023-11-18T20:43:07.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9eeb27640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80d3900 0x7ff9e80b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:07.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 >> v1:172.21.15.60:6789/0 conn(0x7ff9e80d2910 legacy=0x7ff9e80d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80d3900 0x7ff9d0003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d0004070 0x7ff9d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d0004d00 0x7ff9d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80d1870 con 0x7ff9d0004d00 2023-11-18T20:43:07.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d000bac0 con 0x7ff9d0004070 2023-11-18T20:43:07.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d000bc80 con 0x7ff9e80d3900 2023-11-18T20:43:07.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9eeb27640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d0004d00 0x7ff9d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9edb25640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d0004070 0x7ff9d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.163+0000 7ff9ee326640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80d3900 0x7ff9d0003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.164+0000 7ff9eeb27640 1 -- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80d3900 msgr2=0x7ff9d0003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:07.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.164+0000 7ff9eeb27640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80d3900 0x7ff9d0003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.164+0000 7ff9eeb27640 1 -- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d0004070 msgr2=0x7ff9d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:07.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.164+0000 7ff9eeb27640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d0004070 0x7ff9d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.164+0000 7ff9eeb27640 1 -- 172.21.15.60:0/1336975901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0009400 con 0x7ff9d0004d00 2023-11-18T20:43:07.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.164+0000 7ff9eeb27640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d0004d00 0x7ff9d0005180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009c70 tx=0x7ff9d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7473c5506770fa77 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:07.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.165+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e002c910 con 0x7ff9d0004d00 2023-11-18T20:43:07.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.165+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e002ca70 con 0x7ff9d0004d00 2023-11-18T20:43:07.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.165+0000 7ff9ed324640 1 -- 172.21.15.60:0/1336975901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e002ce20 con 0x7ff9d0004d00 2023-11-18T20:43:07.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.165+0000 7ff9f058a640 1 -- 172.21.15.60:0/1336975901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d0004d00 msgr2=0x7ff9d0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.165+0000 7ff9f058a640 1 --2- 172.21.15.60:0/1336975901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9d0004d00 0x7ff9d0005180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009c70 tx=0x7ff9d00057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.165+0000 7ff9eeb27640 1 -- 172.21.15.60:0/1336975901 reap_dead start 2023-11-18T20:43:07.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.165+0000 7ff9f058a640 1 -- 172.21.15.60:0/1336975901 shutdown_connections 2023-11-18T20:43:07.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.165+0000 7ff9f058a640 1 -- 172.21.15.60:0/1336975901 >> 172.21.15.60:0/1336975901 conn(0x7ff9e8072020 msgr2=0x7ff9e8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:07.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.165+0000 7ff9f058a640 1 -- 172.21.15.60:0/1336975901 shutdown_connections 2023-11-18T20:43:07.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.165+0000 7ff9f058a640 1 -- 172.21.15.60:0/1336975901 wait complete. 2023-11-18T20:43:07.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.166+0000 7ff9f058a640 1 Processor -- start 2023-11-18T20:43:07.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.167+0000 7ff9f058a640 1 -- start start 2023-11-18T20:43:07.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.167+0000 7ff9f058a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d0004d00 0x7ff9e80caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.167+0000 7ff9f058a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9e81f3680 0x7ff9e81568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.167+0000 7ff9f058a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80cc6f0 0x7ff9e80ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.167+0000 7ff9f058a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80caa00 con 0x7ff9e81f3680 2023-11-18T20:43:07.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.167+0000 7ff9f058a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80ca440 con 0x7ff9d0004d00 2023-11-18T20:43:07.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.167+0000 7ff9f058a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80ca720 con 0x7ff9e80cc6f0 2023-11-18T20:43:07.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.167+0000 7ff9edb25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9e81f3680 0x7ff9e81568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.167+0000 7ff9edb25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9e81f3680 0x7ff9e81568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42612/0 (socket says 172.21.15.60:42612) 2023-11-18T20:43:07.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.167+0000 7ff9edb25640 1 -- 172.21.15.60:0/1999069345 learned_addr learned my addr 172.21.15.60:0/1999069345 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:07.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9ee326640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d0004d00 0x7ff9e80caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9eeb27640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80cc6f0 0x7ff9e80ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9edb25640 1 -- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80cc6f0 msgr2=0x7ff9e80ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9edb25640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80cc6f0 0x7ff9e80ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9edb25640 1 -- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d0004d00 msgr2=0x7ff9e80caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9edb25640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d0004d00 0x7ff9e80caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9edb25640 1 -- 172.21.15.60:0/1999069345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e80b4140 con 0x7ff9e81f3680 2023-11-18T20:43:07.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9edb25640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9e81f3680 0x7ff9e81568c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc007310 tx=0x7ff9dc007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:07.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9e6ffd640 1 -- 172.21.15.60:0/1999069345 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc001650 con 0x7ff9e81f3680 2023-11-18T20:43:07.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9e6ffd640 1 -- 172.21.15.60:0/1999069345 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc00f960 con 0x7ff9e81f3680 2023-11-18T20:43:07.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9f058a640 1 -- 172.21.15.60:0/1999069345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e80b4420 con 0x7ff9e81f3680 2023-11-18T20:43:07.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9e6ffd640 1 -- 172.21.15.60:0/1999069345 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc00fd10 con 0x7ff9e81f3680 2023-11-18T20:43:07.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.168+0000 7ff9f058a640 1 -- 172.21.15.60:0/1999069345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e80cd0b0 con 0x7ff9e81f3680 2023-11-18T20:43:07.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.169+0000 7ff9f058a640 1 -- 172.21.15.60:0/1999069345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e8069400 con 0x7ff9e81f3680 2023-11-18T20:43:07.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.170+0000 7ff9e6ffd640 1 -- 172.21.15.60:0/1999069345 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9dc01d460 con 0x7ff9e81f3680 2023-11-18T20:43:07.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.170+0000 7ff9e6ffd640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff9d406ae70 0x7ff9d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.170+0000 7ff9e6ffd640 1 -- 172.21.15.60:0/1999069345 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9dc089610 con 0x7ff9e81f3680 2023-11-18T20:43:07.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.171+0000 7ff9ee326640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff9d406ae70 0x7ff9d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.171+0000 7ff9ee326640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff9d406ae70 0x7ff9d406d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8002010 tx=0x7ff9d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:07.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.174+0000 7ff9e6ffd640 1 -- 172.21.15.60:0/1999069345 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9dc054770 con 0x7ff9e81f3680 2023-11-18T20:43:07.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.350+0000 7ff9f058a640 1 -- 172.21.15.60:0/1999069345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9e815e030 con 0x7ff9e81f3680 2023-11-18T20:43:07.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.350+0000 7ff9e6ffd640 1 -- 172.21.15.60:0/1999069345 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9dc057d90 con 0x7ff9e81f3680 2023-11-18T20:43:07.352 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:07.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 -- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff9d406ae70 msgr2=0x7ff9d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff9d406ae70 0x7ff9d406d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8002010 tx=0x7ff9d8009040 comp rx=0 tx=0).stop 2023-11-18T20:43:07.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 -- 172.21.15.60:0/1999069345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9e81f3680 msgr2=0x7ff9e81568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9e81f3680 0x7ff9e81568c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc007310 tx=0x7ff9dc007710 comp rx=0 tx=0).stop 2023-11-18T20:43:07.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 -- 172.21.15.60:0/1999069345 shutdown_connections 2023-11-18T20:43:07.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff9d406ae70 0x7ff9d406d330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9e80cc6f0 0x7ff9e80ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9e81f3680 0x7ff9e81568c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 --2- 172.21.15.60:0/1999069345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9d0004d00 0x7ff9e80caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 -- 172.21.15.60:0/1999069345 >> 172.21.15.60:0/1999069345 conn(0x7ff9e8072020 msgr2=0x7ff9e815e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:07.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 -- 172.21.15.60:0/1999069345 shutdown_connections 2023-11-18T20:43:07.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.352+0000 7ff9f058a640 1 -- 172.21.15.60:0/1999069345 wait complete. 2023-11-18T20:43:07.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:07.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.472+0000 7ff89a87c640 1 Processor -- start 2023-11-18T20:43:07.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.472+0000 7ff89a87c640 1 -- start start 2023-11-18T20:43:07.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.473+0000 7ff89a87c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff894144980 0x7ff89415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.473+0000 7ff89a87c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8940ceb70 0x7ff8940cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.473+0000 7ff89a87c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff894071770 con 0x7ff8940b4370 2023-11-18T20:43:07.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.473+0000 7ff89a87c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8940ccdc0 con 0x7ff894144980 2023-11-18T20:43:07.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.473+0000 7ff89a87c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940cd0a0 con 0x7ff8940ceb70 2023-11-18T20:43:07.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.473+0000 7ff893fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff894144980 0x7ff89415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.473+0000 7ff8937fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8940ceb70 0x7ff8940cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.473+0000 7ff898e19640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff8940b4370 0x7ff894071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:39050/0 (socket says 172.21.15.60:39050) 2023-11-18T20:43:07.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.473+0000 7ff898e19640 1 -- 172.21.15.60:0/1581402258 learned_addr learned my addr 172.21.15.60:0/1581402258 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:07.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.474+0000 7ff893fff640 1 -- 172.21.15.60:0/1581402258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8940ceb70 msgr2=0x7ff8940cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:07.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.474+0000 7ff893fff640 1 --2- 172.21.15.60:0/1581402258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8940ceb70 0x7ff8940cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.474+0000 7ff893fff640 1 -- 172.21.15.60:0/1581402258 >> v1:172.21.15.100:6789/0 conn(0x7ff8940b4370 legacy=0x7ff894071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.474+0000 7ff893fff640 1 -- 172.21.15.60:0/1581402258 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8940cd380 con 0x7ff894144980 2023-11-18T20:43:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.474+0000 7ff892ffd640 1 -- 172.21.15.60:0/1581402258 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094484898 0 0) 0x7ff894071770 con 0x7ff8940b4370 2023-11-18T20:43:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.474+0000 7ff893fff640 1 --2- 172.21.15.60:0/1581402258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff894144980 0x7ff89415c7c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0097a0 tx=0x7ff87c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f1d2c716c31aaf server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff892ffd640 1 -- 172.21.15.60:0/1581402258 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c031830 con 0x7ff894144980 2023-11-18T20:43:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff892ffd640 1 -- 172.21.15.60:0/1581402258 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c031990 con 0x7ff894144980 2023-11-18T20:43:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff892ffd640 1 -- 172.21.15.60:0/1581402258 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c031d20 con 0x7ff894144980 2023-11-18T20:43:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff89a87c640 1 -- 172.21.15.60:0/1581402258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff894144980 msgr2=0x7ff89415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff89a87c640 1 --2- 172.21.15.60:0/1581402258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff894144980 0x7ff89415c7c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0097a0 tx=0x7ff87c02b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff89a87c640 1 -- 172.21.15.60:0/1581402258 shutdown_connections 2023-11-18T20:43:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff89a87c640 1 --2- 172.21.15.60:0/1581402258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8940ceb70 0x7ff8940cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff89a87c640 1 --2- 172.21.15.60:0/1581402258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff894144980 0x7ff89415c7c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff89a87c640 1 -- 172.21.15.60:0/1581402258 >> 172.21.15.60:0/1581402258 conn(0x7ff89406a4d0 msgr2=0x7ff89406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:07.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff89a87c640 1 -- 172.21.15.60:0/1581402258 shutdown_connections 2023-11-18T20:43:07.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.475+0000 7ff89a87c640 1 -- 172.21.15.60:0/1581402258 wait complete. 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.476+0000 7ff89a87c640 1 Processor -- start 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff89a87c640 1 -- start start 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff89a87c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8940b4370 0x7ff8941f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff89a87c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8940ceb70 0x7ff8940b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff89a87c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff894144980 0x7ff894142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff89a87c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8941f6ef0 con 0x7ff894144980 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff89a87c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8941f29f0 con 0x7ff8940ceb70 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff89a87c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8941f2c90 con 0x7ff8940b4370 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff898e19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff894144980 0x7ff894142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff898e19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff894144980 0x7ff894142a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42628/0 (socket says 172.21.15.60:42628) 2023-11-18T20:43:07.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff898e19640 1 -- 172.21.15.60:0/2357010518 learned_addr learned my addr 172.21.15.60:0/2357010518 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:07.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.477+0000 7ff8937fe640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8940ceb70 0x7ff8940b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff893fff640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8940b4370 0x7ff8941f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff898e19640 1 -- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8940b4370 msgr2=0x7ff8941f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff898e19640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8940b4370 0x7ff8941f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff898e19640 1 -- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8940ceb70 msgr2=0x7ff8940b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff898e19640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8940ceb70 0x7ff8940b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff898e19640 1 -- 172.21.15.60:0/2357010518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c009400 con 0x7ff894144980 2023-11-18T20:43:07.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff898e19640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff894144980 0x7ff894142a60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00c7a0 tx=0x7ff88c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:07.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff890ff9640 1 -- 172.21.15.60:0/2357010518 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00a640 con 0x7ff894144980 2023-11-18T20:43:07.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff890ff9640 1 -- 172.21.15.60:0/2357010518 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c00a7a0 con 0x7ff894144980 2023-11-18T20:43:07.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff890ff9640 1 -- 172.21.15.60:0/2357010518 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00abd0 con 0x7ff894144980 2023-11-18T20:43:07.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff89a87c640 1 -- 172.21.15.60:0/2357010518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8940b1b50 con 0x7ff894144980 2023-11-18T20:43:07.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.478+0000 7ff89a87c640 1 -- 172.21.15.60:0/2357010518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8941f6930 con 0x7ff894144980 2023-11-18T20:43:07.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.479+0000 7ff89a87c640 1 -- 172.21.15.60:0/2357010518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff894069400 con 0x7ff894144980 2023-11-18T20:43:07.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.480+0000 7ff890ff9640 1 -- 172.21.15.60:0/2357010518 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff88c00ad30 con 0x7ff894144980 2023-11-18T20:43:07.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.480+0000 7ff890ff9640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff87406ae40 0x7ff87406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.480+0000 7ff890ff9640 1 -- 172.21.15.60:0/2357010518 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff88c088af0 con 0x7ff894144980 2023-11-18T20:43:07.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.480+0000 7ff893fff640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff87406ae40 0x7ff87406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.481+0000 7ff893fff640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff87406ae40 0x7ff87406d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff87c036000 tx=0x7ff87c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:07.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.484+0000 7ff890ff9640 1 -- 172.21.15.60:0/2357010518 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff88c053c50 con 0x7ff894144980 2023-11-18T20:43:07.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.659+0000 7ff89a87c640 1 -- 172.21.15.60:0/2357010518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8941f2dd0 con 0x7ff894144980 2023-11-18T20:43:07.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.660+0000 7ff890ff9640 1 -- 172.21.15.60:0/2357010518 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff88c018020 con 0x7ff894144980 2023-11-18T20:43:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:07.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.661+0000 7ff89a87c640 1 -- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff87406ae40 msgr2=0x7ff87406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.661+0000 7ff89a87c640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff87406ae40 0x7ff87406d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff87c036000 tx=0x7ff87c03d040 comp rx=0 tx=0).stop 2023-11-18T20:43:07.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.661+0000 7ff89a87c640 1 -- 172.21.15.60:0/2357010518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff894144980 msgr2=0x7ff894142a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.661+0000 7ff89a87c640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff894144980 0x7ff894142a60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00c7a0 tx=0x7ff88c00cc60 comp rx=0 tx=0).stop 2023-11-18T20:43:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.661+0000 7ff89a87c640 1 -- 172.21.15.60:0/2357010518 shutdown_connections 2023-11-18T20:43:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.661+0000 7ff89a87c640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff87406ae40 0x7ff87406d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.661+0000 7ff89a87c640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff894144980 0x7ff894142a60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.661+0000 7ff89a87c640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8940ceb70 0x7ff8940b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.661+0000 7ff89a87c640 1 --2- 172.21.15.60:0/2357010518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8940b4370 0x7ff8941f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.662+0000 7ff89a87c640 1 -- 172.21.15.60:0/2357010518 >> 172.21.15.60:0/2357010518 conn(0x7ff89406a4d0 msgr2=0x7ff8940b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:07.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.662+0000 7ff89a87c640 1 -- 172.21.15.60:0/2357010518 shutdown_connections 2023-11-18T20:43:07.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.662+0000 7ff89a87c640 1 -- 172.21.15.60:0/2357010518 wait complete. 2023-11-18T20:43:07.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:07.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.782+0000 7f04e7783640 1 Processor -- start 2023-11-18T20:43:07.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.782+0000 7f04e7783640 1 -- start start 2023-11-18T20:43:07.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04e7783640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f04e0146460 0x7f04e00b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04e7783640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e006f1c0 0x7f04e006baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04e7783640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e0146fd0 con 0x7f04e0146840 2023-11-18T20:43:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04e7783640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e006c180 con 0x7f04e0146460 2023-11-18T20:43:07.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04e7783640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e006c460 con 0x7f04e006f1c0 2023-11-18T20:43:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04e4d1e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f04e0146840 0x7f04e006eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50132/0 (socket says 172.21.15.60:50132) 2023-11-18T20:43:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04e4d1e640 1 -- 172.21.15.60:0/933520374 learned_addr learned my addr 172.21.15.60:0/933520374 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04e551f640 1 --2- 172.21.15.60:0/933520374 >> v2:172.21.15.100:3300/0 conn(0x7f04e0146460 0x7f04e00b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04e5d20640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e006f1c0 0x7f04e006baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981777025 0 0) 0x7f04e0146fd0 con 0x7f04e0146840 2023-11-18T20:43:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.783+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04cc003520 con 0x7f04e0146840 2023-11-18T20:43:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1318000900 0 0) 0x7f04cc003520 con 0x7f04e0146840 2023-11-18T20:43:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e006f1c0 msgr2=0x7f04e006baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:07.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e006f1c0 0x7f04e006baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 >> v2:172.21.15.100:3300/0 conn(0x7f04e0146460 msgr2=0x7f04e00b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:07.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 --2- 172.21.15.60:0/933520374 >> v2:172.21.15.100:3300/0 conn(0x7f04e0146460 0x7f04e00b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e006c740 con 0x7f04e0146840 2023-11-18T20:43:07.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f04d0001670 con 0x7f04e0146840 2023-11-18T20:43:07.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04e551f640 1 --2- 172.21.15.60:0/933520374 >> v2:172.21.15.100:3300/0 conn(0x7f04e0146460 0x7f04e00b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:07.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 >> v1:172.21.15.60:6789/0 conn(0x7f04e0146840 legacy=0x7f04e006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e006f1c0 0x7f04cc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc003ef0 0x7f04cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04cc004d00 0x7f04cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e006c180 con 0x7f04cc004d00 2023-11-18T20:43:07.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e006c460 con 0x7f04cc003ef0 2023-11-18T20:43:07.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc00b980 con 0x7f04e006f1c0 2023-11-18T20:43:07.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04e5d20640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04cc004d00 0x7f04cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04e551f640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e006f1c0 0x7f04cc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04e4d1e640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc003ef0 0x7f04cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04e5d20640 1 -- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e006f1c0 msgr2=0x7f04cc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:07.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04e5d20640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e006f1c0 0x7f04cc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04e5d20640 1 -- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc003ef0 msgr2=0x7f04cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:07.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04e5d20640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc003ef0 0x7f04cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04e5d20640 1 -- 172.21.15.60:0/933520374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc003520 con 0x7f04cc004d00 2023-11-18T20:43:07.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.784+0000 7f04e4d1e640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc003ef0 0x7f04cc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:07.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.785+0000 7f04e5d20640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04cc004d00 0x7f04cc005180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009fc0 tx=0x7f04cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a8910a66f510fc server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:07.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.785+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d802cc00 con 0x7f04cc004d00 2023-11-18T20:43:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.785+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d800f040 con 0x7f04cc004d00 2023-11-18T20:43:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.785+0000 7f04d7fff640 1 -- 172.21.15.60:0/933520374 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8034440 con 0x7f04cc004d00 2023-11-18T20:43:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.785+0000 7f04e7783640 1 -- 172.21.15.60:0/933520374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04cc004d00 msgr2=0x7f04cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.785+0000 7f04e7783640 1 --2- 172.21.15.60:0/933520374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04cc004d00 0x7f04cc005180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009fc0 tx=0x7f04cc0057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.786+0000 7f04e5d20640 1 -- 172.21.15.60:0/933520374 reap_dead start 2023-11-18T20:43:07.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.786+0000 7f04e7783640 1 -- 172.21.15.60:0/933520374 shutdown_connections 2023-11-18T20:43:07.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.786+0000 7f04e7783640 1 -- 172.21.15.60:0/933520374 >> 172.21.15.60:0/933520374 conn(0x7f04e006a6b0 msgr2=0x7f04e006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:07.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.786+0000 7f04e7783640 1 -- 172.21.15.60:0/933520374 shutdown_connections 2023-11-18T20:43:07.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.786+0000 7f04e7783640 1 -- 172.21.15.60:0/933520374 wait complete. 2023-11-18T20:43:07.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.787+0000 7f04e7783640 1 Processor -- start 2023-11-18T20:43:07.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.787+0000 7f04e7783640 1 -- start start 2023-11-18T20:43:07.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e7783640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e00cebd0 0x7f04e00ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e7783640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04e00a8870 0x7f04e00ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e7783640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e00cefb0 0x7f04e00cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e7783640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0132c70 con 0x7f04e00a8870 2023-11-18T20:43:07.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e7783640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e01337d0 con 0x7f04e00cefb0 2023-11-18T20:43:07.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e7783640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0132ea0 con 0x7f04e00cebd0 2023-11-18T20:43:07.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e4d1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04e00a8870 0x7f04e00ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e4d1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04e00a8870 0x7f04e00ab7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42654/0 (socket says 172.21.15.60:42654) 2023-11-18T20:43:07.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e4d1e640 1 -- 172.21.15.60:0/893019693 learned_addr learned my addr 172.21.15.60:0/893019693 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:07.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e551f640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e00cebd0 0x7f04e00ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e5d20640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e00cefb0 0x7f04e00cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e4d1e640 1 -- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e00cebd0 msgr2=0x7f04e00ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e4d1e640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e00cebd0 0x7f04e00ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e4d1e640 1 -- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e00cefb0 msgr2=0x7f04e00cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e4d1e640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e00cefb0 0x7f04e00cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e4d1e640 1 -- 172.21.15.60:0/893019693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e0133180 con 0x7f04e00a8870 2023-11-18T20:43:07.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e5d20640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e00cefb0 0x7f04e00cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:07.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e4d1e640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04e00a8870 0x7f04e00ab7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f04d000eaf0 tx=0x7f04d00016a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:07.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04d5ffb640 1 -- 172.21.15.60:0/893019693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d000c8d0 con 0x7f04e00a8870 2023-11-18T20:43:07.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04d5ffb640 1 -- 172.21.15.60:0/893019693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d000ca30 con 0x7f04e00a8870 2023-11-18T20:43:07.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04d5ffb640 1 -- 172.21.15.60:0/893019693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d000cda0 con 0x7f04e00a8870 2023-11-18T20:43:07.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.788+0000 7f04e7783640 1 -- 172.21.15.60:0/893019693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e0133460 con 0x7f04e00a8870 2023-11-18T20:43:07.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.789+0000 7f04e7783640 1 -- 172.21.15.60:0/893019693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e00cfb50 con 0x7f04e00a8870 2023-11-18T20:43:07.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.790+0000 7f04e7783640 1 -- 172.21.15.60:0/893019693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e00cfd70 con 0x7f04e00a8870 2023-11-18T20:43:07.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.790+0000 7f04d5ffb640 1 -- 172.21.15.60:0/893019693 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f04d001d460 con 0x7f04e00a8870 2023-11-18T20:43:07.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.790+0000 7f04d5ffb640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f04c406ae40 0x7f04c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:07.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.790+0000 7f04d5ffb640 1 -- 172.21.15.60:0/893019693 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04d0089600 con 0x7f04e00a8870 2023-11-18T20:43:07.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.790+0000 7f04e551f640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f04c406ae40 0x7f04c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:07.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.791+0000 7f04e551f640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f04c406ae40 0x7f04c406d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009dc0 tx=0x7f04dc009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:07.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.795+0000 7f04d5ffb640 1 -- 172.21.15.60:0/893019693 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04d00546b0 con 0x7f04e00a8870 2023-11-18T20:43:07.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.974+0000 7f04e7783640 1 -- 172.21.15.60:0/893019693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04e0069400 con 0x7f04e00a8870 2023-11-18T20:43:07.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.974+0000 7f04d5ffb640 1 -- 172.21.15.60:0/893019693 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04d0057cd0 con 0x7f04e00a8870 2023-11-18T20:43:07.977 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:07.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 -- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f04c406ae40 msgr2=0x7f04c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f04c406ae40 0x7f04c406d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009dc0 tx=0x7f04dc009190 comp rx=0 tx=0).stop 2023-11-18T20:43:07.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 -- 172.21.15.60:0/893019693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04e00a8870 msgr2=0x7f04e00ab7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:07.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04e00a8870 0x7f04e00ab7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f04d000eaf0 tx=0x7f04d00016a0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 -- 172.21.15.60:0/893019693 shutdown_connections 2023-11-18T20:43:07.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f04c406ae40 0x7f04c406d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e00cefb0 0x7f04e00cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f04e00a8870 0x7f04e00ab7a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 --2- 172.21.15.60:0/893019693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04e00cebd0 0x7f04e00ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:07.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 -- 172.21.15.60:0/893019693 >> 172.21.15.60:0/893019693 conn(0x7f04e006a6b0 msgr2=0x7f04e0070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:07.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 -- 172.21.15.60:0/893019693 shutdown_connections 2023-11-18T20:43:07.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:07.977+0000 7f04e7783640 1 -- 172.21.15.60:0/893019693 wait complete. 2023-11-18T20:43:07.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:08.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.095+0000 7f96f0ffc640 1 Processor -- start 2023-11-18T20:43:08.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.095+0000 7f96f0ffc640 1 -- start start 2023-11-18T20:43:08.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.095+0000 7f96f0ffc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96ec15d330 0x7f96ec0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.095+0000 7f96f0ffc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 0x7f96ec0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.095+0000 7f96f0ffc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec0b2b10 con 0x7f96ec0d3730 2023-11-18T20:43:08.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.095+0000 7f96f0ffc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec0d3b50 con 0x7f96ec15d330 2023-11-18T20:43:08.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.095+0000 7f96f0ffc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0d1590 con 0x7f96ec0d2910 2023-11-18T20:43:08.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.096+0000 7f96ea59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96ec15d330 0x7f96ec0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.096+0000 7f96e3fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 0x7f96ec0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.096+0000 7f96ea59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f96ec15d330 0x7f96ec0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:34052/0 (socket says 172.21.15.60:34052) 2023-11-18T20:43:08.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.096+0000 7f96ea59c640 1 -- 172.21.15.60:0/1153876587 learned_addr learned my addr 172.21.15.60:0/1153876587 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:08.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.096+0000 7f96ead9d640 1 --1- 172.21.15.60:0/1153876587 >> v1:172.21.15.100:6789/0 conn(0x7f96ec0d3730 0x7f96ec0b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:39064/0 (socket says 172.21.15.60:39064) 2023-11-18T20:43:08.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.096+0000 7f96e3fff640 1 -- 172.21.15.60:0/1153876587 >> v2:172.21.15.100:3300/0 conn(0x7f96ec15d330 msgr2=0x7f96ec0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:08.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.096+0000 7f96e3fff640 1 --2- 172.21.15.60:0/1153876587 >> v2:172.21.15.100:3300/0 conn(0x7f96ec15d330 0x7f96ec0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.096+0000 7f96e3fff640 1 -- 172.21.15.60:0/1153876587 >> v1:172.21.15.100:6789/0 conn(0x7f96ec0d3730 legacy=0x7f96ec0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.096+0000 7f96e3fff640 1 -- 172.21.15.60:0/1153876587 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec0d1870 con 0x7f96ec0d2910 2023-11-18T20:43:08.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.097+0000 7f96e3fff640 1 --2- 172.21.15.60:0/1153876587 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 0x7f96ec0d31f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96d40097a0 tx=0x7f96d402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8f0952b91e32bf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:08.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.097+0000 7f96e9d9b640 1 -- 172.21.15.60:0/1153876587 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d402e070 con 0x7f96ec0d2910 2023-11-18T20:43:08.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.097+0000 7f96e9d9b640 1 -- 172.21.15.60:0/1153876587 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d40315a0 con 0x7f96ec0d2910 2023-11-18T20:43:08.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.097+0000 7f96e9d9b640 1 -- 172.21.15.60:0/1153876587 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4031930 con 0x7f96ec0d2910 2023-11-18T20:43:08.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.097+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1153876587 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 msgr2=0x7f96ec0d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.097+0000 7f96f0ffc640 1 --2- 172.21.15.60:0/1153876587 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 0x7f96ec0d31f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96d40097a0 tx=0x7f96d402b840 comp rx=0 tx=0).stop 2023-11-18T20:43:08.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.097+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1153876587 shutdown_connections 2023-11-18T20:43:08.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.097+0000 7f96f0ffc640 1 --2- 172.21.15.60:0/1153876587 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 0x7f96ec0d31f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.097+0000 7f96f0ffc640 1 --2- 172.21.15.60:0/1153876587 >> v2:172.21.15.100:3300/0 conn(0x7f96ec15d330 0x7f96ec0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.097+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1153876587 >> 172.21.15.60:0/1153876587 conn(0x7f96ec072020 msgr2=0x7f96ec072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:08.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.098+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1153876587 shutdown_connections 2023-11-18T20:43:08.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.098+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1153876587 wait complete. 2023-11-18T20:43:08.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.099+0000 7f96f0ffc640 1 Processor -- start 2023-11-18T20:43:08.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.099+0000 7f96f0ffc640 1 -- start start 2023-11-18T20:43:08.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96f0ffc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 0x7f96ec0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96f0ffc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96ec0d3730 0x7f96ec158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96f0ffc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96ec15d330 0x7f96ec0b6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96f0ffc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec15d710 con 0x7f96ec15d330 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96f0ffc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec15e460 con 0x7f96ec0d3730 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96f0ffc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec15d960 con 0x7f96ec0d2910 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96ead9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96ec15d330 0x7f96ec0b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96ead9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96ec15d330 0x7f96ec0b6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42670/0 (socket says 172.21.15.60:42670) 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96ead9d640 1 -- 172.21.15.60:0/1010543744 learned_addr learned my addr 172.21.15.60:0/1010543744 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96ea59c640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 0x7f96ec0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96e3fff640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96ec0d3730 0x7f96ec158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96ead9d640 1 -- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 msgr2=0x7f96ec0cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96ead9d640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 0x7f96ec0cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96ead9d640 1 -- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96ec0d3730 msgr2=0x7f96ec158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96ead9d640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96ec0d3730 0x7f96ec158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96ead9d640 1 -- 172.21.15.60:0/1010543744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4009400 con 0x7f96ec15d330 2023-11-18T20:43:08.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96ead9d640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96ec15d330 0x7f96ec0b6560 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c730 tx=0x7f96e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:08.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.100+0000 7f96e2ffd640 1 -- 172.21.15.60:0/1010543744 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e400ce40 con 0x7f96ec15d330 2023-11-18T20:43:08.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.101+0000 7f96e2ffd640 1 -- 172.21.15.60:0/1010543744 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e400a640 con 0x7f96ec15d330 2023-11-18T20:43:08.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.101+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1010543744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec15dc40 con 0x7f96ec15d330 2023-11-18T20:43:08.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.101+0000 7f96e2ffd640 1 -- 172.21.15.60:0/1010543744 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e400a9f0 con 0x7f96ec15d330 2023-11-18T20:43:08.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.101+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1010543744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec1590b0 con 0x7f96ec15d330 2023-11-18T20:43:08.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.102+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1010543744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec0cb130 con 0x7f96ec15d330 2023-11-18T20:43:08.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.102+0000 7f96e2ffd640 1 -- 172.21.15.60:0/1010543744 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f96e400ab50 con 0x7f96ec15d330 2023-11-18T20:43:08.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.102+0000 7f96e2ffd640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f96cc06ae40 0x7f96cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.102+0000 7f96e2ffd640 1 -- 172.21.15.60:0/1010543744 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96e4088c70 con 0x7f96ec15d330 2023-11-18T20:43:08.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.103+0000 7f96ea59c640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f96cc06ae40 0x7f96cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.103+0000 7f96ea59c640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f96cc06ae40 0x7f96cc06d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009bd0 tx=0x7f96dc009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:08.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.107+0000 7f96e2ffd640 1 -- 172.21.15.60:0/1010543744 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96e4053dd0 con 0x7f96ec15d330 2023-11-18T20:43:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.298+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1010543744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f96ec15de30 con 0x7f96ec15d330 2023-11-18T20:43:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.298+0000 7f96e2ffd640 1 -- 172.21.15.60:0/1010543744 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f96e4017020 con 0x7f96ec15d330 2023-11-18T20:43:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.300+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f96cc06ae40 msgr2=0x7f96cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.300+0000 7f96f0ffc640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f96cc06ae40 0x7f96cc06d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009bd0 tx=0x7f96dc009240 comp rx=0 tx=0).stop 2023-11-18T20:43:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.300+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1010543744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96ec15d330 msgr2=0x7f96ec0b6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.300+0000 7f96f0ffc640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96ec15d330 0x7f96ec0b6560 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c730 tx=0x7f96e400cbf0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.300+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1010543744 shutdown_connections 2023-11-18T20:43:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.300+0000 7f96f0ffc640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f96cc06ae40 0x7f96cc06d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.300+0000 7f96f0ffc640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96ec15d330 0x7f96ec0b6560 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.300+0000 7f96f0ffc640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96ec0d3730 0x7f96ec158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.300+0000 7f96f0ffc640 1 --2- 172.21.15.60:0/1010543744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96ec0d2910 0x7f96ec0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.300+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1010543744 >> 172.21.15.60:0/1010543744 conn(0x7f96ec072020 msgr2=0x7f96ec0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.301+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1010543744 shutdown_connections 2023-11-18T20:43:08.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.301+0000 7f96f0ffc640 1 -- 172.21.15.60:0/1010543744 wait complete. 2023-11-18T20:43:08.314 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:43:08.314 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:08.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:08.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.474+0000 7f95ce19e640 1 Processor -- start 2023-11-18T20:43:08.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.474+0000 7f95ce19e640 1 -- start start 2023-11-18T20:43:08.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95ce19e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d160 0x7f95c80d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95ce19e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95c80d0bb0 0x7f95c80d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95ce19e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c80cdca0 con 0x7f95c815d540 2023-11-18T20:43:08.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95ce19e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c80d1390 con 0x7f95c80d0bb0 2023-11-18T20:43:08.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95ce19e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80cf080 con 0x7f95c815d160 2023-11-18T20:43:08.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95cd99d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95c80d0bb0 0x7f95c80d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95cd19c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d160 0x7f95c80d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95cd99d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95c80d0bb0 0x7f95c80d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:34072/0 (socket says 172.21.15.60:34072) 2023-11-18T20:43:08.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95cd19c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d160 0x7f95c80d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:44044/0 (socket says 172.21.15.60:44044) 2023-11-18T20:43:08.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95cd99d640 1 -- 172.21.15.60:0/645021960 learned_addr learned my addr 172.21.15.60:0/645021960 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:08.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95cc99b640 1 --1- 172.21.15.60:0/645021960 >> v1:172.21.15.100:6789/0 conn(0x7f95c815d540 0x7f95c80ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:39068/0 (socket says 172.21.15.60:39068) 2023-11-18T20:43:08.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.475+0000 7f95bffff640 1 -- 172.21.15.60:0/645021960 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696869219 0 0) 0x7f95c80cdca0 con 0x7f95c815d540 2023-11-18T20:43:08.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.476+0000 7f95bffff640 1 -- 172.21.15.60:0/645021960 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b4003520 con 0x7f95c815d540 2023-11-18T20:43:08.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.476+0000 7f95cd19c640 1 -- 172.21.15.60:0/645021960 >> v2:172.21.15.100:3300/0 conn(0x7f95c80d0bb0 msgr2=0x7f95c80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:08.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.476+0000 7f95cd19c640 1 --2- 172.21.15.60:0/645021960 >> v2:172.21.15.100:3300/0 conn(0x7f95c80d0bb0 0x7f95c80d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.476+0000 7f95cd19c640 1 -- 172.21.15.60:0/645021960 >> v1:172.21.15.100:6789/0 conn(0x7f95c815d540 legacy=0x7f95c80ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.476+0000 7f95cd19c640 1 -- 172.21.15.60:0/645021960 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80cded0 con 0x7f95c815d160 2023-11-18T20:43:08.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.476+0000 7f95cd99d640 1 --2- 172.21.15.60:0/645021960 >> v2:172.21.15.100:3300/0 conn(0x7f95c80d0bb0 0x7f95c80d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:08.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.476+0000 7f95cd19c640 1 --2- 172.21.15.60:0/645021960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d160 0x7f95c80d0350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95c40097a0 tx=0x7f95c402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5088606310fdedc server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:08.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95bffff640 1 -- 172.21.15.60:0/645021960 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4031830 con 0x7f95c815d160 2023-11-18T20:43:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95bffff640 1 -- 172.21.15.60:0/645021960 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c4031990 con 0x7f95c815d160 2023-11-18T20:43:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95bffff640 1 -- 172.21.15.60:0/645021960 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4031d20 con 0x7f95c815d160 2023-11-18T20:43:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95ce19e640 1 -- 172.21.15.60:0/645021960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d160 msgr2=0x7f95c80d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95ce19e640 1 --2- 172.21.15.60:0/645021960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d160 0x7f95c80d0350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95c40097a0 tx=0x7f95c402b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95ce19e640 1 -- 172.21.15.60:0/645021960 shutdown_connections 2023-11-18T20:43:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95ce19e640 1 --2- 172.21.15.60:0/645021960 >> v2:172.21.15.100:3300/0 conn(0x7f95c80d0bb0 0x7f95c80d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95ce19e640 1 --2- 172.21.15.60:0/645021960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d160 0x7f95c80d0350 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95ce19e640 1 -- 172.21.15.60:0/645021960 >> 172.21.15.60:0/645021960 conn(0x7f95c80d2fe0 msgr2=0x7f95c80d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:08.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95ce19e640 1 -- 172.21.15.60:0/645021960 shutdown_connections 2023-11-18T20:43:08.482 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.477+0000 7f95ce19e640 1 -- 172.21.15.60:0/645021960 wait complete. 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95ce19e640 1 Processor -- start 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95ce19e640 1 -- start start 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95ce19e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c80d0bb0 0x7f95c80ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95ce19e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c815d160 0x7f95c815a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95ce19e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d540 0x7f95c815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95ce19e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80ca690 con 0x7f95c815d160 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95ce19e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80ca970 con 0x7f95c80d0bb0 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95ce19e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80cac50 con 0x7f95c815d540 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95cc99b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c815d160 0x7f95c815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95cc99b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c815d160 0x7f95c815a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42682/0 (socket says 172.21.15.60:42682) 2023-11-18T20:43:08.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.479+0000 7f95cc99b640 1 -- 172.21.15.60:0/757242709 learned_addr learned my addr 172.21.15.60:0/757242709 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95cd19c640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c80d0bb0 0x7f95c80ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95cd99d640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d540 0x7f95c815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95cc99b640 1 -- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d540 msgr2=0x7f95c815a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95cc99b640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d540 0x7f95c815a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95cc99b640 1 -- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c80d0bb0 msgr2=0x7f95c80ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95cc99b640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c80d0bb0 0x7f95c80ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95cc99b640 1 -- 172.21.15.60:0/757242709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c0009b30 con 0x7f95c815d160 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95cc99b640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c815d160 0x7f95c815a090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f95b800c430 tx=0x7f95b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95bdffb640 1 -- 172.21.15.60:0/757242709 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80074e0 con 0x7f95c815d160 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95bdffb640 1 -- 172.21.15.60:0/757242709 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b8007660 con 0x7f95c815d160 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95ce19e640 1 -- 172.21.15.60:0/757242709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c4009400 con 0x7f95c815d160 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95bdffb640 1 -- 172.21.15.60:0/757242709 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8004a20 con 0x7f95c815d160 2023-11-18T20:43:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.480+0000 7f95ce19e640 1 -- 172.21.15.60:0/757242709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c80cb210 con 0x7f95c815d160 2023-11-18T20:43:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.481+0000 7f95ce19e640 1 -- 172.21.15.60:0/757242709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c8069320 con 0x7f95c815d160 2023-11-18T20:43:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.482+0000 7f95bdffb640 1 -- 172.21.15.60:0/757242709 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f95b8004b80 con 0x7f95c815d160 2023-11-18T20:43:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.482+0000 7f95bdffb640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95ac06ae40 0x7f95ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.482+0000 7f95bdffb640 1 -- 172.21.15.60:0/757242709 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95b8088e10 con 0x7f95c815d160 2023-11-18T20:43:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.483+0000 7f95cd19c640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95ac06ae40 0x7f95ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.483+0000 7f95cd19c640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95ac06ae40 0x7f95ac06d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95c4036000 tx=0x7f95c403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.487+0000 7f95bdffb640 1 -- 172.21.15.60:0/757242709 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95b8053ed0 con 0x7f95c815d160 2023-11-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.665+0000 7f95ce19e640 1 -- 172.21.15.60:0/757242709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95c808c770 con 0x7f95c815d160 2023-11-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.666+0000 7f95bdffb640 1 -- 172.21.15.60:0/757242709 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f95b80574f0 con 0x7f95c815d160 2023-11-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.668+0000 7f95ce19e640 1 -- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95ac06ae40 msgr2=0x7f95ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95ac06ae40 0x7f95ac06d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95c4036000 tx=0x7f95c403c040 comp rx=0 tx=0).stop 2023-11-18T20:43:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 -- 172.21.15.60:0/757242709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c815d160 msgr2=0x7f95c815a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c815d160 0x7f95c815a090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f95b800c430 tx=0x7f95b800c8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 -- 172.21.15.60:0/757242709 shutdown_connections 2023-11-18T20:43:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f95ac06ae40 0x7f95ac06d300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c815d540 0x7f95c815a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c815d160 0x7f95c815a090 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 --2- 172.21.15.60:0/757242709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c80d0bb0 0x7f95c80ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 -- 172.21.15.60:0/757242709 >> 172.21.15.60:0/757242709 conn(0x7f95c80d2fe0 msgr2=0x7f95c815c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 -- 172.21.15.60:0/757242709 shutdown_connections 2023-11-18T20:43:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.669+0000 7f95ce19e640 1 -- 172.21.15.60:0/757242709 wait complete. 2023-11-18T20:43:08.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:08.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:08.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.789+0000 7f82350a2640 1 Processor -- start 2023-11-18T20:43:08.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.789+0000 7f82350a2640 1 -- start start 2023-11-18T20:43:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.790+0000 7f82350a2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8230142270 0x7f82301426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.790+0000 7f82350a2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8230161d20 con 0x7f8230160fb0 2023-11-18T20:43:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.790+0000 7f82350a2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823015e760 con 0x7f823006ea50 2023-11-18T20:43:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.790+0000 7f82350a2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82300d1ee0 con 0x7f8230142270 2023-11-18T20:43:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.790+0000 7f822e59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8230160fb0 0x7f8230161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50142/0 (socket says 172.21.15.60:50142) 2023-11-18T20:43:08.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.790+0000 7f822e59c640 1 -- 172.21.15.60:0/2752766069 learned_addr learned my addr 172.21.15.60:0/2752766069 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.790+0000 7f822f59e640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8230142270 0x7f82301426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.790+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387936100 0 0) 0x7f8230161d20 con 0x7f8230160fb0 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.790+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f821c003520 con 0x7f8230160fb0 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008823479 0 0) 0x7f823015e760 con 0x7f823006ea50 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8230161d20 con 0x7f823006ea50 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2591736232 0 0) 0x7f821c003520 con 0x7f8230160fb0 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8230142270 msgr2=0x7f82301426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8230142270 0x7f82301426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 >> v1:172.21.15.100:6789/0 conn(0x7f823006ea50 legacy=0x7f823006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823015d9a0 con 0x7f8230160fb0 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f8218001670 con 0x7f8230160fb0 2023-11-18T20:43:08.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 >> v1:172.21.15.60:6789/0 conn(0x7f8230160fb0 legacy=0x7f8230161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 0x7f821c003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8230142270 0x7f821c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300d1ee0 con 0x7f821c003b30 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c00ba30 con 0x7f821c0047c0 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c00bbf0 con 0x7f8230142270 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822ed9d640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 0x7f821c003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822e59c640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822f59e640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8230142270 0x7f821c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822ed9d640 1 -- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8230142270 msgr2=0x7f821c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822ed9d640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8230142270 0x7f821c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822ed9d640 1 -- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c0047c0 msgr2=0x7f821c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:08.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822ed9d640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822ed9d640 1 -- 172.21.15.60:0/2752766069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c003520 con 0x7f821c003b30 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822e59c640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.791+0000 7f822f59e640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8230142270 0x7f821c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.792+0000 7f822ed9d640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 0x7f821c003f30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f82280018d0 tx=0x7f821c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96d989eeecaea7a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.792+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280052d0 con 0x7f821c003b30 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.792+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8228005430 con 0x7f821c003b30 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.792+0000 7f822dd9b640 1 -- 172.21.15.60:0/2752766069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228003810 con 0x7f821c003b30 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.793+0000 7f82350a2640 1 -- 172.21.15.60:0/2752766069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 msgr2=0x7f821c003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.793+0000 7f82350a2640 1 --2- 172.21.15.60:0/2752766069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 0x7f821c003f30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f82280018d0 tx=0x7f821c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.793+0000 7f822f59e640 1 -- 172.21.15.60:0/2752766069 reap_dead start 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.793+0000 7f82350a2640 1 -- 172.21.15.60:0/2752766069 shutdown_connections 2023-11-18T20:43:08.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.793+0000 7f82350a2640 1 -- 172.21.15.60:0/2752766069 >> 172.21.15.60:0/2752766069 conn(0x7f823006a6b0 msgr2=0x7f823006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.793+0000 7f82350a2640 1 -- 172.21.15.60:0/2752766069 shutdown_connections 2023-11-18T20:43:08.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.793+0000 7f82350a2640 1 -- 172.21.15.60:0/2752766069 wait complete. 2023-11-18T20:43:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.794+0000 7f82350a2640 1 Processor -- start 2023-11-18T20:43:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.794+0000 7f82350a2640 1 -- start start 2023-11-18T20:43:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f82350a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 0x7f823013b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f82350a2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f823013b9b0 0x7f823013d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f82350a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f823013d900 0x7f82301563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f82350a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823013dd80 con 0x7f821c003b30 2023-11-18T20:43:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f82350a2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823013c1a0 con 0x7f823013d900 2023-11-18T20:43:08.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f82350a2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8230155f60 con 0x7f823013b9b0 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 0x7f823013b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 0x7f823013b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42688/0 (socket says 172.21.15.60:42688) 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822ed9d640 1 -- 172.21.15.60:0/3016387563 learned_addr learned my addr 172.21.15.60:0/3016387563 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822f59e640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f823013d900 0x7f82301563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822e59c640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f823013b9b0 0x7f823013d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822ed9d640 1 -- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f823013b9b0 msgr2=0x7f823013d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822ed9d640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f823013b9b0 0x7f823013d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822ed9d640 1 -- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f823013d900 msgr2=0x7f82301563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822ed9d640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f823013d900 0x7f82301563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822ed9d640 1 -- 172.21.15.60:0/3016387563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823013b760 con 0x7f821c003b30 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.795+0000 7f822e59c640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f823013b9b0 0x7f823013d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:08.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.796+0000 7f822ed9d640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 0x7f823013b220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8228035040 tx=0x7f82280012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.796+0000 7f82277fe640 1 -- 172.21.15.60:0/3016387563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228006ea0 con 0x7f821c003b30 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.796+0000 7f82277fe640 1 -- 172.21.15.60:0/3016387563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8228007450 con 0x7f821c003b30 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.796+0000 7f82350a2640 1 -- 172.21.15.60:0/3016387563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8230155350 con 0x7f821c003b30 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.796+0000 7f82277fe640 1 -- 172.21.15.60:0/3016387563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280035f0 con 0x7f821c003b30 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.796+0000 7f82350a2640 1 -- 172.21.15.60:0/3016387563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8230155910 con 0x7f821c003b30 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.797+0000 7f82350a2640 1 -- 172.21.15.60:0/3016387563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8230069400 con 0x7f821c003b30 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.797+0000 7f82277fe640 1 -- 172.21.15.60:0/3016387563 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8228003750 con 0x7f821c003b30 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.797+0000 7f82277fe640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f821006ae40 0x7f821006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.798+0000 7f82277fe640 1 -- 172.21.15.60:0/3016387563 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82280ab430 con 0x7f821c003b30 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.798+0000 7f822e59c640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f821006ae40 0x7f821006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.798+0000 7f822e59c640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f821006ae40 0x7f821006d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f823013e720 tx=0x7f8218009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:08.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.802+0000 7f82277fe640 1 -- 172.21.15.60:0/3016387563 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8228076590 con 0x7f821c003b30 2023-11-18T20:43:08.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:43:08.912 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:43:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.979+0000 7f82350a2640 1 -- 172.21.15.60:0/3016387563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82300b4c40 con 0x7f821c003b30 2023-11-18T20:43:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.979+0000 7f82277fe640 1 -- 172.21.15.60:0/3016387563 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8228079bb0 con 0x7f821c003b30 2023-11-18T20:43:08.982 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.981+0000 7f82350a2640 1 -- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f821006ae40 msgr2=0x7f821006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.981+0000 7f82350a2640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f821006ae40 0x7f821006d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f823013e720 tx=0x7f8218009040 comp rx=0 tx=0).stop 2023-11-18T20:43:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.981+0000 7f82350a2640 1 -- 172.21.15.60:0/3016387563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 msgr2=0x7f823013b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.981+0000 7f82350a2640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 0x7f823013b220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8228035040 tx=0x7f82280012d0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.981+0000 7f82350a2640 1 -- 172.21.15.60:0/3016387563 shutdown_connections 2023-11-18T20:43:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.981+0000 7f82350a2640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f821006ae40 0x7f821006d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.981+0000 7f82350a2640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f823013d900 0x7f82301563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.981+0000 7f82350a2640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f823013b9b0 0x7f823013d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.981+0000 7f82350a2640 1 --2- 172.21.15.60:0/3016387563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f821c003b30 0x7f823013b220 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:08.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.981+0000 7f82350a2640 1 -- 172.21.15.60:0/3016387563 >> 172.21.15.60:0/3016387563 conn(0x7f823006a6b0 msgr2=0x7f823015ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:08.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.982+0000 7f82350a2640 1 -- 172.21.15.60:0/3016387563 shutdown_connections 2023-11-18T20:43:08.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:08.982+0000 7f82350a2640 1 -- 172.21.15.60:0/3016387563 wait complete. 2023-11-18T20:43:08.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:09.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.143+0000 7f5595009640 1 Processor -- start 2023-11-18T20:43:09.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.143+0000 7f5595009640 1 -- start start 2023-11-18T20:43:09.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f5595009640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5590155790 0x7f559006f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f5595009640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f559006abe0 0x7f559015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f5595009640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5590155b70 con 0x7f559015b5e0 2023-11-18T20:43:09.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f5595009640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559006ee30 con 0x7f5590155790 2023-11-18T20:43:09.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f5595009640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559006f3b0 con 0x7f559006abe0 2023-11-18T20:43:09.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f558ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5590155790 0x7f559006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f558ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5590155790 0x7f559006f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42698/0 (socket says 172.21.15.60:42698) 2023-11-18T20:43:09.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f558ed9d640 1 -- 172.21.15.60:0/586511000 learned_addr learned my addr 172.21.15.60:0/586511000 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:09.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f558f59e640 1 --2- 172.21.15.60:0/586511000 >> v2:172.21.15.100:3300/0 conn(0x7f559006abe0 0x7f559015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f558ed9d640 1 -- 172.21.15.60:0/586511000 >> v2:172.21.15.100:3300/0 conn(0x7f559006abe0 msgr2=0x7f559015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f558ed9d640 1 --2- 172.21.15.60:0/586511000 >> v2:172.21.15.100:3300/0 conn(0x7f559006abe0 0x7f559015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f558ed9d640 1 -- 172.21.15.60:0/586511000 >> v1:172.21.15.100:6789/0 conn(0x7f559015b5e0 legacy=0x7f559006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.144+0000 7f558ed9d640 1 -- 172.21.15.60:0/586511000 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559015c170 con 0x7f5590155790 2023-11-18T20:43:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.145+0000 7f558ed9d640 1 --2- 172.21.15.60:0/586511000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5590155790 0x7f559006f840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f559006b0e0 tx=0x7f558002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a69dbd74e1b3cba server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.145+0000 7f558e59c640 1 -- 172.21.15.60:0/586511000 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580030860 con 0x7f5590155790 2023-11-18T20:43:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.145+0000 7f558e59c640 1 -- 172.21.15.60:0/586511000 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55800309c0 con 0x7f5590155790 2023-11-18T20:43:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.145+0000 7f558e59c640 1 -- 172.21.15.60:0/586511000 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580030d50 con 0x7f5590155790 2023-11-18T20:43:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.145+0000 7f5595009640 1 -- 172.21.15.60:0/586511000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5590155790 msgr2=0x7f559006f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.145+0000 7f5595009640 1 --2- 172.21.15.60:0/586511000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5590155790 0x7f559006f840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f559006b0e0 tx=0x7f558002b9a0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.146+0000 7f5595009640 1 -- 172.21.15.60:0/586511000 shutdown_connections 2023-11-18T20:43:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.146+0000 7f5595009640 1 --2- 172.21.15.60:0/586511000 >> v2:172.21.15.100:3300/0 conn(0x7f559006abe0 0x7f559015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.146+0000 7f5595009640 1 --2- 172.21.15.60:0/586511000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5590155790 0x7f559006f840 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.146+0000 7f5595009640 1 -- 172.21.15.60:0/586511000 >> 172.21.15.60:0/586511000 conn(0x7f55900b1e50 msgr2=0x7f55900b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.146+0000 7f5595009640 1 -- 172.21.15.60:0/586511000 shutdown_connections 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.146+0000 7f5595009640 1 -- 172.21.15.60:0/586511000 wait complete. 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.147+0000 7f5595009640 1 Processor -- start 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.147+0000 7f5595009640 1 -- start start 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f5595009640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559006abe0 0x7f559014cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f5595009640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5590155790 0x7f5590153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f5595009640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559015b5e0 0x7f55901e3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f5595009640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55901537f0 con 0x7f559015b5e0 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f5595009640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590157080 con 0x7f559006abe0 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f5595009640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5590157360 con 0x7f5590155790 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558f59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559015b5e0 0x7f55901e3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558f59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559015b5e0 0x7f55901e3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42710/0 (socket says 172.21.15.60:42710) 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558f59e640 1 -- 172.21.15.60:0/249783074 learned_addr learned my addr 172.21.15.60:0/249783074 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:09.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558ed9d640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559006abe0 0x7f559014cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f5587fff640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5590155790 0x7f5590153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558f59e640 1 -- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5590155790 msgr2=0x7f5590153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558f59e640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5590155790 0x7f5590153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558f59e640 1 -- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559006abe0 msgr2=0x7f559014cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558f59e640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559006abe0 0x7f559014cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558f59e640 1 -- 172.21.15.60:0/249783074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55800094d0 con 0x7f559015b5e0 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558ed9d640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559006abe0 0x7f559014cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f558f59e640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559015b5e0 0x7f55901e3860 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5588007e40 tx=0x7f558800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.148+0000 7f55877fe640 1 -- 172.21.15.60:0/249783074 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588014620 con 0x7f559015b5e0 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.149+0000 7f55877fe640 1 -- 172.21.15.60:0/249783074 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558800ee90 con 0x7f559015b5e0 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.149+0000 7f5595009640 1 -- 172.21.15.60:0/249783074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5590157640 con 0x7f559015b5e0 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.149+0000 7f55877fe640 1 -- 172.21.15.60:0/249783074 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55880149e0 con 0x7f559015b5e0 2023-11-18T20:43:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.149+0000 7f5595009640 1 -- 172.21.15.60:0/249783074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5590157c00 con 0x7f559015b5e0 2023-11-18T20:43:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.149+0000 7f5595009640 1 -- 172.21.15.60:0/249783074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5590157e20 con 0x7f559015b5e0 2023-11-18T20:43:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.150+0000 7f55877fe640 1 -- 172.21.15.60:0/249783074 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5588014b40 con 0x7f559015b5e0 2023-11-18T20:43:09.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.150+0000 7f55877fe640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f557006ae40 0x7f557006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.150+0000 7f55877fe640 1 -- 172.21.15.60:0/249783074 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5588089fc0 con 0x7f559015b5e0 2023-11-18T20:43:09.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.151+0000 7f558ed9d640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f557006ae40 0x7f557006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.151+0000 7f558ed9d640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f557006ae40 0x7f557006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5580036000 tx=0x7f5580008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.155+0000 7f55877fe640 1 -- 172.21.15.60:0/249783074 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55880bce90 con 0x7f559015b5e0 2023-11-18T20:43:09.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.331+0000 7f5595009640 1 -- 172.21.15.60:0/249783074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5590069400 con 0x7f559015b5e0 2023-11-18T20:43:09.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.332+0000 7f55877fe640 1 -- 172.21.15.60:0/249783074 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5588055100 con 0x7f559015b5e0 2023-11-18T20:43:09.334 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.333+0000 7f5595009640 1 -- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f557006ae40 msgr2=0x7f557006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.333+0000 7f5595009640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f557006ae40 0x7f557006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5580036000 tx=0x7f5580008030 comp rx=0 tx=0).stop 2023-11-18T20:43:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.333+0000 7f5595009640 1 -- 172.21.15.60:0/249783074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559015b5e0 msgr2=0x7f55901e3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.333+0000 7f5595009640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559015b5e0 0x7f55901e3860 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5588007e40 tx=0x7f558800e4b0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.333+0000 7f5595009640 1 -- 172.21.15.60:0/249783074 shutdown_connections 2023-11-18T20:43:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.333+0000 7f5595009640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f557006ae40 0x7f557006d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.333+0000 7f5595009640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559015b5e0 0x7f55901e3860 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.333+0000 7f5595009640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5590155790 0x7f5590153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.333+0000 7f5595009640 1 --2- 172.21.15.60:0/249783074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559006abe0 0x7f559014cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.333+0000 7f5595009640 1 -- 172.21.15.60:0/249783074 >> 172.21.15.60:0/249783074 conn(0x7f55900b1e50 msgr2=0x7f5590071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.334+0000 7f5595009640 1 -- 172.21.15.60:0/249783074 shutdown_connections 2023-11-18T20:43:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.334+0000 7f5595009640 1 -- 172.21.15.60:0/249783074 wait complete. 2023-11-18T20:43:09.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.454+0000 7f9c5ef41640 1 Processor -- start 2023-11-18T20:43:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.454+0000 7f9c5ef41640 1 -- start start 2023-11-18T20:43:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5ef41640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9c580b5370 0x7f9c580cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5ef41640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c580d2040 0x7f9c580cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5ef41640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c580ce4c0 con 0x7f9c5815d2d0 2023-11-18T20:43:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5ef41640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c580d1bb0 con 0x7f9c580d2040 2023-11-18T20:43:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5ef41640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5815d6b0 con 0x7f9c580b5370 2023-11-18T20:43:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5d4de640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c580d2040 0x7f9c580cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5ccdd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9c5815d2d0 0x7f9c580b4c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50144/0 (socket says 172.21.15.60:50144) 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5ccdd640 1 -- 172.21.15.60:0/2258942813 learned_addr learned my addr 172.21.15.60:0/2258942813 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c57fff640 1 --2- 172.21.15.60:0/2258942813 >> v2:172.21.15.100:3300/0 conn(0x7f9c580b5370 0x7f9c580cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5d4de640 1 -- 172.21.15.60:0/2258942813 >> v2:172.21.15.100:3300/0 conn(0x7f9c580b5370 msgr2=0x7f9c580cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5d4de640 1 --2- 172.21.15.60:0/2258942813 >> v2:172.21.15.100:3300/0 conn(0x7f9c580b5370 0x7f9c580cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5d4de640 1 -- 172.21.15.60:0/2258942813 >> v1:172.21.15.60:6789/0 conn(0x7f9c5815d2d0 legacy=0x7f9c580b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c5d4de640 1 -- 172.21.15.60:0/2258942813 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c580ce700 con 0x7f9c580d2040 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.455+0000 7f9c577fe640 1 -- 172.21.15.60:0/2258942813 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482507270 0 0) 0x7f9c4c001110 con 0x7f9c5815d2d0 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.456+0000 7f9c5d4de640 1 --2- 172.21.15.60:0/2258942813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 0x7f9c580cdf80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9c580b2c10 tx=0x7f9c5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178b1978fc3ea1c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.456+0000 7f9c577fe640 1 -- 172.21.15.60:0/2258942813 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5002cc80 con 0x7f9c580d2040 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.456+0000 7f9c577fe640 1 -- 172.21.15.60:0/2258942813 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5002cde0 con 0x7f9c580d2040 2023-11-18T20:43:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.456+0000 7f9c577fe640 1 -- 172.21.15.60:0/2258942813 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50034660 con 0x7f9c580d2040 2023-11-18T20:43:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.457+0000 7f9c5ef41640 1 -- 172.21.15.60:0/2258942813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 msgr2=0x7f9c580cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.457+0000 7f9c5ef41640 1 --2- 172.21.15.60:0/2258942813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 0x7f9c580cdf80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9c580b2c10 tx=0x7f9c5002bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.457+0000 7f9c5ef41640 1 -- 172.21.15.60:0/2258942813 shutdown_connections 2023-11-18T20:43:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.457+0000 7f9c5ef41640 1 --2- 172.21.15.60:0/2258942813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 0x7f9c580cdf80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.457+0000 7f9c5ef41640 1 --2- 172.21.15.60:0/2258942813 >> v2:172.21.15.100:3300/0 conn(0x7f9c580b5370 0x7f9c580cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.457+0000 7f9c5ef41640 1 -- 172.21.15.60:0/2258942813 >> 172.21.15.60:0/2258942813 conn(0x7f9c58072050 msgr2=0x7f9c58072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.457+0000 7f9c5ef41640 1 -- 172.21.15.60:0/2258942813 shutdown_connections 2023-11-18T20:43:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.457+0000 7f9c5ef41640 1 -- 172.21.15.60:0/2258942813 wait complete. 2023-11-18T20:43:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.458+0000 7f9c5ef41640 1 Processor -- start 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.458+0000 7f9c5ef41640 1 -- start start 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c5ef41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c580b5370 0x7f9c580c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c5ef41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 0x7f9c5814a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c5ef41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5815d2d0 0x7f9c5814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c5ef41640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5814ac40 con 0x7f9c580d2040 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c5ef41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580cc130 con 0x7f9c5815d2d0 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c5ef41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580cb090 con 0x7f9c580b5370 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 0x7f9c5814a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 0x7f9c5814a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42726/0 (socket says 172.21.15.60:42726) 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c57fff640 1 -- 172.21.15.60:0/3753334214 learned_addr learned my addr 172.21.15.60:0/3753334214 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c5d4de640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5815d2d0 0x7f9c5814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c5ccdd640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c580b5370 0x7f9c580c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c57fff640 1 -- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c580b5370 msgr2=0x7f9c580c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c57fff640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c580b5370 0x7f9c580c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c57fff640 1 -- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5815d2d0 msgr2=0x7f9c5814a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c57fff640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5815d2d0 0x7f9c5814a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c57fff640 1 -- 172.21.15.60:0/3753334214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c50009a70 con 0x7f9c580d2040 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c5d4de640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5815d2d0 0x7f9c5814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c57fff640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 0x7f9c5814a1c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400d800 tx=0x7f9c4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c557fa640 1 -- 172.21.15.60:0/3753334214 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4400b6c0 con 0x7f9c580d2040 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.459+0000 7f9c557fa640 1 -- 172.21.15.60:0/3753334214 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4400b820 con 0x7f9c580d2040 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.460+0000 7f9c5ef41640 1 -- 172.21.15.60:0/3753334214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c580cb370 con 0x7f9c580d2040 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.460+0000 7f9c557fa640 1 -- 172.21.15.60:0/3753334214 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4400bbb0 con 0x7f9c580d2040 2023-11-18T20:43:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.460+0000 7f9c5ef41640 1 -- 172.21.15.60:0/3753334214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c580cb930 con 0x7f9c580d2040 2023-11-18T20:43:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.460+0000 7f9c5ef41640 1 -- 172.21.15.60:0/3753334214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c58069400 con 0x7f9c580d2040 2023-11-18T20:43:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.461+0000 7f9c557fa640 1 -- 172.21.15.60:0/3753334214 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c4400bd10 con 0x7f9c580d2040 2023-11-18T20:43:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.461+0000 7f9c557fa640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c3c06ae40 0x7f9c3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.461+0000 7f9c557fa640 1 -- 172.21.15.60:0/3753334214 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c44089b20 con 0x7f9c580d2040 2023-11-18T20:43:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.462+0000 7f9c5ccdd640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c3c06ae40 0x7f9c3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.462+0000 7f9c5ccdd640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c3c06ae40 0x7f9c3c06d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c000f60 tx=0x7f9c4c007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:09.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.465+0000 7f9c557fa640 1 -- 172.21.15.60:0/3753334214 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c44054c60 con 0x7f9c580d2040 2023-11-18T20:43:09.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.661+0000 7f9c5ef41640 1 -- 172.21.15.60:0/3753334214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c58072870 con 0x7f9c580d2040 2023-11-18T20:43:09.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.662+0000 7f9c557fa640 1 -- 172.21.15.60:0/3753334214 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c44058280 con 0x7f9c580d2040 2023-11-18T20:43:09.664 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:09.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 -- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c3c06ae40 msgr2=0x7f9c3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c3c06ae40 0x7f9c3c06d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c000f60 tx=0x7f9c4c007c00 comp rx=0 tx=0).stop 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 -- 172.21.15.60:0/3753334214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 msgr2=0x7f9c5814a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 0x7f9c5814a1c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400d800 tx=0x7f9c4400dcc0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 -- 172.21.15.60:0/3753334214 shutdown_connections 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9c3c06ae40 0x7f9c3c06d300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c5815d2d0 0x7f9c5814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c580d2040 0x7f9c5814a1c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 --2- 172.21.15.60:0/3753334214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c580b5370 0x7f9c580c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 -- 172.21.15.60:0/3753334214 >> 172.21.15.60:0/3753334214 conn(0x7f9c58072050 msgr2=0x7f9c580b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 -- 172.21.15.60:0/3753334214 shutdown_connections 2023-11-18T20:43:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.664+0000 7f9c5ef41640 1 -- 172.21.15.60:0/3753334214 wait complete. 2023-11-18T20:43:09.677 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:09.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.782+0000 7fa1bef73640 1 Processor -- start 2023-11-18T20:43:09.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.782+0000 7fa1bef73640 1 -- start start 2023-11-18T20:43:09.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.783+0000 7fa1bef73640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b80b5370 0x7fa1b80cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.783+0000 7fa1bef73640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1b80d2040 0x7fa1b80cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.783+0000 7fa1bef73640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b80ce4c0 con 0x7fa1b815d2d0 2023-11-18T20:43:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.783+0000 7fa1bef73640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b80d1bb0 con 0x7fa1b80d2040 2023-11-18T20:43:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.783+0000 7fa1bef73640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b815d6b0 con 0x7fa1b80b5370 2023-11-18T20:43:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.783+0000 7fa1bd510640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1b80d2040 0x7fa1b80cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.783+0000 7fa1bcd0f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa1b815d2d0 0x7fa1b80b4c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50160/0 (socket says 172.21.15.60:50160) 2023-11-18T20:43:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.783+0000 7fa1bcd0f640 1 -- 172.21.15.60:0/2323068455 learned_addr learned my addr 172.21.15.60:0/2323068455 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.783+0000 7fa1b7fff640 1 --2- 172.21.15.60:0/2323068455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b80b5370 0x7fa1b80cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.784+0000 7fa1bd510640 1 -- 172.21.15.60:0/2323068455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b80b5370 msgr2=0x7fa1b80cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:09.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.784+0000 7fa1bd510640 1 --2- 172.21.15.60:0/2323068455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b80b5370 0x7fa1b80cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.784+0000 7fa1bd510640 1 -- 172.21.15.60:0/2323068455 >> v1:172.21.15.60:6789/0 conn(0x7fa1b815d2d0 legacy=0x7fa1b80b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.784+0000 7fa1b77fe640 1 -- 172.21.15.60:0/2323068455 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049165298 0 0) 0x7fa1b0001490 con 0x7fa1b815d2d0 2023-11-18T20:43:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.784+0000 7fa1bd510640 1 -- 172.21.15.60:0/2323068455 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b80ce700 con 0x7fa1b80d2040 2023-11-18T20:43:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.784+0000 7fa1b7fff640 1 --2- 172.21.15.60:0/2323068455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b80b5370 0x7fa1b80cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1bd510640 1 --2- 172.21.15.60:0/2323068455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80d2040 0x7fa1b80cdf80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00097a0 tx=0x7fa1a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d692d9db2bf176f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1b77fe640 1 -- 172.21.15.60:0/2323068455 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0031830 con 0x7fa1b80d2040 2023-11-18T20:43:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1b77fe640 1 -- 172.21.15.60:0/2323068455 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a0031990 con 0x7fa1b80d2040 2023-11-18T20:43:09.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1b77fe640 1 -- 172.21.15.60:0/2323068455 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0031d20 con 0x7fa1b80d2040 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1bef73640 1 -- 172.21.15.60:0/2323068455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80d2040 msgr2=0x7fa1b80cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1bef73640 1 --2- 172.21.15.60:0/2323068455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80d2040 0x7fa1b80cdf80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00097a0 tx=0x7fa1a002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1bef73640 1 -- 172.21.15.60:0/2323068455 shutdown_connections 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1bef73640 1 --2- 172.21.15.60:0/2323068455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80d2040 0x7fa1b80cdf80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1bef73640 1 --2- 172.21.15.60:0/2323068455 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b80b5370 0x7fa1b80cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1bef73640 1 -- 172.21.15.60:0/2323068455 >> 172.21.15.60:0/2323068455 conn(0x7fa1b8072050 msgr2=0x7fa1b8072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1bef73640 1 -- 172.21.15.60:0/2323068455 shutdown_connections 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.785+0000 7fa1bef73640 1 -- 172.21.15.60:0/2323068455 wait complete. 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.786+0000 7fa1bef73640 1 Processor -- start 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.786+0000 7fa1bef73640 1 -- start start 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.787+0000 7fa1bef73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80b5370 0x7fa1b81533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.787+0000 7fa1bef73640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b80d2040 0x7fa1b80c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.787+0000 7fa1bef73640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b815d2d0 0x7fa1b80c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.787+0000 7fa1bef73640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c7ad0 con 0x7fa1b80b5370 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.787+0000 7fa1bef73640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c7db0 con 0x7fa1b80d2040 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.787+0000 7fa1bef73640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c8090 con 0x7fa1b815d2d0 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.787+0000 7fa1bcd0f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80b5370 0x7fa1b81533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.787+0000 7fa1bcd0f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80b5370 0x7fa1b81533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42742/0 (socket says 172.21.15.60:42742) 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.787+0000 7fa1bcd0f640 1 -- 172.21.15.60:0/3957637424 learned_addr learned my addr 172.21.15.60:0/3957637424 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1bd510640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b815d2d0 0x7fa1b80c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1b7fff640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b80d2040 0x7fa1b80c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1bcd0f640 1 -- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b815d2d0 msgr2=0x7fa1b80c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1bcd0f640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b815d2d0 0x7fa1b80c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1bcd0f640 1 -- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b80d2040 msgr2=0x7fa1b80c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1bcd0f640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b80d2040 0x7fa1b80c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1bcd0f640 1 -- 172.21.15.60:0/3957637424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a0009400 con 0x7fa1b80b5370 2023-11-18T20:43:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1bcd0f640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80b5370 0x7fa1b81533a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000c7c0 tx=0x7fa1b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1b57fa640 1 -- 172.21.15.60:0/3957637424 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b000a640 con 0x7fa1b80b5370 2023-11-18T20:43:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1b57fa640 1 -- 172.21.15.60:0/3957637424 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b000a7c0 con 0x7fa1b80b5370 2023-11-18T20:43:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1bef73640 1 -- 172.21.15.60:0/3957637424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b80c8370 con 0x7fa1b80b5370 2023-11-18T20:43:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1b57fa640 1 -- 172.21.15.60:0/3957637424 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b000ab70 con 0x7fa1b80b5370 2023-11-18T20:43:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.788+0000 7fa1bef73640 1 -- 172.21.15.60:0/3957637424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b81538e0 con 0x7fa1b80b5370 2023-11-18T20:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.789+0000 7fa1bef73640 1 -- 172.21.15.60:0/3957637424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b8069400 con 0x7fa1b80b5370 2023-11-18T20:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.790+0000 7fa1b57fa640 1 -- 172.21.15.60:0/3957637424 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1b000acd0 con 0x7fa1b80b5370 2023-11-18T20:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.790+0000 7fa1b57fa640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa19806ae40 0x7fa19806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.790+0000 7fa1b57fa640 1 -- 172.21.15.60:0/3957637424 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1b0088a40 con 0x7fa1b80b5370 2023-11-18T20:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.790+0000 7fa1b7fff640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa19806ae40 0x7fa19806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.791+0000 7fa1b7fff640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa19806ae40 0x7fa19806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80d2440 tx=0x7fa1a8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:09.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.794+0000 7fa1b57fa640 1 -- 172.21.15.60:0/3957637424 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1b008ae60 con 0x7fa1b80b5370 2023-11-18T20:43:09.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.984+0000 7fa1bef73640 1 -- 172.21.15.60:0/3957637424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa1b80ce4c0 con 0x7fa1b80b5370 2023-11-18T20:43:09.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.984+0000 7fa1b57fa640 1 -- 172.21.15.60:0/3957637424 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa1b80ce4c0 con 0x7fa1b80b5370 2023-11-18T20:43:09.986 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:09.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.986+0000 7fa1bef73640 1 -- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa19806ae40 msgr2=0x7fa19806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.986+0000 7fa1bef73640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa19806ae40 0x7fa19806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80d2440 tx=0x7fa1a8008040 comp rx=0 tx=0).stop 2023-11-18T20:43:09.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.986+0000 7fa1bef73640 1 -- 172.21.15.60:0/3957637424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80b5370 msgr2=0x7fa1b81533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:09.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.986+0000 7fa1bef73640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80b5370 0x7fa1b81533a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000c7c0 tx=0x7fa1b000cc80 comp rx=0 tx=0).stop 2023-11-18T20:43:09.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.986+0000 7fa1bef73640 1 -- 172.21.15.60:0/3957637424 shutdown_connections 2023-11-18T20:43:09.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.986+0000 7fa1bef73640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa19806ae40 0x7fa19806d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.986+0000 7fa1bef73640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1b815d2d0 0x7fa1b80c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.986+0000 7fa1bef73640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1b80d2040 0x7fa1b80c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.986+0000 7fa1bef73640 1 --2- 172.21.15.60:0/3957637424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1b80b5370 0x7fa1b81533a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:09.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.986+0000 7fa1bef73640 1 -- 172.21.15.60:0/3957637424 >> 172.21.15.60:0/3957637424 conn(0x7fa1b8072050 msgr2=0x7fa1b815f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:09.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.987+0000 7fa1bef73640 1 -- 172.21.15.60:0/3957637424 shutdown_connections 2023-11-18T20:43:09.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:09.987+0000 7fa1bef73640 1 -- 172.21.15.60:0/3957637424 wait complete. 2023-11-18T20:43:09.999 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-11-18T20:43:10.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:10.000 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:10.031 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:43:10.031 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:43:10.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:10.032 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-18T20:43:10.094 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:43:10.095 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:10.115 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:43:10.115 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:10.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:10.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.282+0000 7fbc5da0e640 1 Processor -- start 2023-11-18T20:43:10.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.282+0000 7fbc5da0e640 1 -- start start 2023-11-18T20:43:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.283+0000 7fbc5da0e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbc580ceb70 0x7fbc580cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.283+0000 7fbc5da0e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc580b4370 0x7fbc580b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.283+0000 7fbc5da0e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc58071770 con 0x7fbc58144980 2023-11-18T20:43:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.283+0000 7fbc5da0e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5815ced0 con 0x7fbc580ceb70 2023-11-18T20:43:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.283+0000 7fbc5da0e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc580b4ce0 con 0x7fbc580b4370 2023-11-18T20:43:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.283+0000 7fbc56ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbc58144980 0x7fbc5815c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50176/0 (socket says 172.21.15.60:50176) 2023-11-18T20:43:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.283+0000 7fbc56ffd640 1 -- 172.21.15.60:0/2217392910 learned_addr learned my addr 172.21.15.60:0/2217392910 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc567fc640 1 --2- 172.21.15.60:0/2217392910 >> v2:172.21.15.100:3300/0 conn(0x7fbc580ceb70 0x7fbc580cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc577fe640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc580b4370 0x7fbc580b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730061184 0 0) 0x7fbc58071770 con 0x7fbc58144980 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc44003520 con 0x7fbc58144980 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1794565352 0 0) 0x7fbc44003520 con 0x7fbc58144980 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc580b4370 msgr2=0x7fbc580b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc580b4370 0x7fbc580b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 >> v2:172.21.15.100:3300/0 conn(0x7fbc580ceb70 msgr2=0x7fbc580cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 --2- 172.21.15.60:0/2217392910 >> v2:172.21.15.100:3300/0 conn(0x7fbc580ceb70 0x7fbc580cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc580ccdc0 con 0x7fbc58144980 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fbc50001670 con 0x7fbc58144980 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc567fc640 1 --2- 172.21.15.60:0/2217392910 >> v2:172.21.15.100:3300/0 conn(0x7fbc580ceb70 0x7fbc580cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 >> v1:172.21.15.60:6789/0 conn(0x7fbc58144980 legacy=0x7fbc5815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc580b4370 0x7fbc44003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc44003ef0 0x7fbc440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc44004d00 0x7fbc44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5815ced0 con 0x7fbc44003ef0 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc580b4ce0 con 0x7fbc44004d00 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4400b980 con 0x7fbc580b4370 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc567fc640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc44003ef0 0x7fbc440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc56ffd640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc580b4370 0x7fbc44003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc577fe640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc44004d00 0x7fbc44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc567fc640 1 -- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc580b4370 msgr2=0x7fbc44003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc567fc640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc580b4370 0x7fbc44003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc567fc640 1 -- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc44004d00 msgr2=0x7fbc44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc567fc640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc44004d00 0x7fbc44005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc567fc640 1 -- 172.21.15.60:0/2217392910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44003520 con 0x7fbc44003ef0 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.284+0000 7fbc56ffd640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc580b4370 0x7fbc44003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.285+0000 7fbc567fc640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc44003ef0 0x7fbc440047c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009c90 tx=0x7fbc440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781a127b0f000bfe server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.285+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4002c980 con 0x7fbc44003ef0 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.285+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4002cae0 con 0x7fbc44003ef0 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.285+0000 7fbc55ffb640 1 -- 172.21.15.60:0/2217392910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40034440 con 0x7fbc44003ef0 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.286+0000 7fbc5da0e640 1 -- 172.21.15.60:0/2217392910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc44003ef0 msgr2=0x7fbc440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.286+0000 7fbc5da0e640 1 --2- 172.21.15.60:0/2217392910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc44003ef0 0x7fbc440047c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009c90 tx=0x7fbc440057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.286+0000 7fbc577fe640 1 -- 172.21.15.60:0/2217392910 reap_dead start 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.286+0000 7fbc5da0e640 1 -- 172.21.15.60:0/2217392910 shutdown_connections 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.286+0000 7fbc5da0e640 1 -- 172.21.15.60:0/2217392910 >> 172.21.15.60:0/2217392910 conn(0x7fbc5806a4d0 msgr2=0x7fbc5806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.286+0000 7fbc5da0e640 1 -- 172.21.15.60:0/2217392910 shutdown_connections 2023-11-18T20:43:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.286+0000 7fbc5da0e640 1 -- 172.21.15.60:0/2217392910 wait complete. 2023-11-18T20:43:10.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.287+0000 7fbc5da0e640 1 Processor -- start 2023-11-18T20:43:10.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.287+0000 7fbc5da0e640 1 -- start start 2023-11-18T20:43:10.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc5da0e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc44003ef0 0x7fbc58158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc5da0e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc58137180 0x7fbc5813bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc5da0e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc5813c170 0x7fbc5813c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc5da0e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc58140fa0 con 0x7fbc5813c170 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc5da0e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc58155e40 con 0x7fbc44003ef0 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc5da0e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc581409e0 con 0x7fbc58137180 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc5813c170 0x7fbc5813c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc5813c170 0x7fbc5813c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42758/0 (socket says 172.21.15.60:42758) 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc577fe640 1 -- 172.21.15.60:0/1941958860 learned_addr learned my addr 172.21.15.60:0/1941958860 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc56ffd640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc44003ef0 0x7fbc58158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc567fc640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc58137180 0x7fbc5813bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc577fe640 1 -- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc58137180 msgr2=0x7fbc5813bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc577fe640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc58137180 0x7fbc5813bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc577fe640 1 -- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc44003ef0 msgr2=0x7fbc58158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc577fe640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc44003ef0 0x7fbc58158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.288+0000 7fbc577fe640 1 -- 172.21.15.60:0/1941958860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58140cc0 con 0x7fbc5813c170 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.289+0000 7fbc577fe640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc5813c170 0x7fbc5813c5f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc5806fac0 tx=0x7fbc4800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.289+0000 7fbc4ffff640 1 -- 172.21.15.60:0/1941958860 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4800e040 con 0x7fbc5813c170 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.289+0000 7fbc4ffff640 1 -- 172.21.15.60:0/1941958860 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4800f040 con 0x7fbc5813c170 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.289+0000 7fbc4ffff640 1 -- 172.21.15.60:0/1941958860 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48015430 con 0x7fbc5813c170 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.289+0000 7fbc5da0e640 1 -- 172.21.15.60:0/1941958860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc581560e0 con 0x7fbc5813c170 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.289+0000 7fbc5da0e640 1 -- 172.21.15.60:0/1941958860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc581377c0 con 0x7fbc5813c170 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.290+0000 7fbc5da0e640 1 -- 172.21.15.60:0/1941958860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc58069400 con 0x7fbc5813c170 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.290+0000 7fbc4ffff640 1 -- 172.21.15.60:0/1941958860 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc48015590 con 0x7fbc5813c170 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.291+0000 7fbc4ffff640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbc3806ae40 0x7fbc3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.291+0000 7fbc4ffff640 1 -- 172.21.15.60:0/1941958860 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc48089d00 con 0x7fbc5813c170 2023-11-18T20:43:10.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.291+0000 7fbc56ffd640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbc3806ae40 0x7fbc3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.292+0000 7fbc56ffd640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbc3806ae40 0x7fbc3806d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbc50001050 tx=0x7fbc50009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:10.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.295+0000 7fbc4ffff640 1 -- 172.21.15.60:0/1941958860 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc48054db0 con 0x7fbc5813c170 2023-11-18T20:43:10.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.471+0000 7fbc5da0e640 1 -- 172.21.15.60:0/1941958860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc58143d30 con 0x7fbc5813c170 2023-11-18T20:43:10.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.472+0000 7fbc4ffff640 1 -- 172.21.15.60:0/1941958860 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fbc480583d0 con 0x7fbc5813c170 2023-11-18T20:43:10.475 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:10.475 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:10.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.474+0000 7fbc5da0e640 1 -- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbc3806ae40 msgr2=0x7fbc3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.474+0000 7fbc5da0e640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbc3806ae40 0x7fbc3806d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbc50001050 tx=0x7fbc50009040 comp rx=0 tx=0).stop 2023-11-18T20:43:10.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.474+0000 7fbc5da0e640 1 -- 172.21.15.60:0/1941958860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc5813c170 msgr2=0x7fbc5813c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.474+0000 7fbc5da0e640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc5813c170 0x7fbc5813c5f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc5806fac0 tx=0x7fbc4800f580 comp rx=0 tx=0).stop 2023-11-18T20:43:10.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.475+0000 7fbc5da0e640 1 -- 172.21.15.60:0/1941958860 shutdown_connections 2023-11-18T20:43:10.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.475+0000 7fbc5da0e640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbc3806ae40 0x7fbc3806d300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.475+0000 7fbc5da0e640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc5813c170 0x7fbc5813c5f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.475+0000 7fbc5da0e640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc58137180 0x7fbc5813bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.475+0000 7fbc5da0e640 1 --2- 172.21.15.60:0/1941958860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc44003ef0 0x7fbc58158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.475+0000 7fbc5da0e640 1 -- 172.21.15.60:0/1941958860 >> 172.21.15.60:0/1941958860 conn(0x7fbc5806a4d0 msgr2=0x7fbc5815ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:10.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.475+0000 7fbc5da0e640 1 -- 172.21.15.60:0/1941958860 shutdown_connections 2023-11-18T20:43:10.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.475+0000 7fbc5da0e640 1 -- 172.21.15.60:0/1941958860 wait complete. 2023-11-18T20:43:10.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:10.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:10.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.596+0000 7f54f9373640 1 Processor -- start 2023-11-18T20:43:10.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.596+0000 7f54f9373640 1 -- start start 2023-11-18T20:43:10.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f9373640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f54f41415f0 0x7f54f4141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f9373640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f54f406bbf0 0x7f54f40b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f9373640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f40b2ef0 con 0x7f54f4144a00 2023-11-18T20:43:10.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f9373640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f40b3b70 con 0x7f54f41415f0 2023-11-18T20:43:10.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f9373640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f40b31d0 con 0x7f54f406bbf0 2023-11-18T20:43:10.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f2ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f54f41415f0 0x7f54f4141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f2ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f54f41415f0 0x7f54f4141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48872/0 (socket says 172.21.15.60:48872) 2023-11-18T20:43:10.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f2ffd640 1 -- 172.21.15.60:0/1797191737 learned_addr learned my addr 172.21.15.60:0/1797191737 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:10.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f2ffd640 1 -- 172.21.15.60:0/1797191737 >> v2:172.21.15.100:3300/0 conn(0x7f54f406bbf0 msgr2=0x7f54f40b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:10.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f37fe640 1 --2- 172.21.15.60:0/1797191737 >> v2:172.21.15.100:3300/0 conn(0x7f54f406bbf0 0x7f54f40b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f2ffd640 1 --2- 172.21.15.60:0/1797191737 >> v2:172.21.15.100:3300/0 conn(0x7f54f406bbf0 0x7f54f40b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f2ffd640 1 -- 172.21.15.60:0/1797191737 >> v1:172.21.15.100:6789/0 conn(0x7f54f4144a00 legacy=0x7f54f406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:10.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f2ffd640 1 -- 172.21.15.60:0/1797191737 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f40b34b0 con 0x7f54f41415f0 2023-11-18T20:43:10.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.597+0000 7f54f37fe640 1 --2- 172.21.15.60:0/1797191737 >> v2:172.21.15.100:3300/0 conn(0x7f54f406bbf0 0x7f54f40b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:10.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.598+0000 7f54f2ffd640 1 --2- 172.21.15.60:0/1797191737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f41415f0 0x7f54f4141e00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f54f415ebf0 tx=0x7f54e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b245d35744c409 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:10.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.598+0000 7f54f27fc640 1 -- 172.21.15.60:0/1797191737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4031870 con 0x7f54f41415f0 2023-11-18T20:43:10.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.598+0000 7f54f27fc640 1 -- 172.21.15.60:0/1797191737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e40319d0 con 0x7f54f41415f0 2023-11-18T20:43:10.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.598+0000 7f54f27fc640 1 -- 172.21.15.60:0/1797191737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4031d60 con 0x7f54f41415f0 2023-11-18T20:43:10.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.599+0000 7f54f9373640 1 -- 172.21.15.60:0/1797191737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f41415f0 msgr2=0x7f54f4141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.599+0000 7f54f9373640 1 --2- 172.21.15.60:0/1797191737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f41415f0 0x7f54f4141e00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f54f415ebf0 tx=0x7f54e402b910 comp rx=0 tx=0).stop 2023-11-18T20:43:10.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.599+0000 7f54f9373640 1 -- 172.21.15.60:0/1797191737 shutdown_connections 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.599+0000 7f54f9373640 1 --2- 172.21.15.60:0/1797191737 >> v2:172.21.15.100:3300/0 conn(0x7f54f406bbf0 0x7f54f40b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.599+0000 7f54f9373640 1 --2- 172.21.15.60:0/1797191737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f41415f0 0x7f54f4141e00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.599+0000 7f54f9373640 1 -- 172.21.15.60:0/1797191737 >> 172.21.15.60:0/1797191737 conn(0x7f54f4070e70 msgr2=0x7f54f4071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.599+0000 7f54f9373640 1 -- 172.21.15.60:0/1797191737 shutdown_connections 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.599+0000 7f54f9373640 1 -- 172.21.15.60:0/1797191737 wait complete. 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.600+0000 7f54f9373640 1 Processor -- start 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.600+0000 7f54f9373640 1 -- start start 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f9373640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54f406bbf0 0x7f54f4140430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f9373640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54f41415f0 0x7f54f415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f9373640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f4144a00 0x7f54f4157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f9373640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f406f080 con 0x7f54f4144a00 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f9373640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41550e0 con 0x7f54f41415f0 2023-11-18T20:43:10.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f9373640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f4140970 con 0x7f54f406bbf0 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f4144a00 0x7f54f4157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f4144a00 0x7f54f4157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48876/0 (socket says 172.21.15.60:48876) 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f37fe640 1 -- 172.21.15.60:0/4232530876 learned_addr learned my addr 172.21.15.60:0/4232530876 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f37fe640 1 -- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54f406bbf0 msgr2=0x7f54f4140430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54ebfff640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54f41415f0 0x7f54f415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f37fe640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54f406bbf0 0x7f54f4140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f37fe640 1 -- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54f41415f0 msgr2=0x7f54f415a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f37fe640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54f41415f0 0x7f54f415a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f37fe640 1 -- 172.21.15.60:0/4232530876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40094d0 con 0x7f54f4144a00 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54ebfff640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54f41415f0 0x7f54f415a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.601+0000 7f54f37fe640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f4144a00 0x7f54f4157f00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00cf20 tx=0x7f54ec00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.602+0000 7f54eb7fe640 1 -- 172.21.15.60:0/4232530876 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00d840 con 0x7f54f4144a00 2023-11-18T20:43:10.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.602+0000 7f54eb7fe640 1 -- 172.21.15.60:0/4232530876 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec00d9a0 con 0x7f54f4144a00 2023-11-18T20:43:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.602+0000 7f54f9373640 1 -- 172.21.15.60:0/4232530876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f406e7e0 con 0x7f54f4144a00 2023-11-18T20:43:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.602+0000 7f54eb7fe640 1 -- 172.21.15.60:0/4232530876 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00dd50 con 0x7f54f4144a00 2023-11-18T20:43:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.602+0000 7f54f9373640 1 -- 172.21.15.60:0/4232530876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f406eda0 con 0x7f54f4144a00 2023-11-18T20:43:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.603+0000 7f54f9373640 1 -- 172.21.15.60:0/4232530876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f4154eb0 con 0x7f54f4144a00 2023-11-18T20:43:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.603+0000 7f54eb7fe640 1 -- 172.21.15.60:0/4232530876 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f54ec01e3b0 con 0x7f54f4144a00 2023-11-18T20:43:10.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.603+0000 7f54eb7fe640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f54d406ae40 0x7f54d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.604+0000 7f54eb7fe640 1 -- 172.21.15.60:0/4232530876 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54ec08a0d0 con 0x7f54f4144a00 2023-11-18T20:43:10.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.604+0000 7f54f2ffd640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f54d406ae40 0x7f54d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.604+0000 7f54f2ffd640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f54d406ae40 0x7f54d406d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f54e4036000 tx=0x7f54e4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:10.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.608+0000 7f54eb7fe640 1 -- 172.21.15.60:0/4232530876 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54f4154eb0 con 0x7f54f4144a00 2023-11-18T20:43:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.784+0000 7f54f9373640 1 -- 172.21.15.60:0/4232530876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54f4154eb0 con 0x7f54f4144a00 2023-11-18T20:43:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.784+0000 7f54eb7fe640 1 -- 172.21.15.60:0/4232530876 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54f4154eb0 con 0x7f54f4144a00 2023-11-18T20:43:10.786 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 -- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f54d406ae40 msgr2=0x7f54d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f54d406ae40 0x7f54d406d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f54e4036000 tx=0x7f54e4030000 comp rx=0 tx=0).stop 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 -- 172.21.15.60:0/4232530876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f4144a00 msgr2=0x7f54f4157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f4144a00 0x7f54f4157f00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00cf20 tx=0x7f54ec00a7d0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 -- 172.21.15.60:0/4232530876 shutdown_connections 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f54d406ae40 0x7f54d406d300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54f4144a00 0x7f54f4157f00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54f41415f0 0x7f54f415a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 --2- 172.21.15.60:0/4232530876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54f406bbf0 0x7f54f4140430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 -- 172.21.15.60:0/4232530876 >> 172.21.15.60:0/4232530876 conn(0x7f54f4070e70 msgr2=0x7f54f4153230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:10.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 -- 172.21.15.60:0/4232530876 shutdown_connections 2023-11-18T20:43:10.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.786+0000 7f54f9373640 1 -- 172.21.15.60:0/4232530876 wait complete. 2023-11-18T20:43:10.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.906+0000 7f42863fb640 1 Processor -- start 2023-11-18T20:43:10.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.906+0000 7f42863fb640 1 -- start start 2023-11-18T20:43:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f42863fb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f428015d1d0 0x7f428006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f42863fb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42800ad640 0x7f428006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f42863fb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42800aeb80 0x7f4280160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f42863fb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42800adae0 con 0x7f42800ad640 2023-11-18T20:43:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f42863fb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42800addc0 con 0x7f428015d1d0 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f42863fb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42800ae0a0 con 0x7f42800aeb80 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f427f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42800ad640 0x7f428006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f427f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42800ad640 0x7f428006ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48890/0 (socket says 172.21.15.60:48890) 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f427f7fe640 1 -- 172.21.15.60:0/1713156701 learned_addr learned my addr 172.21.15.60:0/1713156701 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f427ffff640 1 --2- 172.21.15.60:0/1713156701 >> v2:172.21.15.100:3300/0 conn(0x7f428015d1d0 0x7f428006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.907+0000 7f4284998640 1 --2- 172.21.15.60:0/1713156701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42800aeb80 0x7f4280160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.908+0000 7f427f7fe640 1 -- 172.21.15.60:0/1713156701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42800aeb80 msgr2=0x7f4280160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.908+0000 7f427f7fe640 1 --2- 172.21.15.60:0/1713156701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42800aeb80 0x7f4280160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.908+0000 7f427f7fe640 1 -- 172.21.15.60:0/1713156701 >> v2:172.21.15.100:3300/0 conn(0x7f428015d1d0 msgr2=0x7f428006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.908+0000 7f427f7fe640 1 --2- 172.21.15.60:0/1713156701 >> v2:172.21.15.100:3300/0 conn(0x7f428015d1d0 0x7f428006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.908+0000 7f427f7fe640 1 -- 172.21.15.60:0/1713156701 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42800ae380 con 0x7f42800ad640 2023-11-18T20:43:10.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.908+0000 7f427f7fe640 1 --2- 172.21.15.60:0/1713156701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42800ad640 0x7f428006ba20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4280072310 tx=0x7f427002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82532430378f5719 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.908+0000 7f427effd640 1 -- 172.21.15.60:0/1713156701 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270031870 con 0x7f42800ad640 2023-11-18T20:43:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f427effd640 1 -- 172.21.15.60:0/1713156701 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42700319d0 con 0x7f42800ad640 2023-11-18T20:43:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f427effd640 1 -- 172.21.15.60:0/1713156701 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270031d60 con 0x7f42800ad640 2023-11-18T20:43:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f42863fb640 1 -- 172.21.15.60:0/1713156701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42800ad640 msgr2=0x7f428006ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f42863fb640 1 --2- 172.21.15.60:0/1713156701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42800ad640 0x7f428006ba20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4280072310 tx=0x7f427002b910 comp rx=0 tx=0).stop 2023-11-18T20:43:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f42863fb640 1 -- 172.21.15.60:0/1713156701 shutdown_connections 2023-11-18T20:43:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f42863fb640 1 --2- 172.21.15.60:0/1713156701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42800aeb80 0x7f4280160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f42863fb640 1 --2- 172.21.15.60:0/1713156701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42800ad640 0x7f428006ba20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f42863fb640 1 --2- 172.21.15.60:0/1713156701 >> v2:172.21.15.100:3300/0 conn(0x7f428015d1d0 0x7f428006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f42863fb640 1 -- 172.21.15.60:0/1713156701 >> 172.21.15.60:0/1713156701 conn(0x7f4280070e70 msgr2=0x7f4280071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f42863fb640 1 -- 172.21.15.60:0/1713156701 shutdown_connections 2023-11-18T20:43:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.909+0000 7f42863fb640 1 -- 172.21.15.60:0/1713156701 wait complete. 2023-11-18T20:43:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.910+0000 7f42863fb640 1 Processor -- start 2023-11-18T20:43:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.910+0000 7f42863fb640 1 -- start start 2023-11-18T20:43:10.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f42863fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800ad640 0x7f428013da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f42863fb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42800aeb80 0x7f428015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f42863fb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f428015d1d0 0x7f42801403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f42863fb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4280160960 con 0x7f428015d1d0 2023-11-18T20:43:10.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f42863fb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42801600c0 con 0x7f42800ad640 2023-11-18T20:43:10.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f42863fb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42801603a0 con 0x7f42800aeb80 2023-11-18T20:43:10.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f4284998640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f428015d1d0 0x7f42801403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f4284998640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f428015d1d0 0x7f42801403e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48902/0 (socket says 172.21.15.60:48902) 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f4284998640 1 -- 172.21.15.60:0/2679230426 learned_addr learned my addr 172.21.15.60:0/2679230426 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f427f7fe640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42800aeb80 0x7f428015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.911+0000 7f427ffff640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800ad640 0x7f428013da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f4284998640 1 -- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42800aeb80 msgr2=0x7f428015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f4284998640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42800aeb80 0x7f428015d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f4284998640 1 -- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800ad640 msgr2=0x7f428013da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f4284998640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800ad640 0x7f428013da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f4284998640 1 -- 172.21.15.60:0/2679230426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42700094d0 con 0x7f428015d1d0 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f4284998640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f428015d1d0 0x7f42801403e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4278007e60 tx=0x7f427800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f427cff9640 1 -- 172.21.15.60:0/2679230426 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278014620 con 0x7f428015d1d0 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f427cff9640 1 -- 172.21.15.60:0/2679230426 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427800ee90 con 0x7f428015d1d0 2023-11-18T20:43:10.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f42863fb640 1 -- 172.21.15.60:0/2679230426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4280160680 con 0x7f428015d1d0 2023-11-18T20:43:10.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f427cff9640 1 -- 172.21.15.60:0/2679230426 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42780149e0 con 0x7f428015d1d0 2023-11-18T20:43:10.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.912+0000 7f42863fb640 1 -- 172.21.15.60:0/2679230426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428013df80 con 0x7f428015d1d0 2023-11-18T20:43:10.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.913+0000 7f42863fb640 1 -- 172.21.15.60:0/2679230426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428015ab30 con 0x7f428015d1d0 2023-11-18T20:43:10.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.913+0000 7f427cff9640 1 -- 172.21.15.60:0/2679230426 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4278014b40 con 0x7f428015d1d0 2023-11-18T20:43:10.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.914+0000 7f427cff9640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f426c06ae40 0x7f426c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:10.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.914+0000 7f427cff9640 1 -- 172.21.15.60:0/2679230426 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4278089f00 con 0x7f428015d1d0 2023-11-18T20:43:10.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.914+0000 7f427ffff640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f426c06ae40 0x7f426c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:10.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.915+0000 7f427ffff640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f426c06ae40 0x7f426c06d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4274006fd0 tx=0x7f4274008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:10.918+0000 7f427cff9640 1 -- 172.21.15.60:0/2679230426 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f428015ab30 con 0x7f428015d1d0 2023-11-18T20:43:11.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.094+0000 7f42863fb640 1 -- 172.21.15.60:0/2679230426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f428015ab30 con 0x7f428015d1d0 2023-11-18T20:43:11.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.094+0000 7f427cff9640 1 -- 172.21.15.60:0/2679230426 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f428015ab30 con 0x7f428015d1d0 2023-11-18T20:43:11.097 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:11.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 -- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f426c06ae40 msgr2=0x7f426c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f426c06ae40 0x7f426c06d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4274006fd0 tx=0x7f4274008040 comp rx=0 tx=0).stop 2023-11-18T20:43:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 -- 172.21.15.60:0/2679230426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f428015d1d0 msgr2=0x7f42801403e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f428015d1d0 0x7f42801403e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4278007e60 tx=0x7f427800e4b0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 -- 172.21.15.60:0/2679230426 shutdown_connections 2023-11-18T20:43:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f426c06ae40 0x7f426c06d300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f428015d1d0 0x7f42801403e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42800aeb80 0x7f428015d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 --2- 172.21.15.60:0/2679230426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42800ad640 0x7f428013da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 -- 172.21.15.60:0/2679230426 >> 172.21.15.60:0/2679230426 conn(0x7f4280070e70 msgr2=0x7f428006d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 -- 172.21.15.60:0/2679230426 shutdown_connections 2023-11-18T20:43:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.096+0000 7f42863fb640 1 -- 172.21.15.60:0/2679230426 wait complete. 2023-11-18T20:43:11.109 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:11.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.218+0000 7f368610c640 1 Processor -- start 2023-11-18T20:43:11.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.218+0000 7f368610c640 1 -- start start 2023-11-18T20:43:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.219+0000 7f368610c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f368015d2d0 0x7f36800b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.219+0000 7f368610c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36800ce150 0x7f36800b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.219+0000 7f368610c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800d1bb0 con 0x7f36800d2040 2023-11-18T20:43:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.219+0000 7f368610c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368015d6b0 con 0x7f368015d2d0 2023-11-18T20:43:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.219+0000 7f368610c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800ce700 con 0x7f36800ce150 2023-11-18T20:43:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.219+0000 7f367f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f368015d2d0 0x7f36800b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.219+0000 7f367f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f368015d2d0 0x7f36800b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48916/0 (socket says 172.21.15.60:48916) 2023-11-18T20:43:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.219+0000 7f367f7fe640 1 -- 172.21.15.60:0/1166758929 learned_addr learned my addr 172.21.15.60:0/1166758929 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.219+0000 7f367f7fe640 1 -- 172.21.15.60:0/1166758929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36800ce150 msgr2=0x7f36800b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.219+0000 7f367f7fe640 1 --2- 172.21.15.60:0/1166758929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36800ce150 0x7f36800b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.220+0000 7f367f7fe640 1 -- 172.21.15.60:0/1166758929 >> v1:172.21.15.100:6789/0 conn(0x7f36800d2040 legacy=0x7f36800cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:11.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.220+0000 7f367f7fe640 1 -- 172.21.15.60:0/1166758929 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800d1280 con 0x7f368015d2d0 2023-11-18T20:43:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.220+0000 7f367f7fe640 1 --2- 172.21.15.60:0/1166758929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368015d2d0 0x7f36800b4c60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3670009870 tx=0x7f367002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5673cb033e5cd8ca server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.220+0000 7f367e7fc640 1 -- 172.21.15.60:0/1166758929 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670031640 con 0x7f368015d2d0 2023-11-18T20:43:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.220+0000 7f367e7fc640 1 -- 172.21.15.60:0/1166758929 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36700317a0 con 0x7f368015d2d0 2023-11-18T20:43:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.220+0000 7f367e7fc640 1 -- 172.21.15.60:0/1166758929 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670031b30 con 0x7f368015d2d0 2023-11-18T20:43:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.221+0000 7f368610c640 1 -- 172.21.15.60:0/1166758929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368015d2d0 msgr2=0x7f36800b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.221+0000 7f368610c640 1 --2- 172.21.15.60:0/1166758929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368015d2d0 0x7f36800b4c60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3670009870 tx=0x7f367002b8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.221+0000 7f368610c640 1 -- 172.21.15.60:0/1166758929 shutdown_connections 2023-11-18T20:43:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.221+0000 7f368610c640 1 --2- 172.21.15.60:0/1166758929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36800ce150 0x7f36800b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.221+0000 7f368610c640 1 --2- 172.21.15.60:0/1166758929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f368015d2d0 0x7f36800b4c60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.221+0000 7f368610c640 1 -- 172.21.15.60:0/1166758929 >> 172.21.15.60:0/1166758929 conn(0x7f3680072050 msgr2=0x7f3680072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:11.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.221+0000 7f368610c640 1 -- 172.21.15.60:0/1166758929 shutdown_connections 2023-11-18T20:43:11.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.221+0000 7f368610c640 1 -- 172.21.15.60:0/1166758929 wait complete. 2023-11-18T20:43:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.222+0000 7f368610c640 1 Processor -- start 2023-11-18T20:43:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.222+0000 7f368610c640 1 -- start start 2023-11-18T20:43:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f368610c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36800ce150 0x7f36800c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f368610c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36800d2040 0x7f3680159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f368610c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f368015d2d0 0x7f36800b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f368610c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800c7e50 con 0x7f36800d2040 2023-11-18T20:43:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f368610c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800c72d0 con 0x7f36800ce150 2023-11-18T20:43:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f368610c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800c75b0 con 0x7f368015d2d0 2023-11-18T20:43:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36800d2040 0x7f3680159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36800d2040 0x7f3680159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48930/0 (socket says 172.21.15.60:48930) 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367effd640 1 -- 172.21.15.60:0/3539284360 learned_addr learned my addr 172.21.15.60:0/3539284360 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367f7fe640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36800ce150 0x7f36800c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367ffff640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f368015d2d0 0x7f36800b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367effd640 1 -- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f368015d2d0 msgr2=0x7f36800b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367effd640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f368015d2d0 0x7f36800b7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367effd640 1 -- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36800ce150 msgr2=0x7f36800c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367effd640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36800ce150 0x7f36800c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367effd640 1 -- 172.21.15.60:0/3539284360 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36700094d0 con 0x7f36800d2040 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367f7fe640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36800ce150 0x7f36800c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.223+0000 7f367ffff640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f368015d2d0 0x7f36800b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.224+0000 7f367effd640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36800d2040 0x7f3680159d00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f367400c410 tx=0x7f367400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.224+0000 7f3667fff640 1 -- 172.21.15.60:0/3539284360 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36740035f0 con 0x7f36800d2040 2023-11-18T20:43:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.224+0000 7f3667fff640 1 -- 172.21.15.60:0/3539284360 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3674003750 con 0x7f36800d2040 2023-11-18T20:43:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.224+0000 7f368610c640 1 -- 172.21.15.60:0/3539284360 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36800c7890 con 0x7f36800d2040 2023-11-18T20:43:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.224+0000 7f3667fff640 1 -- 172.21.15.60:0/3539284360 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674003b40 con 0x7f36800d2040 2023-11-18T20:43:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.224+0000 7f368610c640 1 -- 172.21.15.60:0/3539284360 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368015a240 con 0x7f36800d2040 2023-11-18T20:43:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.225+0000 7f368610c640 1 -- 172.21.15.60:0/3539284360 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36800ca100 con 0x7f36800d2040 2023-11-18T20:43:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.225+0000 7f3667fff640 1 -- 172.21.15.60:0/3539284360 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3674003ca0 con 0x7f36800d2040 2023-11-18T20:43:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.226+0000 7f3667fff640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f366c06ae40 0x7f366c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.226+0000 7f3667fff640 1 -- 172.21.15.60:0/3539284360 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3674088bd0 con 0x7f36800d2040 2023-11-18T20:43:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.226+0000 7f367f7fe640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f366c06ae40 0x7f366c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.226+0000 7f367f7fe640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f366c06ae40 0x7f366c06d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3670036000 tx=0x7f367003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:11.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.230+0000 7f3667fff640 1 -- 172.21.15.60:0/3539284360 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3674053d10 con 0x7f36800d2040 2023-11-18T20:43:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.408+0000 7f368610c640 1 -- 172.21.15.60:0/3539284360 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3680069400 con 0x7f36800d2040 2023-11-18T20:43:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.408+0000 7f3667fff640 1 -- 172.21.15.60:0/3539284360 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3674017020 con 0x7f36800d2040 2023-11-18T20:43:11.410 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:11.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 -- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f366c06ae40 msgr2=0x7f366c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f366c06ae40 0x7f366c06d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3670036000 tx=0x7f367003c040 comp rx=0 tx=0).stop 2023-11-18T20:43:11.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 -- 172.21.15.60:0/3539284360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36800d2040 msgr2=0x7f3680159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36800d2040 0x7f3680159d00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f367400c410 tx=0x7f367400c8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 -- 172.21.15.60:0/3539284360 shutdown_connections 2023-11-18T20:43:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f366c06ae40 0x7f366c06d300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f368015d2d0 0x7f36800b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36800d2040 0x7f3680159d00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 --2- 172.21.15.60:0/3539284360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36800ce150 0x7f36800c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 -- 172.21.15.60:0/3539284360 >> 172.21.15.60:0/3539284360 conn(0x7f3680072050 msgr2=0x7f368015c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 -- 172.21.15.60:0/3539284360 shutdown_connections 2023-11-18T20:43:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.410+0000 7f368610c640 1 -- 172.21.15.60:0/3539284360 wait complete. 2023-11-18T20:43:11.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:11.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.528+0000 7f1387290640 1 Processor -- start 2023-11-18T20:43:11.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.528+0000 7f1387290640 1 -- start start 2023-11-18T20:43:11.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.529+0000 7f1387290640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f13800d2910 0x7f13800d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.529+0000 7f1387290640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d3730 0x7f13800b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.529+0000 7f1387290640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13800b2b10 con 0x7f138015d330 2023-11-18T20:43:11.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.529+0000 7f1387290640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13800d1590 con 0x7f13800d2910 2023-11-18T20:43:11.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.529+0000 7f1387290640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13800d1870 con 0x7f13800d3730 2023-11-18T20:43:11.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.529+0000 7f138482b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f13800d2910 0x7f13800d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.529+0000 7f138482b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f13800d2910 0x7f13800d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48934/0 (socket says 172.21.15.60:48934) 2023-11-18T20:43:11.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.529+0000 7f138482b640 1 -- 172.21.15.60:0/4074718870 learned_addr learned my addr 172.21.15.60:0/4074718870 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:11.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.530+0000 7f138582d640 1 --2- 172.21.15.60:0/4074718870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d3730 0x7f13800b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.530+0000 7f138482b640 1 -- 172.21.15.60:0/4074718870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d3730 msgr2=0x7f13800b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:11.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.530+0000 7f138482b640 1 --2- 172.21.15.60:0/4074718870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d3730 0x7f13800b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.530+0000 7f138482b640 1 -- 172.21.15.60:0/4074718870 >> v1:172.21.15.100:6789/0 conn(0x7f138015d330 legacy=0x7f13800b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.530+0000 7f138482b640 1 -- 172.21.15.60:0/4074718870 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13800d3b90 con 0x7f13800d2910 2023-11-18T20:43:11.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.530+0000 7f138582d640 1 --2- 172.21.15.60:0/4074718870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d3730 0x7f13800b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:11.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.530+0000 7f138482b640 1 --2- 172.21.15.60:0/4074718870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13800d2910 0x7f13800d31f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1374009870 tx=0x7f137402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7a248690016e01 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f137ffff640 1 -- 172.21.15.60:0/4074718870 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374031640 con 0x7f13800d2910 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f137ffff640 1 -- 172.21.15.60:0/4074718870 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13740317a0 con 0x7f13800d2910 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f137ffff640 1 -- 172.21.15.60:0/4074718870 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374031b30 con 0x7f13800d2910 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f1387290640 1 -- 172.21.15.60:0/4074718870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13800d2910 msgr2=0x7f13800d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f1387290640 1 --2- 172.21.15.60:0/4074718870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13800d2910 0x7f13800d31f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1374009870 tx=0x7f137402b8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f1387290640 1 -- 172.21.15.60:0/4074718870 shutdown_connections 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f1387290640 1 --2- 172.21.15.60:0/4074718870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13800d2910 0x7f13800d31f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f1387290640 1 --2- 172.21.15.60:0/4074718870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d3730 0x7f13800b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f1387290640 1 -- 172.21.15.60:0/4074718870 >> 172.21.15.60:0/4074718870 conn(0x7f1380072020 msgr2=0x7f1380072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f1387290640 1 -- 172.21.15.60:0/4074718870 shutdown_connections 2023-11-18T20:43:11.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.531+0000 7f1387290640 1 -- 172.21.15.60:0/4074718870 wait complete. 2023-11-18T20:43:11.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.532+0000 7f1387290640 1 Processor -- start 2023-11-18T20:43:11.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f1387290640 1 -- start start 2023-11-18T20:43:11.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f1387290640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d2910 0x7f13800b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f1387290640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13800d3730 0x7f13800cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f1387290640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138015d330 0x7f13800cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f1387290640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800ccee0 con 0x7f138015d330 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f1387290640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800cc2d0 con 0x7f13800d3730 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f1387290640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13800cc5b0 con 0x7f13800d2910 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f138582d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138015d330 0x7f13800cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f138582d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138015d330 0x7f13800cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48936/0 (socket says 172.21.15.60:48936) 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f138582d640 1 -- 172.21.15.60:0/663300089 learned_addr learned my addr 172.21.15.60:0/663300089 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f138502c640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d2910 0x7f13800b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.533+0000 7f138482b640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13800d3730 0x7f13800cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f138582d640 1 -- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d2910 msgr2=0x7f13800b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f138582d640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d2910 0x7f13800b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f138582d640 1 -- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13800d3730 msgr2=0x7f13800cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f138582d640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13800d3730 0x7f13800cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f138582d640 1 -- 172.21.15.60:0/663300089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13740094d0 con 0x7f138015d330 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f138582d640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138015d330 0x7f13800cbca0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f137800e7e0 tx=0x7f137800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f137dffb640 1 -- 172.21.15.60:0/663300089 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378004030 con 0x7f138015d330 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f137dffb640 1 -- 172.21.15.60:0/663300089 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378004190 con 0x7f138015d330 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f1387290640 1 -- 172.21.15.60:0/663300089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13800cc890 con 0x7f138015d330 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f137dffb640 1 -- 172.21.15.60:0/663300089 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378004500 con 0x7f138015d330 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.534+0000 7f1387290640 1 -- 172.21.15.60:0/663300089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13800c78e0 con 0x7f138015d330 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.535+0000 7f1387290640 1 -- 172.21.15.60:0/663300089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13800cd050 con 0x7f138015d330 2023-11-18T20:43:11.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.536+0000 7f137dffb640 1 -- 172.21.15.60:0/663300089 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f137800c6d0 con 0x7f138015d330 2023-11-18T20:43:11.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.536+0000 7f137dffb640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f136c06ae40 0x7f136c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.536+0000 7f137dffb640 1 -- 172.21.15.60:0/663300089 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1378089230 con 0x7f138015d330 2023-11-18T20:43:11.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.536+0000 7f138502c640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f136c06ae40 0x7f136c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.537+0000 7f138502c640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f136c06ae40 0x7f136c06d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f13700012e0 tx=0x7f1370009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:11.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.540+0000 7f137dffb640 1 -- 172.21.15.60:0/663300089 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1378054300 con 0x7f138015d330 2023-11-18T20:43:11.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.731+0000 7f1387290640 1 -- 172.21.15.60:0/663300089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1380069400 con 0x7f138015d330 2023-11-18T20:43:11.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.731+0000 7f137dffb640 1 -- 172.21.15.60:0/663300089 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1378057920 con 0x7f138015d330 2023-11-18T20:43:11.734 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 -- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f136c06ae40 msgr2=0x7f136c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f136c06ae40 0x7f136c06d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f13700012e0 tx=0x7f1370009040 comp rx=0 tx=0).stop 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 -- 172.21.15.60:0/663300089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138015d330 msgr2=0x7f13800cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138015d330 0x7f13800cbca0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f137800e7e0 tx=0x7f137800eca0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 -- 172.21.15.60:0/663300089 shutdown_connections 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f136c06ae40 0x7f136c06d300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138015d330 0x7f13800cbca0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13800d3730 0x7f13800cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 --2- 172.21.15.60:0/663300089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13800d2910 0x7f13800b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 -- 172.21.15.60:0/663300089 >> 172.21.15.60:0/663300089 conn(0x7f1380072020 msgr2=0x7f1380072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:11.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 -- 172.21.15.60:0/663300089 shutdown_connections 2023-11-18T20:43:11.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.733+0000 7f1387290640 1 -- 172.21.15.60:0/663300089 wait complete. 2023-11-18T20:43:11.746 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:43:11.746 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:11.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:11.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.869+0000 7f2516d2c640 1 Processor -- start 2023-11-18T20:43:11.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.869+0000 7f2516d2c640 1 -- start start 2023-11-18T20:43:11.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.869+0000 7f2516d2c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25100b1400 0x7f251015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.869+0000 7f2516d2c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f251015e730 0x7f251015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.869+0000 7f2516d2c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251015e2a0 con 0x7f2510143490 2023-11-18T20:43:11.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.869+0000 7f2516d2c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251015b520 con 0x7f251015e730 2023-11-18T20:43:11.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.869+0000 7f2516d2c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2510143870 con 0x7f25100b1400 2023-11-18T20:43:11.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.870+0000 7f25152c9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f251015e730 0x7f251015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.870+0000 7f2514ac8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2510143490 0x7f25100b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41190/0 (socket says 172.21.15.60:41190) 2023-11-18T20:43:11.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.870+0000 7f2514ac8640 1 -- 172.21.15.60:0/4289635503 learned_addr learned my addr 172.21.15.60:0/4289635503 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:11.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.870+0000 7f250ffff640 1 --2- 172.21.15.60:0/4289635503 >> v2:172.21.15.100:3300/0 conn(0x7f25100b1400 0x7f251015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.870+0000 7f25152c9640 1 -- 172.21.15.60:0/4289635503 >> v2:172.21.15.100:3300/0 conn(0x7f25100b1400 msgr2=0x7f251015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:11.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.870+0000 7f25152c9640 1 --2- 172.21.15.60:0/4289635503 >> v2:172.21.15.100:3300/0 conn(0x7f25100b1400 0x7f251015a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.870+0000 7f25152c9640 1 -- 172.21.15.60:0/4289635503 >> v1:172.21.15.60:6789/0 conn(0x7f2510143490 legacy=0x7f25100b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.870+0000 7f25152c9640 1 -- 172.21.15.60:0/4289635503 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251015b240 con 0x7f251015e730 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f25152c9640 1 --2- 172.21.15.60:0/4289635503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 0x7f251015aa20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25100aeca0 tx=0x7f250802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a18b86371b3fe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f250f7fe640 1 -- 172.21.15.60:0/4289635503 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250802cc80 con 0x7f251015e730 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f250f7fe640 1 -- 172.21.15.60:0/4289635503 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250802cde0 con 0x7f251015e730 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f250f7fe640 1 -- 172.21.15.60:0/4289635503 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508034660 con 0x7f251015e730 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f2516d2c640 1 -- 172.21.15.60:0/4289635503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 msgr2=0x7f251015aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f2516d2c640 1 --2- 172.21.15.60:0/4289635503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 0x7f251015aa20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25100aeca0 tx=0x7f250802bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f2516d2c640 1 -- 172.21.15.60:0/4289635503 shutdown_connections 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f2516d2c640 1 --2- 172.21.15.60:0/4289635503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 0x7f251015aa20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f2516d2c640 1 --2- 172.21.15.60:0/4289635503 >> v2:172.21.15.100:3300/0 conn(0x7f25100b1400 0x7f251015a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f2516d2c640 1 -- 172.21.15.60:0/4289635503 >> 172.21.15.60:0/4289635503 conn(0x7f251006a4d0 msgr2=0x7f251006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:11.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f2516d2c640 1 -- 172.21.15.60:0/4289635503 shutdown_connections 2023-11-18T20:43:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.871+0000 7f2516d2c640 1 -- 172.21.15.60:0/4289635503 wait complete. 2023-11-18T20:43:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.873+0000 7f2516d2c640 1 Processor -- start 2023-11-18T20:43:11.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.873+0000 7f2516d2c640 1 -- start start 2023-11-18T20:43:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.873+0000 7f2516d2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25100b1400 0x7f25100ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.873+0000 7f2516d2c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2510143490 0x7f25100ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.873+0000 7f2516d2c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 0x7f251006c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.873+0000 7f2516d2c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100a93c0 con 0x7f251015e730 2023-11-18T20:43:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.873+0000 7f2516d2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25100a84d0 con 0x7f25100b1400 2023-11-18T20:43:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.873+0000 7f2516d2c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25100a87b0 con 0x7f2510143490 2023-11-18T20:43:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f25152c9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 0x7f251006c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f25152c9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 0x7f251006c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48952/0 (socket says 172.21.15.60:48952) 2023-11-18T20:43:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f25152c9640 1 -- 172.21.15.60:0/2788992514 learned_addr learned my addr 172.21.15.60:0/2788992514 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:11.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f2514ac8640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25100b1400 0x7f25100ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f250ffff640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2510143490 0x7f25100ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f25152c9640 1 -- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2510143490 msgr2=0x7f25100ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f25152c9640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2510143490 0x7f25100ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f25152c9640 1 -- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25100b1400 msgr2=0x7f25100ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f25152c9640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25100b1400 0x7f25100ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f25152c9640 1 -- 172.21.15.60:0/2788992514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2508009a70 con 0x7f251015e730 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f25152c9640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 0x7f251006c690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2508009e80 tx=0x7f2508004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f250d7fa640 1 -- 172.21.15.60:0/2788992514 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508040650 con 0x7f251015e730 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f250d7fa640 1 -- 172.21.15.60:0/2788992514 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25080407b0 con 0x7f251015e730 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f2516d2c640 1 -- 172.21.15.60:0/2788992514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25100a8a90 con 0x7f251015e730 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f250d7fa640 1 -- 172.21.15.60:0/2788992514 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508040b60 con 0x7f251015e730 2023-11-18T20:43:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.874+0000 7f2516d2c640 1 -- 172.21.15.60:0/2788992514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25100a9050 con 0x7f251015e730 2023-11-18T20:43:11.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.875+0000 7f2516d2c640 1 -- 172.21.15.60:0/2788992514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25100a9530 con 0x7f251015e730 2023-11-18T20:43:11.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.876+0000 7f250d7fa640 1 -- 172.21.15.60:0/2788992514 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2508040cc0 con 0x7f251015e730 2023-11-18T20:43:11.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.876+0000 7f250d7fa640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f24f406ae40 0x7f24f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:11.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.876+0000 7f250d7fa640 1 -- 172.21.15.60:0/2788992514 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25080ac3c0 con 0x7f251015e730 2023-11-18T20:43:11.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.876+0000 7f2514ac8640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f24f406ae40 0x7f24f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:11.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.877+0000 7f2514ac8640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f24f406ae40 0x7f24f406d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2504005880 tx=0x7f2504009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:11.880+0000 7f250d7fa640 1 -- 172.21.15.60:0/2788992514 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2508077470 con 0x7f251015e730 2023-11-18T20:43:12.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.057+0000 7f2516d2c640 1 -- 172.21.15.60:0/2788992514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2510069400 con 0x7f251015e730 2023-11-18T20:43:12.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.058+0000 7f250d7fa640 1 -- 172.21.15.60:0/2788992514 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f2508009450 con 0x7f251015e730 2023-11-18T20:43:12.060 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:12.060 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:12.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 -- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f24f406ae40 msgr2=0x7f24f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f24f406ae40 0x7f24f406d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2504005880 tx=0x7f2504009040 comp rx=0 tx=0).stop 2023-11-18T20:43:12.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 -- 172.21.15.60:0/2788992514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 msgr2=0x7f251006c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 0x7f251006c690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2508009e80 tx=0x7f2508004a40 comp rx=0 tx=0).stop 2023-11-18T20:43:12.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 -- 172.21.15.60:0/2788992514 shutdown_connections 2023-11-18T20:43:12.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f24f406ae40 0x7f24f406d300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f251015e730 0x7f251006c690 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2510143490 0x7f25100ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 --2- 172.21.15.60:0/2788992514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25100b1400 0x7f25100ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 -- 172.21.15.60:0/2788992514 >> 172.21.15.60:0/2788992514 conn(0x7f251006a4d0 msgr2=0x7f251006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:12.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 -- 172.21.15.60:0/2788992514 shutdown_connections 2023-11-18T20:43:12.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.060+0000 7f2516d2c640 1 -- 172.21.15.60:0/2788992514 wait complete. 2023-11-18T20:43:12.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:12.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:12.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.181+0000 7f27bda26640 1 Processor -- start 2023-11-18T20:43:12.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.181+0000 7f27bda26640 1 -- start start 2023-11-18T20:43:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.181+0000 7f27bda26640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27b8140310 0x7f27b81618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27bda26640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f27b81409d0 0x7f27b806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27bda26640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b81406f0 con 0x7f27b81428d0 2023-11-18T20:43:12.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27bda26640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b8161e20 con 0x7f27b81409d0 2023-11-18T20:43:12.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27bda26640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8143960 con 0x7f27b8140310 2023-11-18T20:43:12.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27ae7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f27b81409d0 0x7f27b806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27b77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f27b81428d0 0x7f27b80714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41194/0 (socket says 172.21.15.60:41194) 2023-11-18T20:43:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27b77fe640 1 -- 172.21.15.60:0/2996280466 learned_addr learned my addr 172.21.15.60:0/2996280466 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27b6ffd640 1 --2- 172.21.15.60:0/2996280466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27b8140310 0x7f27b81618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27ae7fc640 1 -- 172.21.15.60:0/2996280466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27b8140310 msgr2=0x7f27b81618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27ae7fc640 1 --2- 172.21.15.60:0/2996280466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27b8140310 0x7f27b81618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27ae7fc640 1 -- 172.21.15.60:0/2996280466 >> v1:172.21.15.60:6789/0 conn(0x7f27b81428d0 legacy=0x7f27b80714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27ae7fc640 1 -- 172.21.15.60:0/2996280466 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8142de0 con 0x7f27b81409d0 2023-11-18T20:43:12.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.182+0000 7f27b67fc640 1 -- 172.21.15.60:0/2996280466 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248101598 0 0) 0x7f27b81406f0 con 0x7f27b81428d0 2023-11-18T20:43:12.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.183+0000 7f27ae7fc640 1 --2- 172.21.15.60:0/2996280466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b81409d0 0x7f27b806a4d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f27a00097a0 tx=0x7f27a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01ea40f56d2c0f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:12.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.183+0000 7f27b67fc640 1 -- 172.21.15.60:0/2996280466 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a002e070 con 0x7f27b81409d0 2023-11-18T20:43:12.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.183+0000 7f27b67fc640 1 -- 172.21.15.60:0/2996280466 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a00315a0 con 0x7f27b81409d0 2023-11-18T20:43:12.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.183+0000 7f27b67fc640 1 -- 172.21.15.60:0/2996280466 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0031930 con 0x7f27b81409d0 2023-11-18T20:43:12.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.183+0000 7f27bda26640 1 -- 172.21.15.60:0/2996280466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b81409d0 msgr2=0x7f27b806a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.183+0000 7f27bda26640 1 --2- 172.21.15.60:0/2996280466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b81409d0 0x7f27b806a4d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f27a00097a0 tx=0x7f27a002b840 comp rx=0 tx=0).stop 2023-11-18T20:43:12.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.184+0000 7f27bda26640 1 -- 172.21.15.60:0/2996280466 shutdown_connections 2023-11-18T20:43:12.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.184+0000 7f27bda26640 1 --2- 172.21.15.60:0/2996280466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b81409d0 0x7f27b806a4d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.184+0000 7f27bda26640 1 --2- 172.21.15.60:0/2996280466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27b8140310 0x7f27b81618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.184+0000 7f27bda26640 1 -- 172.21.15.60:0/2996280466 >> 172.21.15.60:0/2996280466 conn(0x7f27b80b1e50 msgr2=0x7f27b80b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.184+0000 7f27bda26640 1 -- 172.21.15.60:0/2996280466 shutdown_connections 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.184+0000 7f27bda26640 1 -- 172.21.15.60:0/2996280466 wait complete. 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.185+0000 7f27bda26640 1 Processor -- start 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.185+0000 7f27bda26640 1 -- start start 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27bda26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b8140310 0x7f27b812ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27bda26640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27b81409d0 0x7f27b81f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27bda26640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27b81428d0 0x7f27b812d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27bda26640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b81f2a30 con 0x7f27b8140310 2023-11-18T20:43:12.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27bda26640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b812e8c0 con 0x7f27b81409d0 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27bda26640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8154740 con 0x7f27b81428d0 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b8140310 0x7f27b812ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b8140310 0x7f27b812ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48970/0 (socket says 172.21.15.60:48970) 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b6ffd640 1 -- 172.21.15.60:0/735644325 learned_addr learned my addr 172.21.15.60:0/735644325 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27ae7fc640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27b81409d0 0x7f27b81f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b77fe640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27b81428d0 0x7f27b812d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b6ffd640 1 -- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27b81428d0 msgr2=0x7f27b812d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b6ffd640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27b81428d0 0x7f27b812d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b6ffd640 1 -- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27b81409d0 msgr2=0x7f27b81f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b6ffd640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27b81409d0 0x7f27b81f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b6ffd640 1 -- 172.21.15.60:0/735644325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a0009400 con 0x7f27b8140310 2023-11-18T20:43:12.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b6ffd640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b8140310 0x7f27b812ed50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f27a800d620 tx=0x7f27a800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b4ff9640 1 -- 172.21.15.60:0/735644325 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a800b620 con 0x7f27b8140310 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b4ff9640 1 -- 172.21.15.60:0/735644325 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a800b780 con 0x7f27b8140310 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27bda26640 1 -- 172.21.15.60:0/735644325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b812dcb0 con 0x7f27b8140310 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27b4ff9640 1 -- 172.21.15.60:0/735644325 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a800bb30 con 0x7f27b8140310 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.186+0000 7f27bda26640 1 -- 172.21.15.60:0/735644325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b812e270 con 0x7f27b8140310 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.187+0000 7f27bda26640 1 -- 172.21.15.60:0/735644325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b8069400 con 0x7f27b8140310 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.188+0000 7f27b4ff9640 1 -- 172.21.15.60:0/735644325 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27a800bc90 con 0x7f27b8140310 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.188+0000 7f27b4ff9640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f279806ae40 0x7f279806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.188+0000 7f27b4ff9640 1 -- 172.21.15.60:0/735644325 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27a8089990 con 0x7f27b8140310 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.189+0000 7f27ae7fc640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f279806ae40 0x7f279806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.189+0000 7f27ae7fc640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f279806ae40 0x7f279806d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f27a0036000 tx=0x7f27a003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:12.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.193+0000 7f27b4ff9640 1 -- 172.21.15.60:0/735644325 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27a8054af0 con 0x7f27b8140310 2023-11-18T20:43:12.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.368+0000 7f27bda26640 1 -- 172.21.15.60:0/735644325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27b806ea60 con 0x7f27b8140310 2023-11-18T20:43:12.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.369+0000 7f27b4ff9640 1 -- 172.21.15.60:0/735644325 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27a8058110 con 0x7f27b8140310 2023-11-18T20:43:12.371 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 -- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f279806ae40 msgr2=0x7f279806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f279806ae40 0x7f279806d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f27a0036000 tx=0x7f27a003d040 comp rx=0 tx=0).stop 2023-11-18T20:43:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 -- 172.21.15.60:0/735644325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b8140310 msgr2=0x7f27b812ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b8140310 0x7f27b812ed50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f27a800d620 tx=0x7f27a800dae0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 -- 172.21.15.60:0/735644325 shutdown_connections 2023-11-18T20:43:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f279806ae40 0x7f279806d300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27b81428d0 0x7f27b812d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27b81409d0 0x7f27b81f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 --2- 172.21.15.60:0/735644325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27b8140310 0x7f27b812ed50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 -- 172.21.15.60:0/735644325 >> 172.21.15.60:0/735644325 conn(0x7f27b80b1e50 msgr2=0x7f27b815fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 -- 172.21.15.60:0/735644325 shutdown_connections 2023-11-18T20:43:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.371+0000 7f27bda26640 1 -- 172.21.15.60:0/735644325 wait complete. 2023-11-18T20:43:12.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.491+0000 7f1dd4f1d640 1 Processor -- start 2023-11-18T20:43:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.491+0000 7f1dd4f1d640 1 -- start start 2023-11-18T20:43:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.491+0000 7f1dd4f1d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1dd0140310 0x7f1dd01618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.491+0000 7f1dd4f1d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1dd01409d0 0x7f1dd006a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dd4f1d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd01406f0 con 0x7f1dd01428d0 2023-11-18T20:43:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dd4f1d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd0161e20 con 0x7f1dd0140310 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dd4f1d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd0143960 con 0x7f1dd01409d0 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dce59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1dd0140310 0x7f1dd01618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dced9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1dd01428d0 0x7f1dd00714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41198/0 (socket says 172.21.15.60:41198) 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dced9d640 1 -- 172.21.15.60:0/1372325713 learned_addr learned my addr 172.21.15.60:0/1372325713 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dc7fff640 1 --2- 172.21.15.60:0/1372325713 >> v2:172.21.15.100:3300/0 conn(0x7f1dd01409d0 0x7f1dd006a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dce59c640 1 -- 172.21.15.60:0/1372325713 >> v2:172.21.15.100:3300/0 conn(0x7f1dd01409d0 msgr2=0x7f1dd006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dce59c640 1 --2- 172.21.15.60:0/1372325713 >> v2:172.21.15.100:3300/0 conn(0x7f1dd01409d0 0x7f1dd006a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dce59c640 1 -- 172.21.15.60:0/1372325713 >> v1:172.21.15.60:6789/0 conn(0x7f1dd01428d0 legacy=0x7f1dd00714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dce59c640 1 -- 172.21.15.60:0/1372325713 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd0142de0 con 0x7f1dd0140310 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dcdd9b640 1 -- 172.21.15.60:0/1372325713 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987134608 0 0) 0x7f1dd01406f0 con 0x7f1dd01428d0 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.492+0000 7f1dc7fff640 1 --2- 172.21.15.60:0/1372325713 >> v2:172.21.15.100:3300/0 conn(0x7f1dd01409d0 0x7f1dd006a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.493+0000 7f1dce59c640 1 --2- 172.21.15.60:0/1372325713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd0140310 0x7f1dd01618e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0161f60 tx=0x7f1dc002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d4187f79d9aeea server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.493+0000 7f1dcdd9b640 1 -- 172.21.15.60:0/1372325713 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc002e070 con 0x7f1dd0140310 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.493+0000 7f1dcdd9b640 1 -- 172.21.15.60:0/1372325713 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc00315a0 con 0x7f1dd0140310 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.493+0000 7f1dcdd9b640 1 -- 172.21.15.60:0/1372325713 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0031930 con 0x7f1dd0140310 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.493+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/1372325713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd0140310 msgr2=0x7f1dd01618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.493+0000 7f1dd4f1d640 1 --2- 172.21.15.60:0/1372325713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd0140310 0x7f1dd01618e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0161f60 tx=0x7f1dc002b840 comp rx=0 tx=0).stop 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.493+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/1372325713 shutdown_connections 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.493+0000 7f1dd4f1d640 1 --2- 172.21.15.60:0/1372325713 >> v2:172.21.15.100:3300/0 conn(0x7f1dd01409d0 0x7f1dd006a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.493+0000 7f1dd4f1d640 1 --2- 172.21.15.60:0/1372325713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd0140310 0x7f1dd01618e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.493+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/1372325713 >> 172.21.15.60:0/1372325713 conn(0x7f1dd00b1e50 msgr2=0x7f1dd00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.494+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/1372325713 shutdown_connections 2023-11-18T20:43:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.494+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/1372325713 wait complete. 2023-11-18T20:43:12.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.495+0000 7f1dd4f1d640 1 Processor -- start 2023-11-18T20:43:12.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.495+0000 7f1dd4f1d640 1 -- start start 2023-11-18T20:43:12.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dd4f1d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dd0140310 0x7f1dd00af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dd4f1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd01409d0 0x7f1dd0153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dd4f1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd01428d0 0x7f1dd015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dd4f1d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00adf70 con 0x7f1dd01428d0 2023-11-18T20:43:12.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dd4f1d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00af9e0 con 0x7f1dd01409d0 2023-11-18T20:43:12.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dd4f1d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd00ae150 con 0x7f1dd0140310 2023-11-18T20:43:12.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dced9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd01428d0 0x7f1dd015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dced9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd01428d0 0x7f1dd015d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49002/0 (socket says 172.21.15.60:49002) 2023-11-18T20:43:12.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dced9d640 1 -- 172.21.15.60:0/512510819 learned_addr learned my addr 172.21.15.60:0/512510819 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dce59c640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dd0140310 0x7f1dd00af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dc7fff640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd01409d0 0x7f1dd0153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dced9d640 1 -- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dd0140310 msgr2=0x7f1dd00af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dced9d640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dd0140310 0x7f1dd00af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dced9d640 1 -- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd01409d0 msgr2=0x7f1dd0153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dced9d640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd01409d0 0x7f1dd0153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dced9d640 1 -- 172.21.15.60:0/512510819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc0009400 con 0x7f1dd01428d0 2023-11-18T20:43:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dce59c640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dd0140310 0x7f1dd00af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dc7fff640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd01409d0 0x7f1dd0153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:12.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dced9d640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd01428d0 0x7f1dd015d880 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800c7e0 tx=0x7f1dc800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dc6ffd640 1 -- 172.21.15.60:0/512510819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800a640 con 0x7f1dd01428d0 2023-11-18T20:43:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dc6ffd640 1 -- 172.21.15.60:0/512510819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc800a7c0 con 0x7f1dd01428d0 2023-11-18T20:43:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/512510819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd00ae430 con 0x7f1dd01428d0 2023-11-18T20:43:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dc6ffd640 1 -- 172.21.15.60:0/512510819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800ab70 con 0x7f1dd01428d0 2023-11-18T20:43:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.496+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/512510819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd0153c70 con 0x7f1dd01428d0 2023-11-18T20:43:12.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.498+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/512510819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd0069400 con 0x7f1dd01428d0 2023-11-18T20:43:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.498+0000 7f1dc6ffd640 1 -- 172.21.15.60:0/512510819 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1dc800acd0 con 0x7f1dd01428d0 2023-11-18T20:43:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.498+0000 7f1dc6ffd640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1db006ae40 0x7f1db006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.498+0000 7f1dc6ffd640 1 -- 172.21.15.60:0/512510819 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1dc8088df0 con 0x7f1dd01428d0 2023-11-18T20:43:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.499+0000 7f1dce59c640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1db006ae40 0x7f1db006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.499+0000 7f1dce59c640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1db006ae40 0x7f1db006d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0036000 tx=0x7f1dc003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:12.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.503+0000 7f1dc6ffd640 1 -- 172.21.15.60:0/512510819 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1dc8053f30 con 0x7f1dd01428d0 2023-11-18T20:43:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.678+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/512510819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1dd006ea60 con 0x7f1dd01428d0 2023-11-18T20:43:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.679+0000 7f1dc6ffd640 1 -- 172.21.15.60:0/512510819 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1dc80180a0 con 0x7f1dd01428d0 2023-11-18T20:43:12.681 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:12.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.680+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1db006ae40 msgr2=0x7f1db006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1db006ae40 0x7f1db006d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0036000 tx=0x7f1dc003d040 comp rx=0 tx=0).stop 2023-11-18T20:43:12.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/512510819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd01428d0 msgr2=0x7f1dd015d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd01428d0 0x7f1dd015d880 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800c7e0 tx=0x7f1dc800cca0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/512510819 shutdown_connections 2023-11-18T20:43:12.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1db006ae40 0x7f1db006d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd01428d0 0x7f1dd015d880 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dd01409d0 0x7f1dd0153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 --2- 172.21.15.60:0/512510819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dd0140310 0x7f1dd00af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/512510819 >> 172.21.15.60:0/512510819 conn(0x7f1dd00b1e50 msgr2=0x7f1dd0161670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:12.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/512510819 shutdown_connections 2023-11-18T20:43:12.684 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.681+0000 7f1dd4f1d640 1 -- 172.21.15.60:0/512510819 wait complete. 2023-11-18T20:43:12.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:12.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.801+0000 7facf0409640 1 Processor -- start 2023-11-18T20:43:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.801+0000 7facf0409640 1 -- start start 2023-11-18T20:43:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.802+0000 7facf0409640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7face80d2910 0x7face80d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.802+0000 7facf0409640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face80d3730 0x7face80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.802+0000 7facf0409640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face80b2b10 con 0x7face815d330 2023-11-18T20:43:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.802+0000 7facf0409640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face80d1590 con 0x7face80d2910 2023-11-18T20:43:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.802+0000 7facf0409640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face80d1870 con 0x7face80d3730 2023-11-18T20:43:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.802+0000 7faced9a4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7face80d2910 0x7face80d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.802+0000 7facee1a5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7face815d330 0x7face80b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41214/0 (socket says 172.21.15.60:41214) 2023-11-18T20:43:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.802+0000 7facee1a5640 1 -- 172.21.15.60:0/2661734869 learned_addr learned my addr 172.21.15.60:0/2661734869 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.803+0000 7facee9a6640 1 --2- 172.21.15.60:0/2661734869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face80d3730 0x7face80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.803+0000 7faced1a3640 1 -- 172.21.15.60:0/2661734869 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754975976 0 0) 0x7face80b2b10 con 0x7face815d330 2023-11-18T20:43:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.803+0000 7faced9a4640 1 -- 172.21.15.60:0/2661734869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face80d3730 msgr2=0x7face80b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.803+0000 7faced9a4640 1 --2- 172.21.15.60:0/2661734869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face80d3730 0x7face80b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.803+0000 7faced9a4640 1 -- 172.21.15.60:0/2661734869 >> v1:172.21.15.60:6789/0 conn(0x7face815d330 legacy=0x7face80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.803+0000 7faced9a4640 1 -- 172.21.15.60:0/2661734869 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face80d3b90 con 0x7face80d2910 2023-11-18T20:43:12.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.803+0000 7faced9a4640 1 --2- 172.21.15.60:0/2661734869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d2910 0x7face80d31f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097a0 tx=0x7facdc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af55b9090c79d8a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7faced1a3640 1 -- 172.21.15.60:0/2661734869 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc02e070 con 0x7face80d2910 2023-11-18T20:43:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7faced1a3640 1 -- 172.21.15.60:0/2661734869 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc0315a0 con 0x7face80d2910 2023-11-18T20:43:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7faced1a3640 1 -- 172.21.15.60:0/2661734869 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc031930 con 0x7face80d2910 2023-11-18T20:43:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7facf0409640 1 -- 172.21.15.60:0/2661734869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d2910 msgr2=0x7face80d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7facf0409640 1 --2- 172.21.15.60:0/2661734869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d2910 0x7face80d31f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097a0 tx=0x7facdc02b840 comp rx=0 tx=0).stop 2023-11-18T20:43:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7facf0409640 1 -- 172.21.15.60:0/2661734869 shutdown_connections 2023-11-18T20:43:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7facf0409640 1 --2- 172.21.15.60:0/2661734869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face80d3730 0x7face80b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7facf0409640 1 --2- 172.21.15.60:0/2661734869 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d2910 0x7face80d31f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7facf0409640 1 -- 172.21.15.60:0/2661734869 >> 172.21.15.60:0/2661734869 conn(0x7face8072020 msgr2=0x7face8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7facf0409640 1 -- 172.21.15.60:0/2661734869 shutdown_connections 2023-11-18T20:43:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.804+0000 7facf0409640 1 -- 172.21.15.60:0/2661734869 wait complete. 2023-11-18T20:43:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.805+0000 7facf0409640 1 Processor -- start 2023-11-18T20:43:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7facf0409640 1 -- start start 2023-11-18T20:43:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7facf0409640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face80d2910 0x7face8157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7facf0409640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d3730 0x7face815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7facf0409640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face815d330 0x7face80b8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7facf0409640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80c9c60 con 0x7face80d3730 2023-11-18T20:43:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7facf0409640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face81581d0 con 0x7face80d2910 2023-11-18T20:43:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7facf0409640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face815d710 con 0x7face815d330 2023-11-18T20:43:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7faced9a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d3730 0x7face815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7faced9a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d3730 0x7face815ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49032/0 (socket says 172.21.15.60:49032) 2023-11-18T20:43:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7faced9a4640 1 -- 172.21.15.60:0/1524767186 learned_addr learned my addr 172.21.15.60:0/1524767186 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7facee1a5640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face80d2910 0x7face8157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.806+0000 7facee9a6640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face815d330 0x7face80b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7faced9a4640 1 -- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face815d330 msgr2=0x7face80b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7faced9a4640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face815d330 0x7face80b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7faced9a4640 1 -- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face80d2910 msgr2=0x7face8157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7faced9a4640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face80d2910 0x7face8157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7faced9a4640 1 -- 172.21.15.60:0/1524767186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc009400 con 0x7face80d3730 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7faced9a4640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d3730 0x7face815ef60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7facdc034040 tx=0x7facdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7face6ffd640 1 -- 172.21.15.60:0/1524767186 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc004230 con 0x7face80d3730 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7face6ffd640 1 -- 172.21.15.60:0/1524767186 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc031f00 con 0x7face80d3730 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7facf0409640 1 -- 172.21.15.60:0/1524767186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face80c8870 con 0x7face80d3730 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7face6ffd640 1 -- 172.21.15.60:0/1524767186 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc032640 con 0x7face80d3730 2023-11-18T20:43:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.807+0000 7facf0409640 1 -- 172.21.15.60:0/1524767186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face815d960 con 0x7face80d3730 2023-11-18T20:43:12.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.808+0000 7facf0409640 1 -- 172.21.15.60:0/1524767186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face8069400 con 0x7face80d3730 2023-11-18T20:43:12.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.808+0000 7face6ffd640 1 -- 172.21.15.60:0/1524767186 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7facdc0327a0 con 0x7face80d3730 2023-11-18T20:43:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.809+0000 7face6ffd640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7facd406ae40 0x7facd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.809+0000 7face6ffd640 1 -- 172.21.15.60:0/1524767186 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facdc0ab420 con 0x7face80d3730 2023-11-18T20:43:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.809+0000 7facee1a5640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7facd406ae40 0x7facd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.809+0000 7facee1a5640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7facd406ae40 0x7facd406d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7face0000f60 tx=0x7face0007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.813+0000 7face6ffd640 1 -- 172.21.15.60:0/1524767186 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7facdc076610 con 0x7face80d3730 2023-11-18T20:43:12.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.987+0000 7facf0409640 1 -- 172.21.15.60:0/1524767186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7face80d0870 con 0x7face80d3730 2023-11-18T20:43:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.987+0000 7face6ffd640 1 -- 172.21.15.60:0/1524767186 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7facdc079c30 con 0x7face80d3730 2023-11-18T20:43:12.990 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.989+0000 7facf0409640 1 -- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7facd406ae40 msgr2=0x7facd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.989+0000 7facf0409640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7facd406ae40 0x7facd406d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7face0000f60 tx=0x7face0007d50 comp rx=0 tx=0).stop 2023-11-18T20:43:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.989+0000 7facf0409640 1 -- 172.21.15.60:0/1524767186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d3730 msgr2=0x7face815ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.989+0000 7facf0409640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d3730 0x7face815ef60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7facdc034040 tx=0x7facdc0046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.989+0000 7facf0409640 1 -- 172.21.15.60:0/1524767186 shutdown_connections 2023-11-18T20:43:12.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.989+0000 7facf0409640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7facd406ae40 0x7facd406d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.989+0000 7facf0409640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face815d330 0x7face80b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.989+0000 7facf0409640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face80d3730 0x7face815ef60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.989+0000 7facf0409640 1 --2- 172.21.15.60:0/1524767186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face80d2910 0x7face8157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:12.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.989+0000 7facf0409640 1 -- 172.21.15.60:0/1524767186 >> 172.21.15.60:0/1524767186 conn(0x7face8072020 msgr2=0x7face8072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:12.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.990+0000 7facf0409640 1 -- 172.21.15.60:0/1524767186 shutdown_connections 2023-11-18T20:43:12.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:12.990+0000 7facf0409640 1 -- 172.21.15.60:0/1524767186 wait complete. 2023-11-18T20:43:13.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:13.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.109+0000 7fa8c9b9e640 1 Processor -- start 2023-11-18T20:43:13.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.109+0000 7fa8c9b9e640 1 -- start start 2023-11-18T20:43:13.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c9b9e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa8c41443a0 0x7fa8c4144900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c9b9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c40cc560 0x7fa8c40cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c9b9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c40ccee0 con 0x7fa8c415bec0 2023-11-18T20:43:13.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c9b9e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c415b4d0 con 0x7fa8c41443a0 2023-11-18T20:43:13.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c9b9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4144780 con 0x7fa8c40cc560 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c3fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa8c415bec0 0x7fa8c40ce6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41230/0 (socket says 172.21.15.60:41230) 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c3fff640 1 -- 172.21.15.60:0/3424147742 learned_addr learned my addr 172.21.15.60:0/3424147742 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c8b9c640 1 --2- 172.21.15.60:0/3424147742 >> v2:172.21.15.100:3300/0 conn(0x7fa8c41443a0 0x7fa8c4144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c939d640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c40cc560 0x7fa8c40cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994401018 0 0) 0x7fa8c40ccee0 con 0x7fa8c415bec0 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b4003520 con 0x7fa8c415bec0 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1695027345 0 0) 0x7fa8b4003520 con 0x7fa8c415bec0 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c40cc560 msgr2=0x7fa8c40cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.110+0000 7fa8c37fe640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c40cc560 0x7fa8c40cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 >> v2:172.21.15.100:3300/0 conn(0x7fa8c41443a0 msgr2=0x7fa8c4144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 --2- 172.21.15.60:0/3424147742 >> v2:172.21.15.100:3300/0 conn(0x7fa8c41443a0 0x7fa8c4144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c415f570 con 0x7fa8c415bec0 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fa8b0001670 con 0x7fa8c415bec0 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c8b9c640 1 --2- 172.21.15.60:0/3424147742 >> v2:172.21.15.100:3300/0 conn(0x7fa8c41443a0 0x7fa8c4144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 >> v1:172.21.15.60:6789/0 conn(0x7fa8c415bec0 legacy=0x7fa8c40ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40039a0 0x7fa8b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c40cc560 0x7fa8b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8b4004d00 0x7fa8b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4144780 con 0x7fa8b4004d00 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b400b980 con 0x7fa8b40039a0 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b400bac0 con 0x7fa8c40cc560 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c939d640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8b4004d00 0x7fa8b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c8b9c640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40039a0 0x7fa8b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c3fff640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c40cc560 0x7fa8b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c939d640 1 -- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c40cc560 msgr2=0x7fa8b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:13.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c939d640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c40cc560 0x7fa8b40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c939d640 1 -- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40039a0 msgr2=0x7fa8b4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:13.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c939d640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b40039a0 0x7fa8b4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.111+0000 7fa8c939d640 1 -- 172.21.15.60:0/3424147742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b4003520 con 0x7fa8b4004d00 2023-11-18T20:43:13.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.112+0000 7fa8c939d640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8b4004d00 0x7fa8b4005180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00e170 tx=0x7fa8b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a874f23a89d0bbe server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:13.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.112+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc02cbf0 con 0x7fa8b4004d00 2023-11-18T20:43:13.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.112+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc00f040 con 0x7fa8b4004d00 2023-11-18T20:43:13.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.112+0000 7fa8c37fe640 1 -- 172.21.15.60:0/3424147742 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc034440 con 0x7fa8b4004d00 2023-11-18T20:43:13.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.112+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/3424147742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8b4004d00 msgr2=0x7fa8b4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.112+0000 7fa8c9b9e640 1 --2- 172.21.15.60:0/3424147742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8b4004d00 0x7fa8b4005180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00e170 tx=0x7fa8b40057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.112+0000 7fa8c939d640 1 -- 172.21.15.60:0/3424147742 reap_dead start 2023-11-18T20:43:13.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.112+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/3424147742 shutdown_connections 2023-11-18T20:43:13.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.112+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/3424147742 >> 172.21.15.60:0/3424147742 conn(0x7fa8c406a4d0 msgr2=0x7fa8c406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:13.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.112+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/3424147742 shutdown_connections 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.113+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/3424147742 wait complete. 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.114+0000 7fa8c9b9e640 1 Processor -- start 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.114+0000 7fa8c9b9e640 1 -- start start 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c9b9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4004d00 0x7fa8c415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c9b9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c413a0e0 0x7fa8c413a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c9b9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c41f2ce0 0x7fa8c41f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c9b9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c413aaa0 con 0x7fa8c413a0e0 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c9b9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c415a8d0 con 0x7fa8b4004d00 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c9b9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40ad6d0 con 0x7fa8c41f2ce0 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c413a0e0 0x7fa8c413a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c413a0e0 0x7fa8c413a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49050/0 (socket says 172.21.15.60:49050) 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c3fff640 1 -- 172.21.15.60:0/1154735979 learned_addr learned my addr 172.21.15.60:0/1154735979 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:13.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c8b9c640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4004d00 0x7fa8c415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c939d640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c41f2ce0 0x7fa8c41f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c3fff640 1 -- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c41f2ce0 msgr2=0x7fa8c41f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c3fff640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c41f2ce0 0x7fa8c41f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c3fff640 1 -- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4004d00 msgr2=0x7fa8c415a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c3fff640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4004d00 0x7fa8c415a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c3fff640 1 -- 172.21.15.60:0/1154735979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40ad9b0 con 0x7fa8c413a0e0 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c939d640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c41f2ce0 0x7fa8c41f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.115+0000 7fa8c3fff640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c413a0e0 0x7fa8c413a560 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00072c0 tx=0x7fa8b0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.116+0000 7fa8c17fa640 1 -- 172.21.15.60:0/1154735979 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0001570 con 0x7fa8c413a0e0 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.116+0000 7fa8c17fa640 1 -- 172.21.15.60:0/1154735979 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b000fa20 con 0x7fa8c413a0e0 2023-11-18T20:43:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.116+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/1154735979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c40adc90 con 0x7fa8c413a0e0 2023-11-18T20:43:13.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.116+0000 7fa8c17fa640 1 -- 172.21.15.60:0/1154735979 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b000fdd0 con 0x7fa8c413a0e0 2023-11-18T20:43:13.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.116+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/1154735979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c40ae250 con 0x7fa8c413a0e0 2023-11-18T20:43:13.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.117+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/1154735979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c40afac0 con 0x7fa8c413a0e0 2023-11-18T20:43:13.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.117+0000 7fa8c17fa640 1 -- 172.21.15.60:0/1154735979 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa8b001d460 con 0x7fa8c413a0e0 2023-11-18T20:43:13.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.117+0000 7fa8c17fa640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8a806ae70 0x7fa8a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.118+0000 7fa8c17fa640 1 -- 172.21.15.60:0/1154735979 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8b00898a0 con 0x7fa8c413a0e0 2023-11-18T20:43:13.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.118+0000 7fa8c8b9c640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8a806ae70 0x7fa8a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.118+0000 7fa8c8b9c640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8a806ae70 0x7fa8a806d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80093d0 tx=0x7fa8b8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:13.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.122+0000 7fa8c17fa640 1 -- 172.21.15.60:0/1154735979 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8b00548c0 con 0x7fa8c413a0e0 2023-11-18T20:43:13.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.311+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/1154735979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa8c4069320 con 0x7fa8c413a0e0 2023-11-18T20:43:13.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.311+0000 7fa8c17fa640 1 -- 172.21.15.60:0/1154735979 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa8b0057ee0 con 0x7fa8c413a0e0 2023-11-18T20:43:13.314 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:13.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8a806ae70 msgr2=0x7fa8a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8a806ae70 0x7fa8a806d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80093d0 tx=0x7fa8b8009190 comp rx=0 tx=0).stop 2023-11-18T20:43:13.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/1154735979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c413a0e0 msgr2=0x7fa8c413a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c413a0e0 0x7fa8c413a560 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00072c0 tx=0x7fa8b0007780 comp rx=0 tx=0).stop 2023-11-18T20:43:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/1154735979 shutdown_connections 2023-11-18T20:43:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8a806ae70 0x7fa8a806d330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8c41f2ce0 0x7fa8c41f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8c413a0e0 0x7fa8c413a560 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 --2- 172.21.15.60:0/1154735979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8b4004d00 0x7fa8c415a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/1154735979 >> 172.21.15.60:0/1154735979 conn(0x7fa8c406a4d0 msgr2=0x7fa8c41428d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/1154735979 shutdown_connections 2023-11-18T20:43:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.313+0000 7fa8c9b9e640 1 -- 172.21.15.60:0/1154735979 wait complete. 2023-11-18T20:43:13.326 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2023-11-18T20:43:13.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:13.327 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-18T20:43:13.360 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:43:13.360 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:43:13.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:13.360 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-18T20:43:13.430 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:43:13.430 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:13.446 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:43:13.446 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:13.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:13.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.646+0000 7f5057675640 1 Processor -- start 2023-11-18T20:43:13.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.647+0000 7f5057675640 1 -- start start 2023-11-18T20:43:13.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.647+0000 7f5057675640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 0x7f50500d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.647+0000 7f5057675640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50500d3730 0x7f50500b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.647+0000 7f5057675640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50500b2b10 con 0x7f505015d330 2023-11-18T20:43:13.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.647+0000 7f5057675640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50500d3b50 con 0x7f50500d3730 2023-11-18T20:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.647+0000 7f5057675640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50500d1590 con 0x7f50500d2910 2023-11-18T20:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.647+0000 7f5055411640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f505015d330 0x7f50500b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41246/0 (socket says 172.21.15.60:41246) 2023-11-18T20:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.647+0000 7f5055411640 1 -- 172.21.15.60:0/2216716221 learned_addr learned my addr 172.21.15.60:0/2216716221 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.647+0000 7f5055c12640 1 --2- 172.21.15.60:0/2216716221 >> v2:172.21.15.60:3300/0 conn(0x7f50500d3730 0x7f50500b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.648+0000 7f5054c10640 1 --2- 172.21.15.60:0/2216716221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 0x7f50500d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.648+0000 7f504ffff640 1 -- 172.21.15.60:0/2216716221 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704960522 0 0) 0x7f50500b2b10 con 0x7f505015d330 2023-11-18T20:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.648+0000 7f504ffff640 1 -- 172.21.15.60:0/2216716221 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5038003520 con 0x7f505015d330 2023-11-18T20:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.648+0000 7f5055c12640 1 -- 172.21.15.60:0/2216716221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 msgr2=0x7f50500d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.648+0000 7f5055c12640 1 --2- 172.21.15.60:0/2216716221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 0x7f50500d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.648+0000 7f5055c12640 1 -- 172.21.15.60:0/2216716221 >> v1:172.21.15.60:6789/0 conn(0x7f505015d330 legacy=0x7f50500b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.648+0000 7f5055c12640 1 -- 172.21.15.60:0/2216716221 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500d1870 con 0x7f50500d3730 2023-11-18T20:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f5055c12640 1 --2- 172.21.15.60:0/2216716221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50500d3730 0x7f50500b6390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5040009d90 tx=0x7f504002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c18ddefd8c255d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f504ffff640 1 -- 172.21.15.60:0/2216716221 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504002cbc0 con 0x7f50500d3730 2023-11-18T20:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f504ffff640 1 -- 172.21.15.60:0/2216716221 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504002cd20 con 0x7f50500d3730 2023-11-18T20:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f504ffff640 1 -- 172.21.15.60:0/2216716221 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5040034660 con 0x7f50500d3730 2023-11-18T20:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f5057675640 1 -- 172.21.15.60:0/2216716221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50500d3730 msgr2=0x7f50500b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f5057675640 1 --2- 172.21.15.60:0/2216716221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50500d3730 0x7f50500b6390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5040009d90 tx=0x7f504002be10 comp rx=0 tx=0).stop 2023-11-18T20:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f5057675640 1 -- 172.21.15.60:0/2216716221 shutdown_connections 2023-11-18T20:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f5057675640 1 --2- 172.21.15.60:0/2216716221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50500d3730 0x7f50500b6390 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f5057675640 1 --2- 172.21.15.60:0/2216716221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 0x7f50500d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f5057675640 1 -- 172.21.15.60:0/2216716221 >> 172.21.15.60:0/2216716221 conn(0x7f5050072020 msgr2=0x7f5050072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f5057675640 1 -- 172.21.15.60:0/2216716221 shutdown_connections 2023-11-18T20:43:13.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.649+0000 7f5057675640 1 -- 172.21.15.60:0/2216716221 wait complete. 2023-11-18T20:43:13.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.651+0000 7f5057675640 1 Processor -- start 2023-11-18T20:43:13.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.651+0000 7f5057675640 1 -- start start 2023-11-18T20:43:13.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.651+0000 7f5057675640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 0x7f505015d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.651+0000 7f5057675640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50500d3730 0x7f50500cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.651+0000 7f5057675640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505015d330 0x7f505015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.651+0000 7f5057675640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505015dc50 con 0x7f505015d330 2023-11-18T20:43:13.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.651+0000 7f5057675640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505014c5b0 con 0x7f50500d3730 2023-11-18T20:43:13.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.651+0000 7f5057675640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505014c850 con 0x7f50500d2910 2023-11-18T20:43:13.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055c12640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505015d330 0x7f505015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055c12640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505015d330 0x7f505015ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49064/0 (socket says 172.21.15.60:49064) 2023-11-18T20:43:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055c12640 1 -- 172.21.15.60:0/4021057566 learned_addr learned my addr 172.21.15.60:0/4021057566 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5054c10640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50500d3730 0x7f50500cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055411640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 0x7f505015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055c12640 1 -- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 msgr2=0x7f505015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055c12640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 0x7f505015d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055c12640 1 -- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50500d3730 msgr2=0x7f50500cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055c12640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50500d3730 0x7f50500cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055c12640 1 -- 172.21.15.60:0/4021057566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5040009a70 con 0x7f505015d330 2023-11-18T20:43:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5054c10640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50500d3730 0x7f50500cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055411640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 0x7f505015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5055c12640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505015d330 0x7f505015ff20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5040037040 tx=0x7f5040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f504dffb640 1 -- 172.21.15.60:0/4021057566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5040040650 con 0x7f505015d330 2023-11-18T20:43:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f504dffb640 1 -- 172.21.15.60:0/4021057566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504002ce90 con 0x7f505015d330 2023-11-18T20:43:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5057675640 1 -- 172.21.15.60:0/4021057566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505014cb30 con 0x7f505015d330 2023-11-18T20:43:13.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f504dffb640 1 -- 172.21.15.60:0/4021057566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50400409f0 con 0x7f505015d330 2023-11-18T20:43:13.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.652+0000 7f5057675640 1 -- 172.21.15.60:0/4021057566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505014d0f0 con 0x7f505015d330 2023-11-18T20:43:13.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.653+0000 7f5057675640 1 -- 172.21.15.60:0/4021057566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50500d1df0 con 0x7f505015d330 2023-11-18T20:43:13.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.654+0000 7f504dffb640 1 -- 172.21.15.60:0/4021057566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5040040b50 con 0x7f505015d330 2023-11-18T20:43:13.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.654+0000 7f504dffb640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f503c06ae40 0x7f503c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.654+0000 7f504dffb640 1 -- 172.21.15.60:0/4021057566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50400ac230 con 0x7f505015d330 2023-11-18T20:43:13.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.655+0000 7f5055411640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f503c06ae40 0x7f503c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.655+0000 7f5055411640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f503c06ae40 0x7f503c06d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f50480011e0 tx=0x7f5048009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.658+0000 7f504dffb640 1 -- 172.21.15.60:0/4021057566 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50400772e0 con 0x7f505015d330 2023-11-18T20:43:13.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.836+0000 7f5057675640 1 -- 172.21.15.60:0/4021057566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50500b7260 con 0x7f505015d330 2023-11-18T20:43:13.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.837+0000 7f504dffb640 1 -- 172.21.15.60:0/4021057566 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f5040009450 con 0x7f505015d330 2023-11-18T20:43:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:13.840 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 -- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f503c06ae40 msgr2=0x7f503c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f503c06ae40 0x7f503c06d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f50480011e0 tx=0x7f5048009040 comp rx=0 tx=0).stop 2023-11-18T20:43:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 -- 172.21.15.60:0/4021057566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505015d330 msgr2=0x7f505015ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505015d330 0x7f505015ff20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5040037040 tx=0x7f5040004a40 comp rx=0 tx=0).stop 2023-11-18T20:43:13.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 -- 172.21.15.60:0/4021057566 shutdown_connections 2023-11-18T20:43:13.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f503c06ae40 0x7f503c06d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f505015d330 0x7f505015ff20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50500d3730 0x7f50500cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 --2- 172.21.15.60:0/4021057566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50500d2910 0x7f505015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 -- 172.21.15.60:0/4021057566 >> 172.21.15.60:0/4021057566 conn(0x7f5050072020 msgr2=0x7f50500cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:13.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 -- 172.21.15.60:0/4021057566 shutdown_connections 2023-11-18T20:43:13.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.839+0000 7f5057675640 1 -- 172.21.15.60:0/4021057566 wait complete. 2023-11-18T20:43:13.852 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:13.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:13.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.960+0000 7fed63301640 1 Processor -- start 2023-11-18T20:43:13.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.961+0000 7fed63301640 1 -- start start 2023-11-18T20:43:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.961+0000 7fed63301640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed5c15d330 0x7fed5c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.961+0000 7fed63301640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed5c0d2d90 0x7fed5c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.961+0000 7fed63301640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5c0b2b10 con 0x7fed5c0d2910 2023-11-18T20:43:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.961+0000 7fed63301640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5c0d1590 con 0x7fed5c0d2d90 2023-11-18T20:43:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.961+0000 7fed63301640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c0d1870 con 0x7fed5c15d330 2023-11-18T20:43:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.962+0000 7fed6189e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed5c0d2d90 0x7fed5c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.962+0000 7fed6189e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed5c0d2d90 0x7fed5c0b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49078/0 (socket says 172.21.15.60:49078) 2023-11-18T20:43:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.962+0000 7fed6189e640 1 -- 172.21.15.60:0/448644691 learned_addr learned my addr 172.21.15.60:0/448644691 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.962+0000 7fed6109d640 1 --2- 172.21.15.60:0/448644691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed5c15d330 0x7fed5c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.962+0000 7fed6189e640 1 -- 172.21.15.60:0/448644691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed5c15d330 msgr2=0x7fed5c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.962+0000 7fed6189e640 1 --2- 172.21.15.60:0/448644691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed5c15d330 0x7fed5c0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.962+0000 7fed6189e640 1 -- 172.21.15.60:0/448644691 >> v1:172.21.15.100:6789/0 conn(0x7fed5c0d2910 legacy=0x7fed5c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.962+0000 7fed6189e640 1 -- 172.21.15.60:0/448644691 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c0d1b50 con 0x7fed5c0d2d90 2023-11-18T20:43:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed6189e640 1 --2- 172.21.15.60:0/448644691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2d90 0x7fed5c0b6390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed5402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3965bb41a63ba541 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed5bfff640 1 -- 172.21.15.60:0/448644691 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54031640 con 0x7fed5c0d2d90 2023-11-18T20:43:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed5bfff640 1 -- 172.21.15.60:0/448644691 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed540317a0 con 0x7fed5c0d2d90 2023-11-18T20:43:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed5bfff640 1 -- 172.21.15.60:0/448644691 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54031b30 con 0x7fed5c0d2d90 2023-11-18T20:43:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed63301640 1 -- 172.21.15.60:0/448644691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2d90 msgr2=0x7fed5c0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed63301640 1 --2- 172.21.15.60:0/448644691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2d90 0x7fed5c0b6390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed54009870 tx=0x7fed5402b8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed63301640 1 -- 172.21.15.60:0/448644691 shutdown_connections 2023-11-18T20:43:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed63301640 1 --2- 172.21.15.60:0/448644691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2d90 0x7fed5c0b6390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed63301640 1 --2- 172.21.15.60:0/448644691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed5c15d330 0x7fed5c0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed63301640 1 -- 172.21.15.60:0/448644691 >> 172.21.15.60:0/448644691 conn(0x7fed5c072020 msgr2=0x7fed5c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed63301640 1 -- 172.21.15.60:0/448644691 shutdown_connections 2023-11-18T20:43:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.963+0000 7fed63301640 1 -- 172.21.15.60:0/448644691 wait complete. 2023-11-18T20:43:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.965+0000 7fed63301640 1 Processor -- start 2023-11-18T20:43:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.965+0000 7fed63301640 1 -- start start 2023-11-18T20:43:13.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.965+0000 7fed63301640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2910 0x7fed5c150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.965+0000 7fed63301640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed5c0d2d90 0x7fed5c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.965+0000 7fed63301640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed5c15d330 0x7fed5c15fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.965+0000 7fed63301640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c15eaf0 con 0x7fed5c0d2910 2023-11-18T20:43:13.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.965+0000 7fed63301640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c151260 con 0x7fed5c0d2d90 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.965+0000 7fed63301640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c15dee0 con 0x7fed5c15d330 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6109d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2910 0x7fed5c150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6109d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2910 0x7fed5c150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49092/0 (socket says 172.21.15.60:49092) 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6109d640 1 -- 172.21.15.60:0/10541729 learned_addr learned my addr 172.21.15.60:0/10541729 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6089c640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed5c0d2d90 0x7fed5c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6189e640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed5c15d330 0x7fed5c15fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6109d640 1 -- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed5c15d330 msgr2=0x7fed5c15fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6109d640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed5c15d330 0x7fed5c15fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6109d640 1 -- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed5c0d2d90 msgr2=0x7fed5c0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6109d640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed5c0d2d90 0x7fed5c0c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6109d640 1 -- 172.21.15.60:0/10541729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed540094d0 con 0x7fed5c0d2910 2023-11-18T20:43:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6089c640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed5c0d2d90 0x7fed5c0c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed6109d640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2910 0x7fed5c150d20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed4c007ef0 tx=0x7fed4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed59ffb640 1 -- 172.21.15.60:0/10541729 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c014620 con 0x7fed5c0d2910 2023-11-18T20:43:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed59ffb640 1 -- 172.21.15.60:0/10541729 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c014780 con 0x7fed5c0d2910 2023-11-18T20:43:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed59ffb640 1 -- 172.21.15.60:0/10541729 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c014b30 con 0x7fed5c0d2910 2023-11-18T20:43:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed63301640 1 -- 172.21.15.60:0/10541729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5c15e1c0 con 0x7fed5c0d2910 2023-11-18T20:43:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.966+0000 7fed63301640 1 -- 172.21.15.60:0/10541729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5c15e780 con 0x7fed5c0d2910 2023-11-18T20:43:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.968+0000 7fed63301640 1 -- 172.21.15.60:0/10541729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5c15e940 con 0x7fed5c0d2910 2023-11-18T20:43:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.968+0000 7fed59ffb640 1 -- 172.21.15.60:0/10541729 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed4c014c90 con 0x7fed5c0d2910 2023-11-18T20:43:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.968+0000 7fed59ffb640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed4806ae40 0x7fed4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:13.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.968+0000 7fed59ffb640 1 -- 172.21.15.60:0/10541729 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed4c089ff0 con 0x7fed5c0d2910 2023-11-18T20:43:13.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.969+0000 7fed6089c640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed4806ae40 0x7fed4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:13.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.969+0000 7fed6089c640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed4806ae40 0x7fed4806d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fed5c158940 tx=0x7fed50009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:13.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:13.973+0000 7fed59ffb640 1 -- 172.21.15.60:0/10541729 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed4c055090 con 0x7fed5c0d2910 2023-11-18T20:43:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.147+0000 7fed63301640 1 -- 172.21.15.60:0/10541729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed5c069400 con 0x7fed5c0d2910 2023-11-18T20:43:14.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.147+0000 7fed59ffb640 1 -- 172.21.15.60:0/10541729 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed4c0586b0 con 0x7fed5c0d2910 2023-11-18T20:43:14.150 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 -- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed4806ae40 msgr2=0x7fed4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed4806ae40 0x7fed4806d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fed5c158940 tx=0x7fed50009040 comp rx=0 tx=0).stop 2023-11-18T20:43:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 -- 172.21.15.60:0/10541729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2910 msgr2=0x7fed5c150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2910 0x7fed5c150d20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed4c007ef0 tx=0x7fed4c00e4b0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 -- 172.21.15.60:0/10541729 shutdown_connections 2023-11-18T20:43:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed4806ae40 0x7fed4806d300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed5c15d330 0x7fed5c15fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed5c0d2d90 0x7fed5c0c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 --2- 172.21.15.60:0/10541729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed5c0d2910 0x7fed5c150d20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 -- 172.21.15.60:0/10541729 >> 172.21.15.60:0/10541729 conn(0x7fed5c072020 msgr2=0x7fed5c0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 -- 172.21.15.60:0/10541729 shutdown_connections 2023-11-18T20:43:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.149+0000 7fed63301640 1 -- 172.21.15.60:0/10541729 wait complete. 2023-11-18T20:43:14.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:14.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.271+0000 7f48ab2bb640 1 Processor -- start 2023-11-18T20:43:14.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.272+0000 7f48ab2bb640 1 -- start start 2023-11-18T20:43:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.272+0000 7f48ab2bb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f48a40d2910 0x7f48a40d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.272+0000 7f48ab2bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40d3730 0x7f48a40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.272+0000 7f48ab2bb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a40b2b10 con 0x7f48a415d330 2023-11-18T20:43:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.272+0000 7f48ab2bb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a40d1590 con 0x7f48a40d2910 2023-11-18T20:43:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.272+0000 7f48ab2bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40d1870 con 0x7f48a40d3730 2023-11-18T20:43:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.272+0000 7f48a9057640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f48a415d330 0x7f48a40b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41260/0 (socket says 172.21.15.60:41260) 2023-11-18T20:43:14.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.272+0000 7f48a9057640 1 -- 172.21.15.60:0/2988387720 learned_addr learned my addr 172.21.15.60:0/2988387720 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715616354 0 0) 0x7f48a40b2b10 con 0x7f48a415d330 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a9858640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40d3730 0x7f48a40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a8856640 1 --2- 172.21.15.60:0/2988387720 >> v2:172.21.15.100:3300/0 conn(0x7f48a40d2910 0x7f48a40d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f488c003520 con 0x7f48a415d330 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4162425267 0 0) 0x7f488c003520 con 0x7f48a415d330 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40d3730 msgr2=0x7f48a40b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40d3730 0x7f48a40b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 >> v2:172.21.15.100:3300/0 conn(0x7f48a40d2910 msgr2=0x7f48a40d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 --2- 172.21.15.60:0/2988387720 >> v2:172.21.15.100:3300/0 conn(0x7f48a40d2910 0x7f48a40d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40d3b90 con 0x7f48a415d330 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f489c001670 con 0x7f48a415d330 2023-11-18T20:43:14.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 >> v1:172.21.15.60:6789/0 conn(0x7f48a415d330 legacy=0x7f48a40b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f488c003970 0x7f488c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40d3730 0x7f488c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f488c004d00 0x7f488c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40d1590 con 0x7f488c003970 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40d1870 con 0x7f488c004d00 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c00b9b0 con 0x7f48a40d3730 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a9057640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f488c003970 0x7f488c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a8856640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40d3730 0x7f488c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.273+0000 7f48a9858640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f488c004d00 0x7f488c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.274+0000 7f48a9057640 1 -- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40d3730 msgr2=0x7f488c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.274+0000 7f48a9057640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40d3730 0x7f488c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.274+0000 7f48a9057640 1 -- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f488c004d00 msgr2=0x7f488c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.274+0000 7f48a9057640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f488c004d00 0x7f488c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.274+0000 7f48a9057640 1 -- 172.21.15.60:0/2988387720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c003520 con 0x7f488c003970 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.274+0000 7f48a9057640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f488c003970 0x7f488c003d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f489c00c190 tx=0x7f488c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3950de367316dd58 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.274+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c005ad0 con 0x7f488c003970 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.274+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c005c30 con 0x7f488c003970 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.274+0000 7f48a3fff640 1 -- 172.21.15.60:0/2988387720 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c02c850 con 0x7f488c003970 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.275+0000 7f48ab2bb640 1 -- 172.21.15.60:0/2988387720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f488c003970 msgr2=0x7f488c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.275+0000 7f48ab2bb640 1 --2- 172.21.15.60:0/2988387720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f488c003970 0x7f488c003d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f489c00c190 tx=0x7f488c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.275+0000 7f48a9858640 1 -- 172.21.15.60:0/2988387720 reap_dead start 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.275+0000 7f48ab2bb640 1 -- 172.21.15.60:0/2988387720 shutdown_connections 2023-11-18T20:43:14.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.275+0000 7f48ab2bb640 1 -- 172.21.15.60:0/2988387720 >> 172.21.15.60:0/2988387720 conn(0x7f48a4072020 msgr2=0x7f48a4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.275+0000 7f48ab2bb640 1 -- 172.21.15.60:0/2988387720 shutdown_connections 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.275+0000 7f48ab2bb640 1 -- 172.21.15.60:0/2988387720 wait complete. 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.276+0000 7f48ab2bb640 1 Processor -- start 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.276+0000 7f48ab2bb640 1 -- start start 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48ab2bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a4149640 0x7f48a40c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48ab2bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a41f2bc0 0x7f48a41537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48ab2bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41600e0 0x7f48a4153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48ab2bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40c7e90 con 0x7f48a4149640 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48ab2bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40c9c10 con 0x7f48a41600e0 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48ab2bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40c7280 con 0x7f48a41f2bc0 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48a9057640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a4149640 0x7f48a40c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48a9057640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a4149640 0x7f48a40c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49104/0 (socket says 172.21.15.60:49104) 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48a9057640 1 -- 172.21.15.60:0/1032185457 learned_addr learned my addr 172.21.15.60:0/1032185457 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48a9858640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41600e0 0x7f48a4153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.277+0000 7f48a8856640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a41f2bc0 0x7f48a41537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48a9057640 1 -- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a41f2bc0 msgr2=0x7f48a41537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48a9057640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a41f2bc0 0x7f48a41537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48a9057640 1 -- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41600e0 msgr2=0x7f48a4153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48a9057640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41600e0 0x7f48a4153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48a9057640 1 -- 172.21.15.60:0/1032185457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40c7560 con 0x7f48a4149640 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48a9057640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a4149640 0x7f48a40c6ba0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f489c00c2b0 tx=0x7f489c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48a1ffb640 1 -- 172.21.15.60:0/1032185457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c03c040 con 0x7f48a4149640 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48a1ffb640 1 -- 172.21.15.60:0/1032185457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c02cce0 con 0x7f48a4149640 2023-11-18T20:43:14.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48ab2bb640 1 -- 172.21.15.60:0/1032185457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a40c7840 con 0x7f48a4149640 2023-11-18T20:43:14.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48a1ffb640 1 -- 172.21.15.60:0/1032185457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0072b0 con 0x7f48a4149640 2023-11-18T20:43:14.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.278+0000 7f48ab2bb640 1 -- 172.21.15.60:0/1032185457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a4149a20 con 0x7f48a4149640 2023-11-18T20:43:14.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.279+0000 7f48ab2bb640 1 -- 172.21.15.60:0/1032185457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a4069400 con 0x7f48a4149640 2023-11-18T20:43:14.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.280+0000 7f48a1ffb640 1 -- 172.21.15.60:0/1032185457 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f489c032070 con 0x7f48a4149640 2023-11-18T20:43:14.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.280+0000 7f48a1ffb640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f489006ae70 0x7f489006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.280+0000 7f48a1ffb640 1 -- 172.21.15.60:0/1032185457 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f489c0aaa00 con 0x7f48a4149640 2023-11-18T20:43:14.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.280+0000 7f48a8856640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f489006ae70 0x7f489006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.281+0000 7f48a8856640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f489006ae70 0x7f489006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48a415c450 tx=0x7f4898008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:14.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.284+0000 7f48a1ffb640 1 -- 172.21.15.60:0/1032185457 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f489c075ab0 con 0x7f48a4149640 2023-11-18T20:43:14.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.464+0000 7f48ab2bb640 1 -- 172.21.15.60:0/1032185457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48a40d33d0 con 0x7f48a4149640 2023-11-18T20:43:14.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.464+0000 7f48a1ffb640 1 -- 172.21.15.60:0/1032185457 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f489c046020 con 0x7f48a4149640 2023-11-18T20:43:14.467 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 -- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f489006ae70 msgr2=0x7f489006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f489006ae70 0x7f489006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48a415c450 tx=0x7f4898008040 comp rx=0 tx=0).stop 2023-11-18T20:43:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 -- 172.21.15.60:0/1032185457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a4149640 msgr2=0x7f48a40c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a4149640 0x7f48a40c6ba0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f489c00c2b0 tx=0x7f489c02e070 comp rx=0 tx=0).stop 2023-11-18T20:43:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 -- 172.21.15.60:0/1032185457 shutdown_connections 2023-11-18T20:43:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f489006ae70 0x7f489006d330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41600e0 0x7f48a4153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a41f2bc0 0x7f48a41537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 --2- 172.21.15.60:0/1032185457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a4149640 0x7f48a40c6ba0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 -- 172.21.15.60:0/1032185457 >> 172.21.15.60:0/1032185457 conn(0x7f48a4072020 msgr2=0x7f48a40b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 -- 172.21.15.60:0/1032185457 shutdown_connections 2023-11-18T20:43:14.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.466+0000 7f48ab2bb640 1 -- 172.21.15.60:0/1032185457 wait complete. 2023-11-18T20:43:14.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:14.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.586+0000 7f3f764e7640 1 Processor -- start 2023-11-18T20:43:14.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.586+0000 7f3f764e7640 1 -- start start 2023-11-18T20:43:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.587+0000 7f3f764e7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3f7015d330 0x7f3f700b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.587+0000 7f3f764e7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f700b2b10 con 0x7f3f700d3900 2023-11-18T20:43:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.587+0000 7f3f764e7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f700d1590 con 0x7f3f700d2910 2023-11-18T20:43:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.587+0000 7f3f764e7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f700d1870 con 0x7f3f7015d330 2023-11-18T20:43:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.587+0000 7f3f74a84640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3f700d3900 0x7f3f700b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41272/0 (socket says 172.21.15.60:41272) 2023-11-18T20:43:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.587+0000 7f3f74a84640 1 -- 172.21.15.60:0/2756066777 learned_addr learned my addr 172.21.15.60:0/2756066777 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965938190 0 0) 0x7f3f700b2b10 con 0x7f3f700d3900 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6ffff640 1 --2- 172.21.15.60:0/2756066777 >> v2:172.21.15.100:3300/0 conn(0x7f3f7015d330 0x7f3f700b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f5c003520 con 0x7f3f700d3900 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 713681475 0 0) 0x7f3f5c003520 con 0x7f3f700d3900 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 >> v2:172.21.15.100:3300/0 conn(0x7f3f7015d330 msgr2=0x7f3f700b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 --2- 172.21.15.60:0/2756066777 >> v2:172.21.15.100:3300/0 conn(0x7f3f7015d330 0x7f3f700b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 >> v1:172.21.15.100:6789/0 conn(0x7f3f700d2910 legacy=0x7f3f700d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f700d1b50 con 0x7f3f700d3900 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f3f680019f0 con 0x7f3f700d3900 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 >> v1:172.21.15.60:6789/0 conn(0x7f3f700d3900 legacy=0x7f3f700b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f5c003970 0x7f3f5c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5c0047c0 0x7f3f5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f700d1870 con 0x7f3f5c005180 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c00c120 con 0x7f3f5c003970 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c00c2e0 con 0x7f3f5c0047c0 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f74a84640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6ffff640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f5c003970 0x7f3f5c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f6f7fe640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5c0047c0 0x7f3f5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f74a84640 1 -- 172.21.15.60:0/2756066777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5c0047c0 msgr2=0x7f3f5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f74a84640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5c0047c0 0x7f3f5c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f74a84640 1 -- 172.21.15.60:0/2756066777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f5c003970 msgr2=0x7f3f5c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f74a84640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f5c003970 0x7f3f5c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.588+0000 7f3f74a84640 1 -- 172.21.15.60:0/2756066777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c003520 con 0x7f3f5c005180 2023-11-18T20:43:14.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.589+0000 7f3f74a84640 1 -- 172.21.15.60:0/2756066777 reap_dead start 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.589+0000 7f3f74a84640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f5c00b980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800a800 tx=0x7f3f5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2afa56588a11ce server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.589+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f680072d0 con 0x7f3f5c005180 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.589+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6802f070 con 0x7f3f5c005180 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.589+0000 7f3f6effd640 1 -- 172.21.15.60:0/2756066777 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f68007670 con 0x7f3f5c005180 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.590+0000 7f3f764e7640 1 -- 172.21.15.60:0/2756066777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 msgr2=0x7f3f5c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.590+0000 7f3f764e7640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f5c00b980 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800a800 tx=0x7f3f5c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.590+0000 7f3f764e7640 1 -- 172.21.15.60:0/2756066777 shutdown_connections 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.590+0000 7f3f764e7640 1 --2- 172.21.15.60:0/2756066777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f5c00b980 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.590+0000 7f3f764e7640 1 -- 172.21.15.60:0/2756066777 >> 172.21.15.60:0/2756066777 conn(0x7f3f70072020 msgr2=0x7f3f70072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.590+0000 7f3f764e7640 1 -- 172.21.15.60:0/2756066777 shutdown_connections 2023-11-18T20:43:14.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.590+0000 7f3f764e7640 1 -- 172.21.15.60:0/2756066777 wait complete. 2023-11-18T20:43:14.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.591+0000 7f3f764e7640 1 Processor -- start 2023-11-18T20:43:14.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.591+0000 7f3f764e7640 1 -- start start 2023-11-18T20:43:14.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f764e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f700c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f764e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70153f50 0x7f3f7015f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f764e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f7015f990 0x7f3f701532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f764e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f700cb5e0 con 0x7f3f5c005180 2023-11-18T20:43:14.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f764e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7015fe20 con 0x7f3f7015f990 2023-11-18T20:43:14.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f764e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f70160100 con 0x7f3f70153f50 2023-11-18T20:43:14.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f6ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f700c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f6ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f700c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49130/0 (socket says 172.21.15.60:49130) 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f6ffff640 1 -- 172.21.15.60:0/3633610977 learned_addr learned my addr 172.21.15.60:0/3633610977 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f6ffff640 1 -- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70153f50 msgr2=0x7f3f7015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f6ffff640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70153f50 0x7f3f7015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f6ffff640 1 -- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f7015f990 msgr2=0x7f3f701532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f6ffff640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f7015f990 0x7f3f701532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.592+0000 7f3f6ffff640 1 -- 172.21.15.60:0/3633610977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f701603e0 con 0x7f3f5c005180 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.593+0000 7f3f6ffff640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f700c9000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3f600118e0 tx=0x7f3f60011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.593+0000 7f3f6cff9640 1 -- 172.21.15.60:0/3633610977 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000faf0 con 0x7f3f5c005180 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.593+0000 7f3f6cff9640 1 -- 172.21.15.60:0/3633610977 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6000fc50 con 0x7f3f5c005180 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.593+0000 7f3f6cff9640 1 -- 172.21.15.60:0/3633610977 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60014440 con 0x7f3f5c005180 2023-11-18T20:43:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.593+0000 7f3f764e7640 1 -- 172.21.15.60:0/3633610977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f700cb270 con 0x7f3f5c005180 2023-11-18T20:43:14.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.593+0000 7f3f764e7640 1 -- 172.21.15.60:0/3633610977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f70153810 con 0x7f3f5c005180 2023-11-18T20:43:14.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.594+0000 7f3f764e7640 1 -- 172.21.15.60:0/3633610977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f70069400 con 0x7f3f5c005180 2023-11-18T20:43:14.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.594+0000 7f3f6cff9640 1 -- 172.21.15.60:0/3633610977 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f600145a0 con 0x7f3f5c005180 2023-11-18T20:43:14.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.595+0000 7f3f6cff9640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f5006ae40 0x7f3f5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.595+0000 7f3f6cff9640 1 -- 172.21.15.60:0/3633610977 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f60089e40 con 0x7f3f5c005180 2023-11-18T20:43:14.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.595+0000 7f3f6f7fe640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f5006ae40 0x7f3f5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.596+0000 7f3f6f7fe640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f5006ae40 0x7f3f5006d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3f58000c00 tx=0x7f3f58009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:14.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.599+0000 7f3f6cff9640 1 -- 172.21.15.60:0/3633610977 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f60054f80 con 0x7f3f5c005180 2023-11-18T20:43:14.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.773+0000 7f3f764e7640 1 -- 172.21.15.60:0/3633610977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f700c9bc0 con 0x7f3f5c005180 2023-11-18T20:43:14.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.773+0000 7f3f6cff9640 1 -- 172.21.15.60:0/3633610977 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f600585a0 con 0x7f3f5c005180 2023-11-18T20:43:14.776 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:14.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 -- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f5006ae40 msgr2=0x7f3f5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f5006ae40 0x7f3f5006d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3f58000c00 tx=0x7f3f58009040 comp rx=0 tx=0).stop 2023-11-18T20:43:14.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 -- 172.21.15.60:0/3633610977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 msgr2=0x7f3f700c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f700c9000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3f600118e0 tx=0x7f3f60011da0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 -- 172.21.15.60:0/3633610977 shutdown_connections 2023-11-18T20:43:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f5006ae40 0x7f3f5006d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f7015f990 0x7f3f701532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f70153f50 0x7f3f7015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 --2- 172.21.15.60:0/3633610977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c005180 0x7f3f700c9000 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 -- 172.21.15.60:0/3633610977 >> 172.21.15.60:0/3633610977 conn(0x7f3f70072020 msgr2=0x7f3f7015c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 -- 172.21.15.60:0/3633610977 shutdown_connections 2023-11-18T20:43:14.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.775+0000 7f3f764e7640 1 -- 172.21.15.60:0/3633610977 wait complete. 2023-11-18T20:43:14.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.893+0000 7f8d8f451640 1 Processor -- start 2023-11-18T20:43:14.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.894+0000 7f8d8f451640 1 -- start start 2023-11-18T20:43:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.894+0000 7f8d8f451640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d880d2b00 0x7f8d881682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.894+0000 7f8d8f451640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d881614e0 con 0x7f8d8815d2d0 2023-11-18T20:43:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.894+0000 7f8d8f451640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d881617c0 con 0x7f8d88160f70 2023-11-18T20:43:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.894+0000 7f8d8f451640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88161aa0 con 0x7f8d880d2b00 2023-11-18T20:43:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.894+0000 7f8d8d1ed640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8d8815d2d0 0x7f8d880b8b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41288/0 (socket says 172.21.15.60:41288) 2023-11-18T20:43:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.894+0000 7f8d8d1ed640 1 -- 172.21.15.60:0/54791214 learned_addr learned my addr 172.21.15.60:0/54791214 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d8d9ee640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d880d2b00 0x7f8d881682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742574947 0 0) 0x7f8d881614e0 con 0x7f8d8815d2d0 2023-11-18T20:43:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d70003520 con 0x7f8d8815d2d0 2023-11-18T20:43:14.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676132812 0 0) 0x7f8d881617c0 con 0x7f8d88160f70 2023-11-18T20:43:14.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d881614e0 con 0x7f8d88160f70 2023-11-18T20:43:14.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2983591352 0 0) 0x7f8d70003520 con 0x7f8d8815d2d0 2023-11-18T20:43:14.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d880d2b00 msgr2=0x7f8d881682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:14.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d880d2b00 0x7f8d881682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 >> v1:172.21.15.100:6789/0 conn(0x7f8d88160f70 legacy=0x7f8d880cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d88161d80 con 0x7f8d8815d2d0 2023-11-18T20:43:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f8d80001670 con 0x7f8d8815d2d0 2023-11-18T20:43:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 >> v1:172.21.15.60:6789/0 conn(0x7f8d8815d2d0 legacy=0x7f8d880b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d70003b30 0x7f8d70003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d880d2b00 0x7f8d700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d70004d00 0x7f8d70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88161aa0 con 0x7f8d70004d00 2023-11-18T20:43:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7000ba30 con 0x7f8d70003b30 2023-11-18T20:43:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7000bba0 con 0x7f8d880d2b00 2023-11-18T20:43:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d8d9ee640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d70004d00 0x7f8d70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d8d1ed640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d70003b30 0x7f8d70003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.895+0000 7f8d8c9ec640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d880d2b00 0x7f8d700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.896+0000 7f8d8d9ee640 1 -- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d880d2b00 msgr2=0x7f8d700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:14.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.896+0000 7f8d8d9ee640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d880d2b00 0x7f8d700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.896+0000 7f8d8d9ee640 1 -- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d70003b30 msgr2=0x7f8d70003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:14.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.896+0000 7f8d8d9ee640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d70003b30 0x7f8d70003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.896+0000 7f8d8d9ee640 1 -- 172.21.15.60:0/54791214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d70003520 con 0x7f8d70004d00 2023-11-18T20:43:14.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.896+0000 7f8d8d9ee640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d70004d00 0x7f8d70005180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009fc0 tx=0x7f8d700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e759e0b6ddfc17b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:14.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.896+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7802cc20 con 0x7f8d70004d00 2023-11-18T20:43:14.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.896+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7800f040 con 0x7f8d70004d00 2023-11-18T20:43:14.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.897+0000 7f8d87fff640 1 -- 172.21.15.60:0/54791214 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78034440 con 0x7f8d70004d00 2023-11-18T20:43:14.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.897+0000 7f8d8f451640 1 -- 172.21.15.60:0/54791214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d70004d00 msgr2=0x7f8d70005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.897+0000 7f8d8f451640 1 --2- 172.21.15.60:0/54791214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d70004d00 0x7f8d70005180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009fc0 tx=0x7f8d700057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.897+0000 7f8d8d9ee640 1 -- 172.21.15.60:0/54791214 reap_dead start 2023-11-18T20:43:14.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.897+0000 7f8d8f451640 1 -- 172.21.15.60:0/54791214 shutdown_connections 2023-11-18T20:43:14.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.897+0000 7f8d8f451640 1 -- 172.21.15.60:0/54791214 >> 172.21.15.60:0/54791214 conn(0x7f8d88072230 msgr2=0x7f8d880cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:14.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.897+0000 7f8d8f451640 1 -- 172.21.15.60:0/54791214 shutdown_connections 2023-11-18T20:43:14.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.897+0000 7f8d8f451640 1 -- 172.21.15.60:0/54791214 wait complete. 2023-11-18T20:43:14.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.898+0000 7f8d8f451640 1 Processor -- start 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.898+0000 7f8d8f451640 1 -- start start 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8f451640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d70004d00 0x7f8d88150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8f451640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d880cc4e0 0x7f8d880cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8f451640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d88156d40 0x7f8d881571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8f451640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88157710 con 0x7f8d880cc4e0 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8f451640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d881511f0 con 0x7f8d70004d00 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8f451640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880ca2d0 con 0x7f8d88156d40 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8c9ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d880cc4e0 0x7f8d880cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8c9ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d880cc4e0 0x7f8d880cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49154/0 (socket says 172.21.15.60:49154) 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8c9ec640 1 -- 172.21.15.60:0/2854977523 learned_addr learned my addr 172.21.15.60:0/2854977523 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8d1ed640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d70004d00 0x7f8d88150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8d9ee640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d88156d40 0x7f8d881571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8c9ec640 1 -- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d88156d40 msgr2=0x7f8d881571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8c9ec640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d88156d40 0x7f8d881571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8c9ec640 1 -- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d70004d00 msgr2=0x7f8d88150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:14.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8c9ec640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d70004d00 0x7f8d88150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:14.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8c9ec640 1 -- 172.21.15.60:0/2854977523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880cd050 con 0x7f8d880cc4e0 2023-11-18T20:43:14.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8d9ee640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d88156d40 0x7f8d881571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:14.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.899+0000 7f8d8d1ed640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d70004d00 0x7f8d88150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:14.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.900+0000 7f8d8c9ec640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d880cc4e0 0x7f8d880cc960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00f7d0 tx=0x7f8d7c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:14.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.900+0000 7f8d85ffb640 1 -- 172.21.15.60:0/2854977523 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c00fe80 con 0x7f8d880cc4e0 2023-11-18T20:43:14.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.900+0000 7f8d85ffb640 1 -- 172.21.15.60:0/2854977523 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c0052e0 con 0x7f8d880cc4e0 2023-11-18T20:43:14.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.900+0000 7f8d8f451640 1 -- 172.21.15.60:0/2854977523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d880ca570 con 0x7f8d880cc4e0 2023-11-18T20:43:14.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.900+0000 7f8d85ffb640 1 -- 172.21.15.60:0/2854977523 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c00d970 con 0x7f8d880cc4e0 2023-11-18T20:43:14.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.900+0000 7f8d8f451640 1 -- 172.21.15.60:0/2854977523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d880cab30 con 0x7f8d880cc4e0 2023-11-18T20:43:14.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.901+0000 7f8d8f451640 1 -- 172.21.15.60:0/2854977523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d880cacc0 con 0x7f8d880cc4e0 2023-11-18T20:43:14.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.901+0000 7f8d85ffb640 1 -- 172.21.15.60:0/2854977523 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d7c00dad0 con 0x7f8d880cc4e0 2023-11-18T20:43:14.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.902+0000 7f8d85ffb640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d7406ae70 0x7f8d7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:14.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.902+0000 7f8d85ffb640 1 -- 172.21.15.60:0/2854977523 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d7c089580 con 0x7f8d880cc4e0 2023-11-18T20:43:14.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.902+0000 7f8d8d1ed640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d7406ae70 0x7f8d7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:14.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.902+0000 7f8d8d1ed640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d7406ae70 0x7f8d7406d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8d80001eb0 tx=0x7f8d80009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:14.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:14.906+0000 7f8d85ffb640 1 -- 172.21.15.60:0/2854977523 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d7c0546c0 con 0x7f8d880cc4e0 2023-11-18T20:43:15.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.093+0000 7f8d8f451640 1 -- 172.21.15.60:0/2854977523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8d88069400 con 0x7f8d880cc4e0 2023-11-18T20:43:15.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.094+0000 7f8d85ffb640 1 -- 172.21.15.60:0/2854977523 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8d7c057ce0 con 0x7f8d880cc4e0 2023-11-18T20:43:15.096 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:15.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.095+0000 7f8d8f451640 1 -- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d7406ae70 msgr2=0x7f8d7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.095+0000 7f8d8f451640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d7406ae70 0x7f8d7406d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8d80001eb0 tx=0x7f8d80009040 comp rx=0 tx=0).stop 2023-11-18T20:43:15.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.096+0000 7f8d8f451640 1 -- 172.21.15.60:0/2854977523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d880cc4e0 msgr2=0x7f8d880cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.096+0000 7f8d8f451640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d880cc4e0 0x7f8d880cc960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00f7d0 tx=0x7f8d7c00fc90 comp rx=0 tx=0).stop 2023-11-18T20:43:15.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.096+0000 7f8d8f451640 1 -- 172.21.15.60:0/2854977523 shutdown_connections 2023-11-18T20:43:15.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.096+0000 7f8d8f451640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d7406ae70 0x7f8d7406d330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.096+0000 7f8d8f451640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d88156d40 0x7f8d881571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.096+0000 7f8d8f451640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d880cc4e0 0x7f8d880cc960 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.096+0000 7f8d8f451640 1 --2- 172.21.15.60:0/2854977523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d70004d00 0x7f8d88150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.096+0000 7f8d8f451640 1 -- 172.21.15.60:0/2854977523 >> 172.21.15.60:0/2854977523 conn(0x7f8d88072230 msgr2=0x7f8d880d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:15.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.096+0000 7f8d8f451640 1 -- 172.21.15.60:0/2854977523 shutdown_connections 2023-11-18T20:43:15.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.096+0000 7f8d8f451640 1 -- 172.21.15.60:0/2854977523 wait complete. 2023-11-18T20:43:15.108 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:43:15.108 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:15.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:15.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.269+0000 7f9d75d53640 1 Processor -- start 2023-11-18T20:43:15.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.269+0000 7f9d75d53640 1 -- start start 2023-11-18T20:43:15.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.270+0000 7f9d75d53640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9d70140310 0x7f9d701618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.270+0000 7f9d75d53640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9d701409d0 0x7f9d7006a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.270+0000 7f9d75d53640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d701406f0 con 0x7f9d701428d0 2023-11-18T20:43:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.270+0000 7f9d75d53640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d70161e20 con 0x7f9d70140310 2023-11-18T20:43:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.270+0000 7f9d75d53640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d70143960 con 0x7f9d701409d0 2023-11-18T20:43:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.270+0000 7f9d6f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9d70140310 0x7f9d701618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.270+0000 7f9d6ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9d701428d0 0x7f9d700714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41290/0 (socket says 172.21.15.60:41290) 2023-11-18T20:43:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.270+0000 7f9d6ffff640 1 -- 172.21.15.60:0/1783016375 learned_addr learned my addr 172.21.15.60:0/1783016375 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.270+0000 7f9d6effd640 1 --2- 172.21.15.60:0/1783016375 >> v2:172.21.15.100:3300/0 conn(0x7f9d701409d0 0x7f9d7006a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.271+0000 7f9d6f7fe640 1 -- 172.21.15.60:0/1783016375 >> v2:172.21.15.100:3300/0 conn(0x7f9d701409d0 msgr2=0x7f9d7006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.271+0000 7f9d6f7fe640 1 --2- 172.21.15.60:0/1783016375 >> v2:172.21.15.100:3300/0 conn(0x7f9d701409d0 0x7f9d7006a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.271+0000 7f9d6f7fe640 1 -- 172.21.15.60:0/1783016375 >> v1:172.21.15.60:6789/0 conn(0x7f9d701428d0 legacy=0x7f9d700714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.271+0000 7f9d6e7fc640 1 -- 172.21.15.60:0/1783016375 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541989616 0 0) 0x7f9d701406f0 con 0x7f9d701428d0 2023-11-18T20:43:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.271+0000 7f9d6f7fe640 1 -- 172.21.15.60:0/1783016375 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70142de0 con 0x7f9d70140310 2023-11-18T20:43:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.271+0000 7f9d6effd640 1 --2- 172.21.15.60:0/1783016375 >> v2:172.21.15.100:3300/0 conn(0x7f9d701409d0 0x7f9d7006a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.271+0000 7f9d6f7fe640 1 --2- 172.21.15.60:0/1783016375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d70140310 0x7f9d701618e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9d70161f60 tx=0x7f9d6002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539b4fb5a4b04752 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.271+0000 7f9d6e7fc640 1 -- 172.21.15.60:0/1783016375 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6002e070 con 0x7f9d70140310 2023-11-18T20:43:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.271+0000 7f9d6e7fc640 1 -- 172.21.15.60:0/1783016375 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d600315a0 con 0x7f9d70140310 2023-11-18T20:43:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.272+0000 7f9d6e7fc640 1 -- 172.21.15.60:0/1783016375 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60031930 con 0x7f9d70140310 2023-11-18T20:43:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.272+0000 7f9d75d53640 1 -- 172.21.15.60:0/1783016375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d70140310 msgr2=0x7f9d701618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.272+0000 7f9d75d53640 1 --2- 172.21.15.60:0/1783016375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d70140310 0x7f9d701618e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9d70161f60 tx=0x7f9d6002b840 comp rx=0 tx=0).stop 2023-11-18T20:43:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.272+0000 7f9d75d53640 1 -- 172.21.15.60:0/1783016375 shutdown_connections 2023-11-18T20:43:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.272+0000 7f9d75d53640 1 --2- 172.21.15.60:0/1783016375 >> v2:172.21.15.100:3300/0 conn(0x7f9d701409d0 0x7f9d7006a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.272+0000 7f9d75d53640 1 --2- 172.21.15.60:0/1783016375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d70140310 0x7f9d701618e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.272+0000 7f9d75d53640 1 -- 172.21.15.60:0/1783016375 >> 172.21.15.60:0/1783016375 conn(0x7f9d700b1e50 msgr2=0x7f9d700b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.272+0000 7f9d75d53640 1 -- 172.21.15.60:0/1783016375 shutdown_connections 2023-11-18T20:43:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.272+0000 7f9d75d53640 1 -- 172.21.15.60:0/1783016375 wait complete. 2023-11-18T20:43:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.273+0000 7f9d75d53640 1 Processor -- start 2023-11-18T20:43:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.273+0000 7f9d75d53640 1 -- start start 2023-11-18T20:43:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.274+0000 7f9d75d53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d70140310 0x7f9d7013d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.274+0000 7f9d75d53640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d701409d0 0x7f9d700b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.274+0000 7f9d75d53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d701428d0 0x7f9d70156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.274+0000 7f9d75d53640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700b48f0 con 0x7f9d701428d0 2023-11-18T20:43:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.274+0000 7f9d75d53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d701f3e50 con 0x7f9d70140310 2023-11-18T20:43:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.274+0000 7f9d75d53640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d700b4330 con 0x7f9d701409d0 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.274+0000 7f9d6ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d701428d0 0x7f9d70156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.274+0000 7f9d6ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d701428d0 0x7f9d70156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49174/0 (socket says 172.21.15.60:49174) 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.274+0000 7f9d6ffff640 1 -- 172.21.15.60:0/2671639785 learned_addr learned my addr 172.21.15.60:0/2671639785 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d6effd640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d701409d0 0x7f9d700b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d6f7fe640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d70140310 0x7f9d7013d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d6ffff640 1 -- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d701409d0 msgr2=0x7f9d700b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d6ffff640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d701409d0 0x7f9d700b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d6ffff640 1 -- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d70140310 msgr2=0x7f9d7013d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d6ffff640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d70140310 0x7f9d7013d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d6ffff640 1 -- 172.21.15.60:0/2671639785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d60009400 con 0x7f9d701428d0 2023-11-18T20:43:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d6ffff640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d701428d0 0x7f9d70156e50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c7e0 tx=0x7f9d6800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d57fff640 1 -- 172.21.15.60:0/2671639785 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6800a640 con 0x7f9d701428d0 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d57fff640 1 -- 172.21.15.60:0/2671639785 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6800a7c0 con 0x7f9d701428d0 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d75d53640 1 -- 172.21.15.60:0/2671639785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d700b4610 con 0x7f9d701428d0 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d57fff640 1 -- 172.21.15.60:0/2671639785 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6800ab70 con 0x7f9d701428d0 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.275+0000 7f9d75d53640 1 -- 172.21.15.60:0/2671639785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d701f4330 con 0x7f9d701428d0 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.276+0000 7f9d75d53640 1 -- 172.21.15.60:0/2671639785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d70069400 con 0x7f9d701428d0 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.276+0000 7f9d57fff640 1 -- 172.21.15.60:0/2671639785 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d6800acd0 con 0x7f9d701428d0 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.277+0000 7f9d57fff640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9d5c06ae40 0x7f9d5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.277+0000 7f9d57fff640 1 -- 172.21.15.60:0/2671639785 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d68088eb0 con 0x7f9d701428d0 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.277+0000 7f9d6f7fe640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9d5c06ae40 0x7f9d5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.278+0000 7f9d6f7fe640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9d5c06ae40 0x7f9d5c06d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9d60036000 tx=0x7f9d6003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:15.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.281+0000 7f9d57fff640 1 -- 172.21.15.60:0/2671639785 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d68053f60 con 0x7f9d701428d0 2023-11-18T20:43:15.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.458+0000 7f9d75d53640 1 -- 172.21.15.60:0/2671639785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d7006ea60 con 0x7f9d701428d0 2023-11-18T20:43:15.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.459+0000 7f9d57fff640 1 -- 172.21.15.60:0/2671639785 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f9d68018020 con 0x7f9d701428d0 2023-11-18T20:43:15.462 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:15.462 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:15.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 -- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9d5c06ae40 msgr2=0x7f9d5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9d5c06ae40 0x7f9d5c06d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9d60036000 tx=0x7f9d6003d040 comp rx=0 tx=0).stop 2023-11-18T20:43:15.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 -- 172.21.15.60:0/2671639785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d701428d0 msgr2=0x7f9d70156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d701428d0 0x7f9d70156e50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c7e0 tx=0x7f9d6800cca0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 -- 172.21.15.60:0/2671639785 shutdown_connections 2023-11-18T20:43:15.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9d5c06ae40 0x7f9d5c06d300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d701428d0 0x7f9d70156e50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d701409d0 0x7f9d700b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 --2- 172.21.15.60:0/2671639785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d70140310 0x7f9d7013d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 -- 172.21.15.60:0/2671639785 >> 172.21.15.60:0/2671639785 conn(0x7f9d700b1e50 msgr2=0x7f9d7013ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:15.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 -- 172.21.15.60:0/2671639785 shutdown_connections 2023-11-18T20:43:15.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.461+0000 7f9d75d53640 1 -- 172.21.15.60:0/2671639785 wait complete. 2023-11-18T20:43:15.474 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:15.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:15.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.581+0000 7f1811c34640 1 Processor -- start 2023-11-18T20:43:15.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.581+0000 7f1811c34640 1 -- start start 2023-11-18T20:43:15.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.582+0000 7f1811c34640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f180c0d2910 0x7f180c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.582+0000 7f1811c34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c0d3730 0x7f180c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.582+0000 7f1811c34640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c0b2b10 con 0x7f180c15d330 2023-11-18T20:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.582+0000 7f1811c34640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c0d1590 con 0x7f180c0d2910 2023-11-18T20:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.582+0000 7f1811c34640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0d1870 con 0x7f180c0d3730 2023-11-18T20:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.582+0000 7f180b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f180c15d330 0x7f180c0b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41306/0 (socket says 172.21.15.60:41306) 2023-11-18T20:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.582+0000 7f180b7fe640 1 -- 172.21.15.60:0/2219922152 learned_addr learned my addr 172.21.15.60:0/2219922152 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.583+0000 7f180affd640 1 --2- 172.21.15.60:0/2219922152 >> v2:172.21.15.100:3300/0 conn(0x7f180c0d2910 0x7f180c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.583+0000 7f180bfff640 1 --2- 172.21.15.60:0/2219922152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c0d3730 0x7f180c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.583+0000 7f180a7fc640 1 -- 172.21.15.60:0/2219922152 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237039664 0 0) 0x7f180c0b2b10 con 0x7f180c15d330 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.583+0000 7f180a7fc640 1 -- 172.21.15.60:0/2219922152 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17f4003520 con 0x7f180c15d330 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.583+0000 7f180a7fc640 1 -- 172.21.15.60:0/2219922152 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3517125096 0 0) 0x7f17f4003520 con 0x7f180c15d330 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.583+0000 7f180bfff640 1 -- 172.21.15.60:0/2219922152 >> v2:172.21.15.100:3300/0 conn(0x7f180c0d2910 msgr2=0x7f180c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.583+0000 7f180bfff640 1 --2- 172.21.15.60:0/2219922152 >> v2:172.21.15.100:3300/0 conn(0x7f180c0d2910 0x7f180c0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.583+0000 7f180bfff640 1 -- 172.21.15.60:0/2219922152 >> v1:172.21.15.60:6789/0 conn(0x7f180c15d330 legacy=0x7f180c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.583+0000 7f180bfff640 1 -- 172.21.15.60:0/2219922152 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0d3b90 con 0x7f180c0d3730 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.583+0000 7f180affd640 1 --2- 172.21.15.60:0/2219922152 >> v2:172.21.15.100:3300/0 conn(0x7f180c0d2910 0x7f180c0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.584+0000 7f180bfff640 1 --2- 172.21.15.60:0/2219922152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c0d3730 0x7f180c0b6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1804009d90 tx=0x7f180402bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f82475ad62c59d52 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.584+0000 7f180a7fc640 1 -- 172.21.15.60:0/2219922152 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc001670 con 0x7f180c0d3730 2023-11-18T20:43:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.584+0000 7f180a7fc640 1 -- 172.21.15.60:0/2219922152 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180402c810 con 0x7f180c0d3730 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.584+0000 7f180a7fc640 1 -- 172.21.15.60:0/2219922152 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180402cb80 con 0x7f180c0d3730 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.584+0000 7f1811c34640 1 -- 172.21.15.60:0/2219922152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c0d3730 msgr2=0x7f180c0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.584+0000 7f1811c34640 1 --2- 172.21.15.60:0/2219922152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c0d3730 0x7f180c0b6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1804009d90 tx=0x7f180402bce0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.585+0000 7f1811c34640 1 -- 172.21.15.60:0/2219922152 shutdown_connections 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.585+0000 7f1811c34640 1 --2- 172.21.15.60:0/2219922152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c0d3730 0x7f180c0b6390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.585+0000 7f1811c34640 1 --2- 172.21.15.60:0/2219922152 >> v2:172.21.15.100:3300/0 conn(0x7f180c0d2910 0x7f180c0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.585+0000 7f1811c34640 1 -- 172.21.15.60:0/2219922152 >> 172.21.15.60:0/2219922152 conn(0x7f180c072020 msgr2=0x7f180c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.585+0000 7f1811c34640 1 -- 172.21.15.60:0/2219922152 shutdown_connections 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.585+0000 7f1811c34640 1 -- 172.21.15.60:0/2219922152 wait complete. 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.586+0000 7f1811c34640 1 Processor -- start 2023-11-18T20:43:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.586+0000 7f1811c34640 1 -- start start 2023-11-18T20:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f1811c34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f180c0d2910 0x7f180c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f1811c34640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f180c0d3730 0x7f180c0b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f1811c34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c15d330 0x7f180c0b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f1811c34640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0cb530 con 0x7f180c0d3730 2023-11-18T20:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f1811c34640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c15e8f0 con 0x7f180c0d2910 2023-11-18T20:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f1811c34640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0cacb0 con 0x7f180c15d330 2023-11-18T20:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f180c0d3730 0x7f180c0b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f180c0d3730 0x7f180c0b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49186/0 (socket says 172.21.15.60:49186) 2023-11-18T20:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180affd640 1 -- 172.21.15.60:0/1332758999 learned_addr learned my addr 172.21.15.60:0/1332758999 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180bfff640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c15d330 0x7f180c0b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180b7fe640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f180c0d2910 0x7f180c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180affd640 1 -- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c15d330 msgr2=0x7f180c0b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180affd640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c15d330 0x7f180c0b8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180affd640 1 -- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f180c0d2910 msgr2=0x7f180c1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180affd640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f180c0d2910 0x7f180c1597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180affd640 1 -- 172.21.15.60:0/1332758999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1800009400 con 0x7f180c0d3730 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f180affd640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f180c0d3730 0x7f180c0b8130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f180000e960 tx=0x7f180000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f17f3fff640 1 -- 172.21.15.60:0/1332758999 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180000ca00 con 0x7f180c0d3730 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f17f3fff640 1 -- 172.21.15.60:0/1332758999 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180000cb60 con 0x7f180c0d3730 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f1811c34640 1 -- 172.21.15.60:0/1332758999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1804009a70 con 0x7f180c0d3730 2023-11-18T20:43:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.587+0000 7f17f3fff640 1 -- 172.21.15.60:0/1332758999 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800013470 con 0x7f180c0d3730 2023-11-18T20:43:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.588+0000 7f1811c34640 1 -- 172.21.15.60:0/1332758999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c0caf70 con 0x7f180c0d3730 2023-11-18T20:43:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.589+0000 7f1811c34640 1 -- 172.21.15.60:0/1332758999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c069400 con 0x7f180c0d3730 2023-11-18T20:43:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.589+0000 7f17f3fff640 1 -- 172.21.15.60:0/1332758999 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18000135d0 con 0x7f180c0d3730 2023-11-18T20:43:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.589+0000 7f17f3fff640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17f806ae40 0x7f17f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.590+0000 7f17f3fff640 1 -- 172.21.15.60:0/1332758999 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1800089800 con 0x7f180c0d3730 2023-11-18T20:43:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.590+0000 7f180b7fe640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17f806ae40 0x7f17f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.590+0000 7f180b7fe640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17f806ae40 0x7f17f806d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f17fc006b50 tx=0x7f17fc001dc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.594+0000 7f17f3fff640 1 -- 172.21.15.60:0/1332758999 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1800054960 con 0x7f180c0d3730 2023-11-18T20:43:15.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.768+0000 7f1811c34640 1 -- 172.21.15.60:0/1332758999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f180c0b2710 con 0x7f180c0d3730 2023-11-18T20:43:15.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.769+0000 7f17f3fff640 1 -- 172.21.15.60:0/1332758999 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1800057f80 con 0x7f180c0d3730 2023-11-18T20:43:15.771 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:15.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 -- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17f806ae40 msgr2=0x7f17f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17f806ae40 0x7f17f806d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f17fc006b50 tx=0x7f17fc001dc0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 -- 172.21.15.60:0/1332758999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f180c0d3730 msgr2=0x7f180c0b8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f180c0d3730 0x7f180c0b8130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f180000e960 tx=0x7f180000ee20 comp rx=0 tx=0).stop 2023-11-18T20:43:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 -- 172.21.15.60:0/1332758999 shutdown_connections 2023-11-18T20:43:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17f806ae40 0x7f17f806d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f180c15d330 0x7f180c0b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f180c0d3730 0x7f180c0b8130 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 --2- 172.21.15.60:0/1332758999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f180c0d2910 0x7f180c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 -- 172.21.15.60:0/1332758999 >> 172.21.15.60:0/1332758999 conn(0x7f180c072020 msgr2=0x7f180c072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:15.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 -- 172.21.15.60:0/1332758999 shutdown_connections 2023-11-18T20:43:15.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.771+0000 7f1811c34640 1 -- 172.21.15.60:0/1332758999 wait complete. 2023-11-18T20:43:15.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:15.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.891+0000 7f93a8b41640 1 Processor -- start 2023-11-18T20:43:15.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.891+0000 7f93a8b41640 1 -- start start 2023-11-18T20:43:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.892+0000 7f93a8b41640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93a40d2910 0x7f93a40d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.892+0000 7f93a8b41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 0x7f93a40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.892+0000 7f93a8b41640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a40b2b10 con 0x7f93a415d330 2023-11-18T20:43:15.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.892+0000 7f93a8b41640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a40d1590 con 0x7f93a40d2910 2023-11-18T20:43:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.892+0000 7f93a8b41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40d1870 con 0x7f93a40d3730 2023-11-18T20:43:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.892+0000 7f93a1d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93a40d2910 0x7f93a40d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.892+0000 7f93a1d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93a40d2910 0x7f93a40d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49190/0 (socket says 172.21.15.60:49190) 2023-11-18T20:43:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.892+0000 7f93a1d9b640 1 -- 172.21.15.60:0/3792143342 learned_addr learned my addr 172.21.15.60:0/3792143342 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.893+0000 7f93a2d9d640 1 --2- 172.21.15.60:0/3792143342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 0x7f93a40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.893+0000 7f93a1d9b640 1 -- 172.21.15.60:0/3792143342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 msgr2=0x7f93a40b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:15.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.893+0000 7f93a1d9b640 1 --2- 172.21.15.60:0/3792143342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 0x7f93a40b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.893+0000 7f93a1d9b640 1 -- 172.21.15.60:0/3792143342 >> v1:172.21.15.100:6789/0 conn(0x7f93a415d330 legacy=0x7f93a40b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:15.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.893+0000 7f93a1d9b640 1 -- 172.21.15.60:0/3792143342 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a40d3b90 con 0x7f93a40d2910 2023-11-18T20:43:15.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.893+0000 7f93a2d9d640 1 --2- 172.21.15.60:0/3792143342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 0x7f93a40b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a1d9b640 1 --2- 172.21.15.60:0/3792143342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a40d2910 0x7f93a40d31f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f938c009870 tx=0x7f938c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbcebe87155f1e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a159a640 1 -- 172.21.15.60:0/3792143342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c0317c0 con 0x7f93a40d2910 2023-11-18T20:43:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a159a640 1 -- 172.21.15.60:0/3792143342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c031920 con 0x7f93a40d2910 2023-11-18T20:43:15.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a159a640 1 -- 172.21.15.60:0/3792143342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c031cb0 con 0x7f93a40d2910 2023-11-18T20:43:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a8b41640 1 -- 172.21.15.60:0/3792143342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a40d2910 msgr2=0x7f93a40d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a8b41640 1 --2- 172.21.15.60:0/3792143342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a40d2910 0x7f93a40d31f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f938c009870 tx=0x7f938c02b860 comp rx=0 tx=0).stop 2023-11-18T20:43:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a8b41640 1 -- 172.21.15.60:0/3792143342 shutdown_connections 2023-11-18T20:43:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a8b41640 1 --2- 172.21.15.60:0/3792143342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a40d2910 0x7f93a40d31f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a8b41640 1 --2- 172.21.15.60:0/3792143342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 0x7f93a40b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a8b41640 1 -- 172.21.15.60:0/3792143342 >> 172.21.15.60:0/3792143342 conn(0x7f93a4072020 msgr2=0x7f93a4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:15.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a8b41640 1 -- 172.21.15.60:0/3792143342 shutdown_connections 2023-11-18T20:43:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.894+0000 7f93a8b41640 1 -- 172.21.15.60:0/3792143342 wait complete. 2023-11-18T20:43:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.896+0000 7f93a8b41640 1 Processor -- start 2023-11-18T20:43:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.896+0000 7f93a8b41640 1 -- start start 2023-11-18T20:43:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.896+0000 7f93a8b41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a40d2910 0x7f93a4151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.896+0000 7f93a8b41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 0x7f93a40b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.896+0000 7f93a8b41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a415d330 0x7f93a40b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.896+0000 7f93a8b41640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40cce80 con 0x7f93a415d330 2023-11-18T20:43:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.896+0000 7f93a8b41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40cd2a0 con 0x7f93a40d2910 2023-11-18T20:43:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.896+0000 7f93a8b41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40cc190 con 0x7f93a40d3730 2023-11-18T20:43:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a415d330 0x7f93a40b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a415d330 0x7f93a40b89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49196/0 (socket says 172.21.15.60:49196) 2023-11-18T20:43:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a2d9d640 1 -- 172.21.15.60:0/3119054266 learned_addr learned my addr 172.21.15.60:0/3119054266 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a1d9b640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 0x7f93a40b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a259c640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a40d2910 0x7f93a4151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a2d9d640 1 -- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 msgr2=0x7f93a40b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a2d9d640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 0x7f93a40b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a2d9d640 1 -- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a40d2910 msgr2=0x7f93a4151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a2d9d640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a40d2910 0x7f93a4151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a2d9d640 1 -- 172.21.15.60:0/3119054266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c0094d0 con 0x7f93a415d330 2023-11-18T20:43:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a259c640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a40d2910 0x7f93a4151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:15.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a1d9b640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 0x7f93a40b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a2d9d640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a415d330 0x7f93a40b89d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f939400cef0 tx=0x7f939400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f939affd640 1 -- 172.21.15.60:0/3119054266 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400d840 con 0x7f93a415d330 2023-11-18T20:43:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f939affd640 1 -- 172.21.15.60:0/3119054266 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939400d9c0 con 0x7f93a415d330 2023-11-18T20:43:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f939affd640 1 -- 172.21.15.60:0/3119054266 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400dd70 con 0x7f93a415d330 2023-11-18T20:43:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a8b41640 1 -- 172.21.15.60:0/3119054266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a40cc4e0 con 0x7f93a415d330 2023-11-18T20:43:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.897+0000 7f93a8b41640 1 -- 172.21.15.60:0/3119054266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a40ccaa0 con 0x7f93a415d330 2023-11-18T20:43:15.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.898+0000 7f93a8b41640 1 -- 172.21.15.60:0/3119054266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a4069400 con 0x7f93a415d330 2023-11-18T20:43:15.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.899+0000 7f939affd640 1 -- 172.21.15.60:0/3119054266 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f939401e3b0 con 0x7f93a415d330 2023-11-18T20:43:15.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.899+0000 7f939affd640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f938406ae40 0x7f938406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:15.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.899+0000 7f939affd640 1 -- 172.21.15.60:0/3119054266 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f939408a060 con 0x7f93a415d330 2023-11-18T20:43:15.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.900+0000 7f93a259c640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f938406ae40 0x7f938406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:15.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.900+0000 7f93a259c640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f938406ae40 0x7f938406d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f939c00a830 tx=0x7f939c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:15.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:15.903+0000 7f939affd640 1 -- 172.21.15.60:0/3119054266 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9394055260 con 0x7f93a415d330 2023-11-18T20:43:16.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.080+0000 7f93a8b41640 1 -- 172.21.15.60:0/3119054266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93a406a4d0 con 0x7f93a415d330 2023-11-18T20:43:16.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.081+0000 7f939affd640 1 -- 172.21.15.60:0/3119054266 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9394058880 con 0x7f93a415d330 2023-11-18T20:43:16.083 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:16.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 -- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f938406ae40 msgr2=0x7f938406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f938406ae40 0x7f938406d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f939c00a830 tx=0x7f939c008040 comp rx=0 tx=0).stop 2023-11-18T20:43:16.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 -- 172.21.15.60:0/3119054266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a415d330 msgr2=0x7f93a40b89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a415d330 0x7f93a40b89d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f939400cef0 tx=0x7f939400a710 comp rx=0 tx=0).stop 2023-11-18T20:43:16.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 -- 172.21.15.60:0/3119054266 shutdown_connections 2023-11-18T20:43:16.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f938406ae40 0x7f938406d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a415d330 0x7f93a40b89d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a40d3730 0x7f93a40b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 --2- 172.21.15.60:0/3119054266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a40d2910 0x7f93a4151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 -- 172.21.15.60:0/3119054266 >> 172.21.15.60:0/3119054266 conn(0x7f93a4072020 msgr2=0x7f93a4072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:16.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 -- 172.21.15.60:0/3119054266 shutdown_connections 2023-11-18T20:43:16.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.083+0000 7f93a8b41640 1 -- 172.21.15.60:0/3119054266 wait complete. 2023-11-18T20:43:16.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.203+0000 7f174bda9640 1 Processor -- start 2023-11-18T20:43:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.203+0000 7f174bda9640 1 -- start start 2023-11-18T20:43:16.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f174bda9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1744144a00 0x7f174406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f174bda9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1744144440 0x7f174406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f174bda9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17440b2eb0 con 0x7f17441415f0 2023-11-18T20:43:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f174bda9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1744144820 con 0x7f1744144440 2023-11-18T20:43:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f174bda9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17440b3190 con 0x7f1744144a00 2023-11-18T20:43:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f1749b45640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f17441415f0 0x7f1744141e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41310/0 (socket says 172.21.15.60:41310) 2023-11-18T20:43:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f1749b45640 1 -- 172.21.15.60:0/842083469 learned_addr learned my addr 172.21.15.60:0/842083469 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f174a346640 1 --2- 172.21.15.60:0/842083469 >> v2:172.21.15.60:3300/0 conn(0x7f1744144440 0x7f174406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f1749344640 1 --2- 172.21.15.60:0/842083469 >> v2:172.21.15.100:3300/0 conn(0x7f1744144a00 0x7f174406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f1748b43640 1 -- 172.21.15.60:0/842083469 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110376407 0 0) 0x7f17440b2eb0 con 0x7f17441415f0 2023-11-18T20:43:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f1748b43640 1 -- 172.21.15.60:0/842083469 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1730003520 con 0x7f17441415f0 2023-11-18T20:43:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f174a346640 1 -- 172.21.15.60:0/842083469 >> v2:172.21.15.100:3300/0 conn(0x7f1744144a00 msgr2=0x7f174406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f174a346640 1 --2- 172.21.15.60:0/842083469 >> v2:172.21.15.100:3300/0 conn(0x7f1744144a00 0x7f174406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f174a346640 1 -- 172.21.15.60:0/842083469 >> v1:172.21.15.60:6789/0 conn(0x7f17441415f0 legacy=0x7f1744141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.204+0000 7f174a346640 1 -- 172.21.15.60:0/842083469 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440b3470 con 0x7f1744144440 2023-11-18T20:43:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.205+0000 7f174a346640 1 --2- 172.21.15.60:0/842083469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144440 0x7f174406ba20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f174415fd30 tx=0x7f173c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878873d100b7c3fb server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:16.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.205+0000 7f1748b43640 1 -- 172.21.15.60:0/842083469 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c02cc80 con 0x7f1744144440 2023-11-18T20:43:16.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.205+0000 7f1748b43640 1 -- 172.21.15.60:0/842083469 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c02cde0 con 0x7f1744144440 2023-11-18T20:43:16.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.205+0000 7f1748b43640 1 -- 172.21.15.60:0/842083469 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c034660 con 0x7f1744144440 2023-11-18T20:43:16.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.206+0000 7f174bda9640 1 -- 172.21.15.60:0/842083469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144440 msgr2=0x7f174406ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.206+0000 7f174bda9640 1 --2- 172.21.15.60:0/842083469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144440 0x7f174406ba20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f174415fd30 tx=0x7f173c02bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.206+0000 7f174bda9640 1 -- 172.21.15.60:0/842083469 shutdown_connections 2023-11-18T20:43:16.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.206+0000 7f174bda9640 1 --2- 172.21.15.60:0/842083469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144440 0x7f174406ba20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.206+0000 7f174bda9640 1 --2- 172.21.15.60:0/842083469 >> v2:172.21.15.100:3300/0 conn(0x7f1744144a00 0x7f174406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.206+0000 7f174bda9640 1 -- 172.21.15.60:0/842083469 >> 172.21.15.60:0/842083469 conn(0x7f1744070e70 msgr2=0x7f1744071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:16.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.206+0000 7f174bda9640 1 -- 172.21.15.60:0/842083469 shutdown_connections 2023-11-18T20:43:16.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.206+0000 7f174bda9640 1 -- 172.21.15.60:0/842083469 wait complete. 2023-11-18T20:43:16.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.207+0000 7f174bda9640 1 Processor -- start 2023-11-18T20:43:16.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.207+0000 7f174bda9640 1 -- start start 2023-11-18T20:43:16.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174bda9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17441415f0 0x7f1744158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174bda9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1744144440 0x7f174413d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174bda9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144a00 0x7f174415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174bda9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744157c60 con 0x7f1744144a00 2023-11-18T20:43:16.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174bda9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744156fe0 con 0x7f17441415f0 2023-11-18T20:43:16.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174bda9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17441572c0 con 0x7f1744144440 2023-11-18T20:43:16.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174a346640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144a00 0x7f174415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174a346640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144a00 0x7f174415b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49214/0 (socket says 172.21.15.60:49214) 2023-11-18T20:43:16.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174a346640 1 -- 172.21.15.60:0/3412665978 learned_addr learned my addr 172.21.15.60:0/3412665978 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:16.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f1749b45640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17441415f0 0x7f1744158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f1749344640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1744144440 0x7f174413d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174a346640 1 -- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1744144440 msgr2=0x7f174413d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174a346640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1744144440 0x7f174413d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174a346640 1 -- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17441415f0 msgr2=0x7f1744158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174a346640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17441415f0 0x7f1744158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174a346640 1 -- 172.21.15.60:0/3412665978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c009a70 con 0x7f1744144a00 2023-11-18T20:43:16.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f174a346640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144a00 0x7f174415b510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f173c009e80 tx=0x7f173c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.208+0000 7f173a7fc640 1 -- 172.21.15.60:0/3412665978 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c040650 con 0x7f1744144a00 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.209+0000 7f173a7fc640 1 -- 172.21.15.60:0/3412665978 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c0407b0 con 0x7f1744144a00 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.209+0000 7f173a7fc640 1 -- 172.21.15.60:0/3412665978 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c040b60 con 0x7f1744144a00 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.209+0000 7f174bda9640 1 -- 172.21.15.60:0/3412665978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17441575a0 con 0x7f1744144a00 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.209+0000 7f174bda9640 1 -- 172.21.15.60:0/3412665978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174413d7f0 con 0x7f1744144a00 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.210+0000 7f174bda9640 1 -- 172.21.15.60:0/3412665978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1744157da0 con 0x7f1744144a00 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.210+0000 7f173a7fc640 1 -- 172.21.15.60:0/3412665978 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f173c040cc0 con 0x7f1744144a00 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.210+0000 7f173a7fc640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f172806ae40 0x7f172806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.211+0000 7f173a7fc640 1 -- 172.21.15.60:0/3412665978 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f173c0ac260 con 0x7f1744144a00 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.211+0000 7f1749b45640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f172806ae40 0x7f172806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.211+0000 7f1749b45640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f172806ae40 0x7f172806d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f17400011e0 tx=0x7f1740009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:16.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.215+0000 7f173a7fc640 1 -- 172.21.15.60:0/3412665978 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1744157da0 con 0x7f1744144a00 2023-11-18T20:43:16.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.391+0000 7f174bda9640 1 -- 172.21.15.60:0/3412665978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1744157da0 con 0x7f1744144a00 2023-11-18T20:43:16.394 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.391+0000 7f173a7fc640 1 -- 172.21.15.60:0/3412665978 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1744157da0 con 0x7f1744144a00 2023-11-18T20:43:16.394 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.393+0000 7f174bda9640 1 -- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f172806ae40 msgr2=0x7f172806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.393+0000 7f174bda9640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f172806ae40 0x7f172806d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f17400011e0 tx=0x7f1740009040 comp rx=0 tx=0).stop 2023-11-18T20:43:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.393+0000 7f174bda9640 1 -- 172.21.15.60:0/3412665978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144a00 msgr2=0x7f174415b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.393+0000 7f174bda9640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144a00 0x7f174415b510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f173c009e80 tx=0x7f173c004a40 comp rx=0 tx=0).stop 2023-11-18T20:43:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.393+0000 7f174bda9640 1 -- 172.21.15.60:0/3412665978 shutdown_connections 2023-11-18T20:43:16.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.393+0000 7f174bda9640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f172806ae40 0x7f172806d300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.393+0000 7f174bda9640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1744144a00 0x7f174415b510 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.393+0000 7f174bda9640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1744144440 0x7f174413d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.393+0000 7f174bda9640 1 --2- 172.21.15.60:0/3412665978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17441415f0 0x7f1744158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.393+0000 7f174bda9640 1 -- 172.21.15.60:0/3412665978 >> 172.21.15.60:0/3412665978 conn(0x7f1744070e70 msgr2=0x7f1744143b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:16.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.394+0000 7f174bda9640 1 -- 172.21.15.60:0/3412665978 shutdown_connections 2023-11-18T20:43:16.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.394+0000 7f174bda9640 1 -- 172.21.15.60:0/3412665978 wait complete. 2023-11-18T20:43:16.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.512+0000 7fd3501a9640 1 Processor -- start 2023-11-18T20:43:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.512+0000 7fd3501a9640 1 -- start start 2023-11-18T20:43:16.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd3501a9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd348140310 0x7fd3481618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd3501a9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd3481409d0 0x7fd34806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd3501a9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3481406f0 con 0x7fd3481428d0 2023-11-18T20:43:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd3501a9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd348161e20 con 0x7fd348140310 2023-11-18T20:43:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd3501a9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd348143960 con 0x7fd3481409d0 2023-11-18T20:43:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34df45640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd348140310 0x7fd3481618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34df45640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd348140310 0x7fd3481618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49230/0 (socket says 172.21.15.60:49230) 2023-11-18T20:43:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34df45640 1 -- 172.21.15.60:0/3674987996 learned_addr learned my addr 172.21.15.60:0/3674987996 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34d744640 1 --2- 172.21.15.60:0/3674987996 >> v2:172.21.15.100:3300/0 conn(0x7fd3481409d0 0x7fd34806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34df45640 1 -- 172.21.15.60:0/3674987996 >> v2:172.21.15.100:3300/0 conn(0x7fd3481409d0 msgr2=0x7fd34806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34df45640 1 --2- 172.21.15.60:0/3674987996 >> v2:172.21.15.100:3300/0 conn(0x7fd3481409d0 0x7fd34806a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34df45640 1 -- 172.21.15.60:0/3674987996 >> v1:172.21.15.100:6789/0 conn(0x7fd3481428d0 legacy=0x7fd3480714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34df45640 1 -- 172.21.15.60:0/3674987996 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348142de0 con 0x7fd348140310 2023-11-18T20:43:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34df45640 1 --2- 172.21.15.60:0/3674987996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd348140310 0x7fd3481618e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd348161f60 tx=0x7fd34402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f28393bd6eb036d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34cf43640 1 -- 172.21.15.60:0/3674987996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344031870 con 0x7fd348140310 2023-11-18T20:43:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34cf43640 1 -- 172.21.15.60:0/3674987996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3440319d0 con 0x7fd348140310 2023-11-18T20:43:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.514+0000 7fd34cf43640 1 -- 172.21.15.60:0/3674987996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344031d60 con 0x7fd348140310 2023-11-18T20:43:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.515+0000 7fd3501a9640 1 -- 172.21.15.60:0/3674987996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd348140310 msgr2=0x7fd3481618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.515+0000 7fd3501a9640 1 --2- 172.21.15.60:0/3674987996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd348140310 0x7fd3481618e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd348161f60 tx=0x7fd34402b910 comp rx=0 tx=0).stop 2023-11-18T20:43:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.515+0000 7fd3501a9640 1 -- 172.21.15.60:0/3674987996 shutdown_connections 2023-11-18T20:43:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.515+0000 7fd3501a9640 1 --2- 172.21.15.60:0/3674987996 >> v2:172.21.15.100:3300/0 conn(0x7fd3481409d0 0x7fd34806a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.515+0000 7fd3501a9640 1 --2- 172.21.15.60:0/3674987996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd348140310 0x7fd3481618e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.515+0000 7fd3501a9640 1 -- 172.21.15.60:0/3674987996 >> 172.21.15.60:0/3674987996 conn(0x7fd3480b1e50 msgr2=0x7fd3480b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.515+0000 7fd3501a9640 1 -- 172.21.15.60:0/3674987996 shutdown_connections 2023-11-18T20:43:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.515+0000 7fd3501a9640 1 -- 172.21.15.60:0/3674987996 wait complete. 2023-11-18T20:43:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.516+0000 7fd3501a9640 1 Processor -- start 2023-11-18T20:43:16.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.516+0000 7fd3501a9640 1 -- start start 2023-11-18T20:43:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd3501a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd348140310 0x7fd3481577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd3501a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3481409d0 0x7fd348132080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd3501a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3481428d0 0x7fd348154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd3501a9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd348157d10 con 0x7fd3481409d0 2023-11-18T20:43:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd3501a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481567a0 con 0x7fd348140310 2023-11-18T20:43:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd3501a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481325c0 con 0x7fd3481428d0 2023-11-18T20:43:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34d744640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3481409d0 0x7fd348132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34d744640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3481409d0 0x7fd348132080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49240/0 (socket says 172.21.15.60:49240) 2023-11-18T20:43:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34d744640 1 -- 172.21.15.60:0/2491715474 learned_addr learned my addr 172.21.15.60:0/2491715474 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:16.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34df45640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd348140310 0x7fd3481577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34e746640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3481428d0 0x7fd348154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34d744640 1 -- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3481428d0 msgr2=0x7fd348154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34d744640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3481428d0 0x7fd348154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34d744640 1 -- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd348140310 msgr2=0x7fd3481577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34d744640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd348140310 0x7fd3481577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34d744640 1 -- 172.21.15.60:0/2491715474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3440094d0 con 0x7fd3481409d0 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34df45640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd348140310 0x7fd3481577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd34d744640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3481409d0 0x7fd348132080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd33800ea10 tx=0x7fd33800eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd33e7fc640 1 -- 172.21.15.60:0/2491715474 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338009670 con 0x7fd3481409d0 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd33e7fc640 1 -- 172.21.15.60:0/2491715474 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33800ca00 con 0x7fd3481409d0 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd3501a9640 1 -- 172.21.15.60:0/2491715474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3481546a0 con 0x7fd3481409d0 2023-11-18T20:43:16.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.517+0000 7fd33e7fc640 1 -- 172.21.15.60:0/2491715474 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800cdb0 con 0x7fd3481409d0 2023-11-18T20:43:16.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.518+0000 7fd3501a9640 1 -- 172.21.15.60:0/2491715474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd348156cf0 con 0x7fd3481409d0 2023-11-18T20:43:16.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.518+0000 7fd3501a9640 1 -- 172.21.15.60:0/2491715474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd348069400 con 0x7fd3481409d0 2023-11-18T20:43:16.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.519+0000 7fd33e7fc640 1 -- 172.21.15.60:0/2491715474 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd338013470 con 0x7fd3481409d0 2023-11-18T20:43:16.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.519+0000 7fd33e7fc640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd32c06ae40 0x7fd32c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:16.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.519+0000 7fd33e7fc640 1 -- 172.21.15.60:0/2491715474 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3380897d0 con 0x7fd3481409d0 2023-11-18T20:43:16.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.520+0000 7fd34df45640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd32c06ae40 0x7fd32c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:16.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.520+0000 7fd34df45640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd32c06ae40 0x7fd32c06d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd344036000 tx=0x7fd34403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:16.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.525+0000 7fd33e7fc640 1 -- 172.21.15.60:0/2491715474 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd338054890 con 0x7fd3481409d0 2023-11-18T20:43:16.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.712+0000 7fd3501a9640 1 -- 172.21.15.60:0/2491715474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd34806ea60 con 0x7fd3481409d0 2023-11-18T20:43:16.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.712+0000 7fd33e7fc640 1 -- 172.21.15.60:0/2491715474 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd338057eb0 con 0x7fd3481409d0 2023-11-18T20:43:16.715 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:16.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 -- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd32c06ae40 msgr2=0x7fd32c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd32c06ae40 0x7fd32c06d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd344036000 tx=0x7fd34403c040 comp rx=0 tx=0).stop 2023-11-18T20:43:16.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 -- 172.21.15.60:0/2491715474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3481409d0 msgr2=0x7fd348132080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:16.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3481409d0 0x7fd348132080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd33800ea10 tx=0x7fd33800eed0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 -- 172.21.15.60:0/2491715474 shutdown_connections 2023-11-18T20:43:16.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd32c06ae40 0x7fd32c06d300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3481428d0 0x7fd348154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3481409d0 0x7fd348132080 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 --2- 172.21.15.60:0/2491715474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd348140310 0x7fd3481577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:16.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 -- 172.21.15.60:0/2491715474 >> 172.21.15.60:0/2491715474 conn(0x7fd3480b1e50 msgr2=0x7fd34806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:16.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 -- 172.21.15.60:0/2491715474 shutdown_connections 2023-11-18T20:43:16.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.714+0000 7fd3501a9640 1 -- 172.21.15.60:0/2491715474 wait complete. 2023-11-18T20:43:16.727 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-11-18T20:43:16.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:16.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-18T20:43:16.758 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:43:16.758 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:43:16.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:16.759 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-18T20:43:16.815 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:43:16.815 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:16.834 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:43:16.834 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:16.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.996+0000 7f17e8547640 1 Processor -- start 2023-11-18T20:43:16.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.996+0000 7f17e8547640 1 -- start start 2023-11-18T20:43:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.997+0000 7f17e8547640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f17e00ad640 0x7f17e006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.997+0000 7f17e8547640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17e006bbf0 0x7f17e006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.997+0000 7f17e8547640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e00adae0 con 0x7f17e015d1d0 2023-11-18T20:43:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.997+0000 7f17e8547640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e00addc0 con 0x7f17e006bbf0 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.997+0000 7f17e8547640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e00ae0a0 con 0x7f17e00ad640 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.997+0000 7f17e6ae4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17e006bbf0 0x7f17e006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.997+0000 7f17e62e3640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f17e015d1d0 0x7f17e006b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41314/0 (socket says 172.21.15.60:41314) 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.997+0000 7f17e62e3640 1 -- 172.21.15.60:0/490764403 learned_addr learned my addr 172.21.15.60:0/490764403 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.998+0000 7f17e5ae2640 1 --2- 172.21.15.60:0/490764403 >> v2:172.21.15.100:3300/0 conn(0x7f17e00ad640 0x7f17e006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.998+0000 7f17e6ae4640 1 -- 172.21.15.60:0/490764403 >> v2:172.21.15.100:3300/0 conn(0x7f17e00ad640 msgr2=0x7f17e006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.998+0000 7f17e6ae4640 1 --2- 172.21.15.60:0/490764403 >> v2:172.21.15.100:3300/0 conn(0x7f17e00ad640 0x7f17e006d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.998+0000 7f17e6ae4640 1 -- 172.21.15.60:0/490764403 >> v1:172.21.15.60:6789/0 conn(0x7f17e015d1d0 legacy=0x7f17e006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.998+0000 7f17e6ae4640 1 -- 172.21.15.60:0/490764403 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e00ae380 con 0x7f17e006bbf0 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.998+0000 7f17e52e1640 1 -- 172.21.15.60:0/490764403 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824319197 0 0) 0x7f17e00adae0 con 0x7f17e015d1d0 2023-11-18T20:43:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.998+0000 7f17e5ae2640 1 --2- 172.21.15.60:0/490764403 >> v2:172.21.15.100:3300/0 conn(0x7f17e00ad640 0x7f17e006d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.998+0000 7f17e6ae4640 1 --2- 172.21.15.60:0/490764403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 0x7f17e006d5f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f17e00cc440 tx=0x7f17d002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804b916841428f34 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e52e1640 1 -- 172.21.15.60:0/490764403 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d002cc80 con 0x7f17e006bbf0 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e52e1640 1 -- 172.21.15.60:0/490764403 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d002cde0 con 0x7f17e006bbf0 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e52e1640 1 -- 172.21.15.60:0/490764403 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d0034660 con 0x7f17e006bbf0 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e8547640 1 -- 172.21.15.60:0/490764403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 msgr2=0x7f17e006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e8547640 1 --2- 172.21.15.60:0/490764403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 0x7f17e006d5f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f17e00cc440 tx=0x7f17d002bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e8547640 1 -- 172.21.15.60:0/490764403 shutdown_connections 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e8547640 1 --2- 172.21.15.60:0/490764403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 0x7f17e006d5f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e8547640 1 --2- 172.21.15.60:0/490764403 >> v2:172.21.15.100:3300/0 conn(0x7f17e00ad640 0x7f17e006d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e8547640 1 -- 172.21.15.60:0/490764403 >> 172.21.15.60:0/490764403 conn(0x7f17e0070e70 msgr2=0x7f17e0071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e8547640 1 -- 172.21.15.60:0/490764403 shutdown_connections 2023-11-18T20:43:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:16.999+0000 7f17e8547640 1 -- 172.21.15.60:0/490764403 wait complete. 2023-11-18T20:43:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e8547640 1 Processor -- start 2023-11-18T20:43:17.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e8547640 1 -- start start 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e8547640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 0x7f17e015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e8547640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00ad640 0x7f17e00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e8547640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17e015d1d0 0x7f17e00cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e8547640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e015f7c0 con 0x7f17e006bbf0 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e8547640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e015faa0 con 0x7f17e00ad640 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e8547640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e015fd80 con 0x7f17e015d1d0 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e62e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 0x7f17e015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e62e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 0x7f17e015ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49262/0 (socket says 172.21.15.60:49262) 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.001+0000 7f17e62e3640 1 -- 172.21.15.60:0/2303480127 learned_addr learned my addr 172.21.15.60:0/2303480127 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17e6ae4640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17e015d1d0 0x7f17e00cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17e5ae2640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00ad640 0x7f17e00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17e62e3640 1 -- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17e015d1d0 msgr2=0x7f17e00cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17e62e3640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17e015d1d0 0x7f17e00cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17e62e3640 1 -- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00ad640 msgr2=0x7f17e00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17e62e3640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00ad640 0x7f17e00cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17e62e3640 1 -- 172.21.15.60:0/2303480127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d0009a70 con 0x7f17e006bbf0 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17e62e3640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 0x7f17e015ba80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f17d800c430 tx=0x7f17d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17deffd640 1 -- 172.21.15.60:0/2303480127 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d80074e0 con 0x7f17e006bbf0 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17deffd640 1 -- 172.21.15.60:0/2303480127 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d8007660 con 0x7f17e006bbf0 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17e8547640 1 -- 172.21.15.60:0/2303480127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e0160060 con 0x7f17e006bbf0 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17deffd640 1 -- 172.21.15.60:0/2303480127 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8004a20 con 0x7f17e006bbf0 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.002+0000 7f17e8547640 1 -- 172.21.15.60:0/2303480127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e0160620 con 0x7f17e006bbf0 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.003+0000 7f17e8547640 1 -- 172.21.15.60:0/2303480127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e0069400 con 0x7f17e006bbf0 2023-11-18T20:43:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.003+0000 7f17deffd640 1 -- 172.21.15.60:0/2303480127 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17d8004b80 con 0x7f17e006bbf0 2023-11-18T20:43:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.004+0000 7f17deffd640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17cc06ae40 0x7f17cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.004+0000 7f17deffd640 1 -- 172.21.15.60:0/2303480127 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17d8089570 con 0x7f17e006bbf0 2023-11-18T20:43:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.004+0000 7f17e5ae2640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17cc06ae40 0x7f17cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.006+0000 7f17e5ae2640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17cc06ae40 0x7f17cc06d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f17e006e910 tx=0x7f17d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.009+0000 7f17deffd640 1 -- 172.21.15.60:0/2303480127 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17d80544f0 con 0x7f17e006bbf0 2023-11-18T20:43:17.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.187+0000 7f17e8547640 1 -- 172.21.15.60:0/2303480127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17e00cf840 con 0x7f17e006bbf0 2023-11-18T20:43:17.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.188+0000 7f17deffd640 1 -- 172.21.15.60:0/2303480127 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f17d8057b10 con 0x7f17e006bbf0 2023-11-18T20:43:17.190 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:17.190 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:42:39.079283+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:17.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 -- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17cc06ae40 msgr2=0x7f17cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17cc06ae40 0x7f17cc06d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f17e006e910 tx=0x7f17d4008040 comp rx=0 tx=0).stop 2023-11-18T20:43:17.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 -- 172.21.15.60:0/2303480127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 msgr2=0x7f17e015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 0x7f17e015ba80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f17d800c430 tx=0x7f17d800c8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 -- 172.21.15.60:0/2303480127 shutdown_connections 2023-11-18T20:43:17.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f17cc06ae40 0x7f17cc06d300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17e015d1d0 0x7f17e00cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00ad640 0x7f17e00cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 --2- 172.21.15.60:0/2303480127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17e006bbf0 0x7f17e015ba80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 -- 172.21.15.60:0/2303480127 >> 172.21.15.60:0/2303480127 conn(0x7f17e0070e70 msgr2=0x7f17e00b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:17.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 -- 172.21.15.60:0/2303480127 shutdown_connections 2023-11-18T20:43:17.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.190+0000 7f17e8547640 1 -- 172.21.15.60:0/2303480127 wait complete. 2023-11-18T20:43:17.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:17.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:17.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc173501640 1 Processor -- start 2023-11-18T20:43:17.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc173501640 1 -- start start 2023-11-18T20:43:17.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc173501640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc16c0ceb70 0x7fc16c0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc173501640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc16c0b4370 0x7fc16c0b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc173501640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16c071770 con 0x7fc16c144980 2023-11-18T20:43:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc173501640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c15ced0 con 0x7fc16c0ceb70 2023-11-18T20:43:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc173501640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c0b4ce0 con 0x7fc16c0b4370 2023-11-18T20:43:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc170a9c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc16c0ceb70 0x7fc16c0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc17129d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc16c144980 0x7fc16c15c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41328/0 (socket says 172.21.15.60:41328) 2023-11-18T20:43:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc17129d640 1 -- 172.21.15.60:0/2536226579 learned_addr learned my addr 172.21.15.60:0/2536226579 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc171a9e640 1 --2- 172.21.15.60:0/2536226579 >> v2:172.21.15.100:3300/0 conn(0x7fc16c0b4370 0x7fc16c0b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc170a9c640 1 -- 172.21.15.60:0/2536226579 >> v2:172.21.15.100:3300/0 conn(0x7fc16c0b4370 msgr2=0x7fc16c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc170a9c640 1 --2- 172.21.15.60:0/2536226579 >> v2:172.21.15.100:3300/0 conn(0x7fc16c0b4370 0x7fc16c0b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc170a9c640 1 -- 172.21.15.60:0/2536226579 >> v1:172.21.15.60:6789/0 conn(0x7fc16c144980 legacy=0x7fc16c15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc170a9c640 1 -- 172.21.15.60:0/2536226579 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c0ccdc0 con 0x7fc16c0ceb70 2023-11-18T20:43:17.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.311+0000 7fc171a9e640 1 --2- 172.21.15.60:0/2536226579 >> v2:172.21.15.100:3300/0 conn(0x7fc16c0b4370 0x7fc16c0b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.312+0000 7fc170a9c640 1 --2- 172.21.15.60:0/2536226579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c0ceb70 0x7fc16c0cc450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0716e0 tx=0x7fc16002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939f03c68ae80dd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.312+0000 7fc16bfff640 1 -- 172.21.15.60:0/2536226579 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160031830 con 0x7fc16c0ceb70 2023-11-18T20:43:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.312+0000 7fc16bfff640 1 -- 172.21.15.60:0/2536226579 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160031990 con 0x7fc16c0ceb70 2023-11-18T20:43:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.312+0000 7fc16bfff640 1 -- 172.21.15.60:0/2536226579 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160031d20 con 0x7fc16c0ceb70 2023-11-18T20:43:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.313+0000 7fc173501640 1 -- 172.21.15.60:0/2536226579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c0ceb70 msgr2=0x7fc16c0cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.313+0000 7fc173501640 1 --2- 172.21.15.60:0/2536226579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c0ceb70 0x7fc16c0cc450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0716e0 tx=0x7fc16002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.313+0000 7fc173501640 1 -- 172.21.15.60:0/2536226579 shutdown_connections 2023-11-18T20:43:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.313+0000 7fc173501640 1 --2- 172.21.15.60:0/2536226579 >> v2:172.21.15.100:3300/0 conn(0x7fc16c0b4370 0x7fc16c0b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.313+0000 7fc173501640 1 --2- 172.21.15.60:0/2536226579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c0ceb70 0x7fc16c0cc450 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.313+0000 7fc173501640 1 -- 172.21.15.60:0/2536226579 >> 172.21.15.60:0/2536226579 conn(0x7fc16c06a4d0 msgr2=0x7fc16c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.313+0000 7fc173501640 1 -- 172.21.15.60:0/2536226579 shutdown_connections 2023-11-18T20:43:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.313+0000 7fc173501640 1 -- 172.21.15.60:0/2536226579 wait complete. 2023-11-18T20:43:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.314+0000 7fc173501640 1 Processor -- start 2023-11-18T20:43:17.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.314+0000 7fc173501640 1 -- start start 2023-11-18T20:43:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc173501640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc16c0b4370 0x7fc16c13b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc173501640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc16c0ceb70 0x7fc16c071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc173501640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c144980 0x7fc16c072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc173501640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c13c770 con 0x7fc16c144980 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc173501640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1f1820 con 0x7fc16c0ceb70 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc173501640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1f1540 con 0x7fc16c0b4370 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc171a9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c144980 0x7fc16c072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc171a9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c144980 0x7fc16c072040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49276/0 (socket says 172.21.15.60:49276) 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc171a9e640 1 -- 172.21.15.60:0/682122477 learned_addr learned my addr 172.21.15.60:0/682122477 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc17129d640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc16c0b4370 0x7fc16c13b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc170a9c640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc16c0ceb70 0x7fc16c071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc171a9e640 1 -- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc16c0b4370 msgr2=0x7fc16c13b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc171a9e640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc16c0b4370 0x7fc16c13b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc171a9e640 1 -- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc16c0ceb70 msgr2=0x7fc16c071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc171a9e640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc16c0ceb70 0x7fc16c071b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.315+0000 7fc171a9e640 1 -- 172.21.15.60:0/682122477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160009400 con 0x7fc16c144980 2023-11-18T20:43:17.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.316+0000 7fc171a9e640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c144980 0x7fc16c072040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00cf20 tx=0x7fc15c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.316+0000 7fc169ffb640 1 -- 172.21.15.60:0/682122477 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c00d840 con 0x7fc16c144980 2023-11-18T20:43:17.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.316+0000 7fc169ffb640 1 -- 172.21.15.60:0/682122477 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c00d9a0 con 0x7fc16c144980 2023-11-18T20:43:17.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.316+0000 7fc169ffb640 1 -- 172.21.15.60:0/682122477 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c00dd50 con 0x7fc16c144980 2023-11-18T20:43:17.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.316+0000 7fc173501640 1 -- 172.21.15.60:0/682122477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c13c120 con 0x7fc16c144980 2023-11-18T20:43:17.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.316+0000 7fc173501640 1 -- 172.21.15.60:0/682122477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c13bd80 con 0x7fc16c144980 2023-11-18T20:43:17.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.317+0000 7fc173501640 1 -- 172.21.15.60:0/682122477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c13c920 con 0x7fc16c144980 2023-11-18T20:43:17.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.317+0000 7fc169ffb640 1 -- 172.21.15.60:0/682122477 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc15c01e3b0 con 0x7fc16c144980 2023-11-18T20:43:17.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.318+0000 7fc169ffb640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc15806ae40 0x7fc15806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.318+0000 7fc169ffb640 1 -- 172.21.15.60:0/682122477 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc15c08a0e0 con 0x7fc16c144980 2023-11-18T20:43:17.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.318+0000 7fc17129d640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc15806ae40 0x7fc15806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.319+0000 7fc17129d640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc15806ae40 0x7fc15806d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc164005880 tx=0x7fc164009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.323+0000 7fc169ffb640 1 -- 172.21.15.60:0/682122477 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc16c13c920 con 0x7fc16c144980 2023-11-18T20:43:17.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.499+0000 7fc173501640 1 -- 172.21.15.60:0/682122477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc16c13c920 con 0x7fc16c144980 2023-11-18T20:43:17.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.499+0000 7fc169ffb640 1 -- 172.21.15.60:0/682122477 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc16c13c920 con 0x7fc16c144980 2023-11-18T20:43:17.501 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:17.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 -- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc15806ae40 msgr2=0x7fc15806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc15806ae40 0x7fc15806d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc164005880 tx=0x7fc164009040 comp rx=0 tx=0).stop 2023-11-18T20:43:17.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 -- 172.21.15.60:0/682122477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c144980 msgr2=0x7fc16c072040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c144980 0x7fc16c072040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00cf20 tx=0x7fc15c00a7d0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 -- 172.21.15.60:0/682122477 shutdown_connections 2023-11-18T20:43:17.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc15806ae40 0x7fc15806d300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc16c144980 0x7fc16c072040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc16c0ceb70 0x7fc16c071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 --2- 172.21.15.60:0/682122477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc16c0b4370 0x7fc16c13b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 -- 172.21.15.60:0/682122477 >> 172.21.15.60:0/682122477 conn(0x7fc16c06a4d0 msgr2=0x7fc16c15bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:17.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 -- 172.21.15.60:0/682122477 shutdown_connections 2023-11-18T20:43:17.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.501+0000 7fc173501640 1 -- 172.21.15.60:0/682122477 wait complete. 2023-11-18T20:43:17.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:17.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.622+0000 7f43f1015640 1 Processor -- start 2023-11-18T20:43:17.623 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.622+0000 7f43f1015640 1 -- start start 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.623+0000 7f43f1015640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43ec0b1400 0x7f43ec15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.623+0000 7f43f1015640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ec15e2a0 con 0x7f43ec15e730 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.623+0000 7f43f1015640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ec15b240 con 0x7f43ec143490 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.623+0000 7f43f1015640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ec15b520 con 0x7f43ec0b1400 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.623+0000 7f43ea59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43ec0b1400 0x7f43ec15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.623+0000 7f43eb59e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f43ec15e730 0x7f43ec15aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41338/0 (socket says 172.21.15.60:41338) 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.623+0000 7f43eb59e640 1 -- 172.21.15.60:0/4197210113 learned_addr learned my addr 172.21.15.60:0/4197210113 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.623+0000 7f43ea59c640 1 -- 172.21.15.60:0/4197210113 >> v1:172.21.15.100:6789/0 conn(0x7f43ec143490 legacy=0x7f43ec0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.624+0000 7f43ea59c640 1 -- 172.21.15.60:0/4197210113 >> v1:172.21.15.60:6789/0 conn(0x7f43ec15e730 legacy=0x7f43ec15aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.624+0000 7f43e9d9b640 1 -- 172.21.15.60:0/4197210113 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449071153 0 0) 0x7f43ec15e2a0 con 0x7f43ec15e730 2023-11-18T20:43:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.624+0000 7f43ea59c640 1 -- 172.21.15.60:0/4197210113 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec15dc50 con 0x7f43ec0b1400 2023-11-18T20:43:17.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.624+0000 7f43ea59c640 1 --2- 172.21.15.60:0/4197210113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec0b1400 0x7f43ec15a4e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43e00097a0 tx=0x7f43e002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ca61647a57521a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.624+0000 7f43e9d9b640 1 -- 172.21.15.60:0/4197210113 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0030610 con 0x7f43ec0b1400 2023-11-18T20:43:17.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.624+0000 7f43e9d9b640 1 -- 172.21.15.60:0/4197210113 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e0030770 con 0x7f43ec0b1400 2023-11-18T20:43:17.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.624+0000 7f43e9d9b640 1 -- 172.21.15.60:0/4197210113 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0030b00 con 0x7f43ec0b1400 2023-11-18T20:43:17.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.625+0000 7f43f1015640 1 -- 172.21.15.60:0/4197210113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec0b1400 msgr2=0x7f43ec15a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.625+0000 7f43f1015640 1 --2- 172.21.15.60:0/4197210113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec0b1400 0x7f43ec15a4e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43e00097a0 tx=0x7f43e002b960 comp rx=0 tx=0).stop 2023-11-18T20:43:17.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.625+0000 7f43f1015640 1 -- 172.21.15.60:0/4197210113 shutdown_connections 2023-11-18T20:43:17.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.625+0000 7f43f1015640 1 --2- 172.21.15.60:0/4197210113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec0b1400 0x7f43ec15a4e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.625+0000 7f43f1015640 1 -- 172.21.15.60:0/4197210113 >> 172.21.15.60:0/4197210113 conn(0x7f43ec06a4d0 msgr2=0x7f43ec06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:17.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.625+0000 7f43f1015640 1 -- 172.21.15.60:0/4197210113 shutdown_connections 2023-11-18T20:43:17.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.625+0000 7f43f1015640 1 -- 172.21.15.60:0/4197210113 wait complete. 2023-11-18T20:43:17.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.626+0000 7f43f1015640 1 Processor -- start 2023-11-18T20:43:17.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.627+0000 7f43f1015640 1 -- start start 2023-11-18T20:43:17.629 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.627+0000 7f43f1015640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43ec0b1400 0x7f43ec158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.627+0000 7f43f1015640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43ec143490 0x7f43ec159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.627+0000 7f43f1015640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec15e730 0x7f43ec159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.627+0000 7f43f1015640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec15c9c0 con 0x7f43ec15e730 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.627+0000 7f43f1015640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec142020 con 0x7f43ec143490 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.627+0000 7f43f1015640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec142300 con 0x7f43ec0b1400 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.627+0000 7f43eb59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec15e730 0x7f43ec159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.627+0000 7f43eb59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec15e730 0x7f43ec159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49292/0 (socket says 172.21.15.60:49292) 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.627+0000 7f43eb59e640 1 -- 172.21.15.60:0/1045396880 learned_addr learned my addr 172.21.15.60:0/1045396880 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43ead9d640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43ec0b1400 0x7f43ec158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43ea59c640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43ec143490 0x7f43ec159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43eb59e640 1 -- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43ec0b1400 msgr2=0x7f43ec158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43eb59e640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43ec0b1400 0x7f43ec158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43eb59e640 1 -- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43ec143490 msgr2=0x7f43ec159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43eb59e640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43ec143490 0x7f43ec159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43eb59e640 1 -- 172.21.15.60:0/1045396880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e0009400 con 0x7f43ec15e730 2023-11-18T20:43:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43eb59e640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec15e730 0x7f43ec159a60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00c780 tx=0x7f43dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43d37fe640 1 -- 172.21.15.60:0/1045396880 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00ce90 con 0x7f43ec15e730 2023-11-18T20:43:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43d37fe640 1 -- 172.21.15.60:0/1045396880 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc00a640 con 0x7f43ec15e730 2023-11-18T20:43:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43f1015640 1 -- 172.21.15.60:0/1045396880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec1425e0 con 0x7f43ec15e730 2023-11-18T20:43:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43d37fe640 1 -- 172.21.15.60:0/1045396880 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00a9d0 con 0x7f43ec15e730 2023-11-18T20:43:17.631 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.628+0000 7f43f1015640 1 -- 172.21.15.60:0/1045396880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec1591d0 con 0x7f43ec15e730 2023-11-18T20:43:17.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.629+0000 7f43f1015640 1 -- 172.21.15.60:0/1045396880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ec142b00 con 0x7f43ec15e730 2023-11-18T20:43:17.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.630+0000 7f43d37fe640 1 -- 172.21.15.60:0/1045396880 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f43dc00ab30 con 0x7f43ec15e730 2023-11-18T20:43:17.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.630+0000 7f43d37fe640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43d806ae40 0x7f43d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.630+0000 7f43d37fe640 1 -- 172.21.15.60:0/1045396880 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43dc088d80 con 0x7f43ec15e730 2023-11-18T20:43:17.632 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.630+0000 7f43ead9d640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43d806ae40 0x7f43d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.631+0000 7f43ead9d640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43d806ae40 0x7f43d806d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000f60 tx=0x7f43e4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.636 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.635+0000 7f43d37fe640 1 -- 172.21.15.60:0/1045396880 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43ec142b00 con 0x7f43ec15e730 2023-11-18T20:43:17.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.812+0000 7f43f1015640 1 -- 172.21.15.60:0/1045396880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43ec142b00 con 0x7f43ec15e730 2023-11-18T20:43:17.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.813+0000 7f43d37fe640 1 -- 172.21.15.60:0/1045396880 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43ec142b00 con 0x7f43ec15e730 2023-11-18T20:43:17.814 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 -- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43d806ae40 msgr2=0x7f43d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43d806ae40 0x7f43d806d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f43e4000f60 tx=0x7f43e4009040 comp rx=0 tx=0).stop 2023-11-18T20:43:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 -- 172.21.15.60:0/1045396880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec15e730 msgr2=0x7f43ec159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec15e730 0x7f43ec159a60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00c780 tx=0x7f43dc00cc40 comp rx=0 tx=0).stop 2023-11-18T20:43:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 -- 172.21.15.60:0/1045396880 shutdown_connections 2023-11-18T20:43:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43d806ae40 0x7f43d806d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43ec15e730 0x7f43ec159a60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43ec143490 0x7f43ec159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 --2- 172.21.15.60:0/1045396880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43ec0b1400 0x7f43ec158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 -- 172.21.15.60:0/1045396880 >> 172.21.15.60:0/1045396880 conn(0x7f43ec06a4d0 msgr2=0x7f43ec06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 -- 172.21.15.60:0/1045396880 shutdown_connections 2023-11-18T20:43:17.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.814+0000 7f43f1015640 1 -- 172.21.15.60:0/1045396880 wait complete. 2023-11-18T20:43:17.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:17.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.935+0000 7f90a560e640 1 Processor -- start 2023-11-18T20:43:17.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.935+0000 7f90a560e640 1 -- start start 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f90a560e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f90a006ea50 0x7f90a006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f90a560e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90a00d0ed0 0x7f90a00d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f90a560e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a0161d20 con 0x7f90a0160fb0 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f90a560e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a006f370 con 0x7f90a006ea50 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f90a560e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a015e760 con 0x7f90a00d0ed0 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f909effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f90a006ea50 0x7f90a006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f9097fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f90a0160fb0 0x7f90a0161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41352/0 (socket says 172.21.15.60:41352) 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f9097fff640 1 -- 172.21.15.60:0/2194726703 learned_addr learned my addr 172.21.15.60:0/2194726703 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f909f7fe640 1 --2- 172.21.15.60:0/2194726703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90a00d0ed0 0x7f90a00d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f909effd640 1 -- 172.21.15.60:0/2194726703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90a00d0ed0 msgr2=0x7f90a00d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:17.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f909effd640 1 --2- 172.21.15.60:0/2194726703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90a00d0ed0 0x7f90a00d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f909effd640 1 -- 172.21.15.60:0/2194726703 >> v1:172.21.15.60:6789/0 conn(0x7f90a0160fb0 legacy=0x7f90a0161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f909effd640 1 -- 172.21.15.60:0/2194726703 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a015d9a0 con 0x7f90a006ea50 2023-11-18T20:43:17.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.936+0000 7f909e7fc640 1 -- 172.21.15.60:0/2194726703 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148083610 0 0) 0x7f90a0161d20 con 0x7f90a0160fb0 2023-11-18T20:43:17.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.937+0000 7f909effd640 1 --2- 172.21.15.60:0/2194726703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 0x7f90a006ee30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f90900097a0 tx=0x7f909002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92b800de91d8ce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.937+0000 7f909e7fc640 1 -- 172.21.15.60:0/2194726703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090031830 con 0x7f90a006ea50 2023-11-18T20:43:17.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.937+0000 7f909e7fc640 1 -- 172.21.15.60:0/2194726703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9090031990 con 0x7f90a006ea50 2023-11-18T20:43:17.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.937+0000 7f909e7fc640 1 -- 172.21.15.60:0/2194726703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090031d20 con 0x7f90a006ea50 2023-11-18T20:43:17.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.938+0000 7f90a560e640 1 -- 172.21.15.60:0/2194726703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 msgr2=0x7f90a006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.938+0000 7f90a560e640 1 --2- 172.21.15.60:0/2194726703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 0x7f90a006ee30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f90900097a0 tx=0x7f909002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.938+0000 7f90a560e640 1 -- 172.21.15.60:0/2194726703 shutdown_connections 2023-11-18T20:43:17.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.938+0000 7f90a560e640 1 --2- 172.21.15.60:0/2194726703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90a00d0ed0 0x7f90a00d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.938+0000 7f90a560e640 1 --2- 172.21.15.60:0/2194726703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 0x7f90a006ee30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.938+0000 7f90a560e640 1 -- 172.21.15.60:0/2194726703 >> 172.21.15.60:0/2194726703 conn(0x7f90a006a6b0 msgr2=0x7f90a006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:17.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.938+0000 7f90a560e640 1 -- 172.21.15.60:0/2194726703 shutdown_connections 2023-11-18T20:43:17.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.938+0000 7f90a560e640 1 -- 172.21.15.60:0/2194726703 wait complete. 2023-11-18T20:43:17.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.939+0000 7f90a560e640 1 Processor -- start 2023-11-18T20:43:17.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.939+0000 7f90a560e640 1 -- start start 2023-11-18T20:43:17.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f90a560e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 0x7f90a015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f90a560e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a00d0ed0 0x7f90a013cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f90a560e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90a0160fb0 0x7f90a0157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f90a560e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a01f2d60 con 0x7f90a006ea50 2023-11-18T20:43:17.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f90a560e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a01592b0 con 0x7f90a00d0ed0 2023-11-18T20:43:17.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f90a560e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a013d310 con 0x7f90a0160fb0 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f909effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 0x7f90a015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f909effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 0x7f90a015a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49314/0 (socket says 172.21.15.60:49314) 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f909effd640 1 -- 172.21.15.60:0/2231551962 learned_addr learned my addr 172.21.15.60:0/2231551962 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f909f7fe640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90a0160fb0 0x7f90a0157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f909effd640 1 -- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90a0160fb0 msgr2=0x7f90a0157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f909effd640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90a0160fb0 0x7f90a0157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f909effd640 1 -- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a00d0ed0 msgr2=0x7f90a013cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f909effd640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a00d0ed0 0x7f90a013cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f909effd640 1 -- 172.21.15.60:0/2231551962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9090009400 con 0x7f90a006ea50 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.940+0000 7f909effd640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 0x7f90a015a220 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9090034040 tx=0x7f90900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.941+0000 7f90977fe640 1 -- 172.21.15.60:0/2231551962 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909003a8f0 con 0x7f90a006ea50 2023-11-18T20:43:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.941+0000 7f90977fe640 1 -- 172.21.15.60:0/2231551962 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909003aa70 con 0x7f90a006ea50 2023-11-18T20:43:17.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.941+0000 7f90a560e640 1 -- 172.21.15.60:0/2231551962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a0158730 con 0x7f90a006ea50 2023-11-18T20:43:17.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.941+0000 7f90977fe640 1 -- 172.21.15.60:0/2231551962 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909003ae20 con 0x7f90a006ea50 2023-11-18T20:43:17.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.941+0000 7f90a560e640 1 -- 172.21.15.60:0/2231551962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a0158cf0 con 0x7f90a006ea50 2023-11-18T20:43:17.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.942+0000 7f90a560e640 1 -- 172.21.15.60:0/2231551962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a0069400 con 0x7f90a006ea50 2023-11-18T20:43:17.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.942+0000 7f90977fe640 1 -- 172.21.15.60:0/2231551962 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f909004d440 con 0x7f90a006ea50 2023-11-18T20:43:17.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.943+0000 7f90977fe640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f908006ae40 0x7f908006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:17.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.943+0000 7f90977fe640 1 -- 172.21.15.60:0/2231551962 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90900ab6d0 con 0x7f90a006ea50 2023-11-18T20:43:17.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.943+0000 7f9097fff640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f908006ae40 0x7f908006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:17.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.944+0000 7f9097fff640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f908006ae40 0x7f908006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9088007fd0 tx=0x7f9088007920 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:17.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:17.947+0000 7f90977fe640 1 -- 172.21.15.60:0/2231551962 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9090076780 con 0x7f90a006ea50 2023-11-18T20:43:18.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.126+0000 7f90a560e640 1 -- 172.21.15.60:0/2231551962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90a00703b0 con 0x7f90a006ea50 2023-11-18T20:43:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.126+0000 7f90977fe640 1 -- 172.21.15.60:0/2231551962 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9090079da0 con 0x7f90a006ea50 2023-11-18T20:43:18.128 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:18.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.128+0000 7f90a560e640 1 -- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f908006ae40 msgr2=0x7f908006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.128+0000 7f90a560e640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f908006ae40 0x7f908006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9088007fd0 tx=0x7f9088007920 comp rx=0 tx=0).stop 2023-11-18T20:43:18.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.129+0000 7f90a560e640 1 -- 172.21.15.60:0/2231551962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 msgr2=0x7f90a015a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.129+0000 7f90a560e640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 0x7f90a015a220 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9090034040 tx=0x7f90900046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.129+0000 7f90a560e640 1 -- 172.21.15.60:0/2231551962 shutdown_connections 2023-11-18T20:43:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.129+0000 7f90a560e640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f908006ae40 0x7f908006d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.129+0000 7f90a560e640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90a0160fb0 0x7f90a0157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.129+0000 7f90a560e640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90a00d0ed0 0x7f90a013cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.129+0000 7f90a560e640 1 --2- 172.21.15.60:0/2231551962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90a006ea50 0x7f90a015a220 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.129+0000 7f90a560e640 1 -- 172.21.15.60:0/2231551962 >> 172.21.15.60:0/2231551962 conn(0x7f90a006a6b0 msgr2=0x7f90a006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.129+0000 7f90a560e640 1 -- 172.21.15.60:0/2231551962 shutdown_connections 2023-11-18T20:43:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.129+0000 7f90a560e640 1 -- 172.21.15.60:0/2231551962 wait complete. 2023-11-18T20:43:18.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:18.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.250+0000 7f25a047c640 1 Processor -- start 2023-11-18T20:43:18.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.250+0000 7f25a047c640 1 -- start start 2023-11-18T20:43:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f25a047c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f259806f210 0x7f2598140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f25a047c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2598140d40 0x7f259813f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f25a047c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259815ba40 con 0x7f259815c2f0 2023-11-18T20:43:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f25a047c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259815c6d0 con 0x7f2598140d40 2023-11-18T20:43:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f25a047c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259813e530 con 0x7f259806f210 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f259e218640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f259815c2f0 0x7f259806eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41356/0 (socket says 172.21.15.60:41356) 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f259e218640 1 -- 172.21.15.60:0/1585672516 learned_addr learned my addr 172.21.15.60:0/1585672516 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566729979 0 0) 0x7f259815ba40 con 0x7f259815c2f0 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f259ea19640 1 --2- 172.21.15.60:0/1585672516 >> v2:172.21.15.100:3300/0 conn(0x7f2598140d40 0x7f259813f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2584003520 con 0x7f259815c2f0 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.251+0000 7f259da17640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f259806f210 0x7f2598140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3448118765 0 0) 0x7f2584003520 con 0x7f259815c2f0 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f259806f210 msgr2=0x7f2598140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f259806f210 0x7f2598140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 >> v2:172.21.15.100:3300/0 conn(0x7f2598140d40 msgr2=0x7f259813f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 --2- 172.21.15.60:0/1585672516 >> v2:172.21.15.100:3300/0 conn(0x7f2598140d40 0x7f259813f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25981411d0 con 0x7f259815c2f0 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f2590001670 con 0x7f259815c2f0 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 >> v1:172.21.15.60:6789/0 conn(0x7f259815c2f0 legacy=0x7f259806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25840039a0 0x7f2584003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f259806f210 0x7f25840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2584004d00 0x7f2584005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259813e530 con 0x7f25840039a0 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258400b980 con 0x7f2584004d00 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258400bac0 con 0x7f259806f210 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259e218640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25840039a0 0x7f2584003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259da17640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f259806f210 0x7f25840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259ea19640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2584004d00 0x7f2584005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259e218640 1 -- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f259806f210 msgr2=0x7f25840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:18.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259e218640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f259806f210 0x7f25840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259e218640 1 -- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2584004d00 msgr2=0x7f2584005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:18.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259e218640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2584004d00 0x7f2584005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259e218640 1 -- 172.21.15.60:0/1585672516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2584003520 con 0x7f25840039a0 2023-11-18T20:43:18.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259da17640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f259806f210 0x7f25840047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:18.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.252+0000 7f259ea19640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2584004d00 0x7f2584005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:18.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.253+0000 7f259e218640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25840039a0 0x7f2584003dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f259000c210 tx=0x7f25840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4386026597f7edb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:18.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.253+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590005ad0 con 0x7f25840039a0 2023-11-18T20:43:18.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.253+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590005c30 con 0x7f25840039a0 2023-11-18T20:43:18.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.253+0000 7f259d216640 1 -- 172.21.15.60:0/1585672516 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259002c870 con 0x7f25840039a0 2023-11-18T20:43:18.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.254+0000 7f25a047c640 1 -- 172.21.15.60:0/1585672516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25840039a0 msgr2=0x7f2584003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.254+0000 7f25a047c640 1 --2- 172.21.15.60:0/1585672516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25840039a0 0x7f2584003dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f259000c210 tx=0x7f25840057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.260 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.254+0000 7f259ea19640 1 -- 172.21.15.60:0/1585672516 reap_dead start 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.254+0000 7f25a047c640 1 -- 172.21.15.60:0/1585672516 shutdown_connections 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.254+0000 7f25a047c640 1 -- 172.21.15.60:0/1585672516 >> 172.21.15.60:0/1585672516 conn(0x7f259806a6b0 msgr2=0x7f259806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.254+0000 7f25a047c640 1 -- 172.21.15.60:0/1585672516 shutdown_connections 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.254+0000 7f25a047c640 1 -- 172.21.15.60:0/1585672516 wait complete. 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.255+0000 7f25a047c640 1 Processor -- start 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.255+0000 7f25a047c640 1 -- start start 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f25a047c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25840039a0 0x7f259815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f25a047c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25980b42b0 0x7f2598155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f25a047c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f259815a080 0x7f259815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f25a047c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259815aa40 con 0x7f259815a080 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f25a047c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980d11b0 con 0x7f25980b42b0 2023-11-18T20:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f25a047c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2598151c20 con 0x7f25840039a0 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259ea19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f259815a080 0x7f259815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259ea19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f259815a080 0x7f259815a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49330/0 (socket says 172.21.15.60:49330) 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259ea19640 1 -- 172.21.15.60:0/1796967802 learned_addr learned my addr 172.21.15.60:0/1796967802 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259e218640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25840039a0 0x7f259815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259da17640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25980b42b0 0x7f2598155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259ea19640 1 -- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25840039a0 msgr2=0x7f259815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259ea19640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25840039a0 0x7f259815f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259ea19640 1 -- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25980b42b0 msgr2=0x7f2598155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259ea19640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25980b42b0 0x7f2598155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259ea19640 1 -- 172.21.15.60:0/1796967802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598152330 con 0x7f259815a080 2023-11-18T20:43:18.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259da17640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25980b42b0 0x7f2598155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259e218640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25840039a0 0x7f259815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f259ea19640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f259815a080 0x7f259815a500 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f25980b4180 tx=0x7f2594010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f258effd640 1 -- 172.21.15.60:0/1796967802 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259400e050 con 0x7f259815a080 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f258effd640 1 -- 172.21.15.60:0/1796967802 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259400f040 con 0x7f259815a080 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.256+0000 7f258effd640 1 -- 172.21.15.60:0/1796967802 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594010e00 con 0x7f259815a080 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.257+0000 7f25a047c640 1 -- 172.21.15.60:0/1796967802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25981525d0 con 0x7f259815a080 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.257+0000 7f25a047c640 1 -- 172.21.15.60:0/1796967802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25980d0e40 con 0x7f259815a080 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.258+0000 7f25a047c640 1 -- 172.21.15.60:0/1796967802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25980d1000 con 0x7f259815a080 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.258+0000 7f258effd640 1 -- 172.21.15.60:0/1796967802 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2594015440 con 0x7f259815a080 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.258+0000 7f258effd640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f257c06ae40 0x7f257c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.258+0000 7f258effd640 1 -- 172.21.15.60:0/1796967802 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2594089c90 con 0x7f259815a080 2023-11-18T20:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.259+0000 7f259e218640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f257c06ae40 0x7f257c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.259+0000 7f259e218640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f257c06ae40 0x7f257c06d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2590034000 tx=0x7f2590037040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.263+0000 7f258effd640 1 -- 172.21.15.60:0/1796967802 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2594054d50 con 0x7f259815a080 2023-11-18T20:43:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.458+0000 7f25a047c640 1 -- 172.21.15.60:0/1796967802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f25980d09c0 con 0x7f259815a080 2023-11-18T20:43:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.459+0000 7f258effd640 1 -- 172.21.15.60:0/1796967802 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f25980d09c0 con 0x7f259815a080 2023-11-18T20:43:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 -- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f257c06ae40 msgr2=0x7f257c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f257c06ae40 0x7f257c06d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2590034000 tx=0x7f2590037040 comp rx=0 tx=0).stop 2023-11-18T20:43:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 -- 172.21.15.60:0/1796967802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f259815a080 msgr2=0x7f259815a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f259815a080 0x7f259815a500 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f25980b4180 tx=0x7f2594010590 comp rx=0 tx=0).stop 2023-11-18T20:43:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 -- 172.21.15.60:0/1796967802 shutdown_connections 2023-11-18T20:43:18.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f257c06ae40 0x7f257c06d300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f259815a080 0x7f259815a500 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25980b42b0 0x7f2598155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 --2- 172.21.15.60:0/1796967802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25840039a0 0x7f259815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 -- 172.21.15.60:0/1796967802 >> 172.21.15.60:0/1796967802 conn(0x7f259806a6b0 msgr2=0x7f259806eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:18.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 -- 172.21.15.60:0/1796967802 shutdown_connections 2023-11-18T20:43:18.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.461+0000 7f25a047c640 1 -- 172.21.15.60:0/1796967802 wait complete. 2023-11-18T20:43:18.472 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:43:18.473 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:18.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.597+0000 7efda4f41640 1 Processor -- start 2023-11-18T20:43:18.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.597+0000 7efda4f41640 1 -- start start 2023-11-18T20:43:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efda4f41640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efda0150940 0x7efda015d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efda4f41640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efda015dec0 0x7efda015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efda4f41640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda00b49f0 con 0x7efda0150560 2023-11-18T20:43:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efda4f41640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda00b3560 con 0x7efda0150940 2023-11-18T20:43:18.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efda4f41640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda00b3de0 con 0x7efda015dec0 2023-11-18T20:43:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efd9dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efda0150940 0x7efda015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efd9e59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efda0150560 0x7efda00b2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41364/0 (socket says 172.21.15.60:41364) 2023-11-18T20:43:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efd9e59c640 1 -- 172.21.15.60:0/3710820626 learned_addr learned my addr 172.21.15.60:0/3710820626 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efd9dd9b640 1 -- 172.21.15.60:0/3710820626 >> v2:172.21.15.100:3300/0 conn(0x7efda015dec0 msgr2=0x7efda015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efd9ed9d640 1 --2- 172.21.15.60:0/3710820626 >> v2:172.21.15.100:3300/0 conn(0x7efda015dec0 0x7efda015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efd9dd9b640 1 --2- 172.21.15.60:0/3710820626 >> v2:172.21.15.100:3300/0 conn(0x7efda015dec0 0x7efda015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efd9dd9b640 1 -- 172.21.15.60:0/3710820626 >> v1:172.21.15.60:6789/0 conn(0x7efda0150560 legacy=0x7efda00b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efd9dd9b640 1 -- 172.21.15.60:0/3710820626 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda00b40c0 con 0x7efda0150940 2023-11-18T20:43:18.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.598+0000 7efd9d59a640 1 -- 172.21.15.60:0/3710820626 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590039953 0 0) 0x7efda00b49f0 con 0x7efda0150560 2023-11-18T20:43:18.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.599+0000 7efd9dd9b640 1 --2- 172.21.15.60:0/3710820626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150940 0x7efda015d200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efda015d160 tx=0x7efd9402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67638ee8d4e9fb00 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:18.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.599+0000 7efd9d59a640 1 -- 172.21.15.60:0/3710820626 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9402f8a0 con 0x7efda0150940 2023-11-18T20:43:18.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.599+0000 7efd9d59a640 1 -- 172.21.15.60:0/3710820626 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9402fa00 con 0x7efda0150940 2023-11-18T20:43:18.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.599+0000 7efd9d59a640 1 -- 172.21.15.60:0/3710820626 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9402fd90 con 0x7efda0150940 2023-11-18T20:43:18.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.600+0000 7efda4f41640 1 -- 172.21.15.60:0/3710820626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150940 msgr2=0x7efda015d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.600+0000 7efda4f41640 1 --2- 172.21.15.60:0/3710820626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150940 0x7efda015d200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efda015d160 tx=0x7efd9402dae0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.600+0000 7efda4f41640 1 -- 172.21.15.60:0/3710820626 shutdown_connections 2023-11-18T20:43:18.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.600+0000 7efda4f41640 1 --2- 172.21.15.60:0/3710820626 >> v2:172.21.15.100:3300/0 conn(0x7efda015dec0 0x7efda015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.600+0000 7efda4f41640 1 --2- 172.21.15.60:0/3710820626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150940 0x7efda015d200 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.600+0000 7efda4f41640 1 -- 172.21.15.60:0/3710820626 >> 172.21.15.60:0/3710820626 conn(0x7efda006a4d0 msgr2=0x7efda006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:18.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.600+0000 7efda4f41640 1 -- 172.21.15.60:0/3710820626 shutdown_connections 2023-11-18T20:43:18.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.600+0000 7efda4f41640 1 -- 172.21.15.60:0/3710820626 wait complete. 2023-11-18T20:43:18.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.601+0000 7efda4f41640 1 Processor -- start 2023-11-18T20:43:18.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.601+0000 7efda4f41640 1 -- start start 2023-11-18T20:43:18.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efda4f41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150560 0x7efda01e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efda4f41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015dec0 0x7efda015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efda4f41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda01454f0 0x7efda0145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efda4f41640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda00ad8f0 con 0x7efda0150560 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efda4f41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda00ad610 con 0x7efda01454f0 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efda4f41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda01e31e0 con 0x7efda015dec0 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150560 0x7efda01e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150560 0x7efda01e3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49348/0 (socket says 172.21.15.60:49348) 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9e59c640 1 -- 172.21.15.60:0/1850393986 learned_addr learned my addr 172.21.15.60:0/1850393986 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9dd9b640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015dec0 0x7efda015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9ed9d640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda01454f0 0x7efda0145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9e59c640 1 -- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015dec0 msgr2=0x7efda015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9e59c640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015dec0 0x7efda015c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9e59c640 1 -- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda01454f0 msgr2=0x7efda0145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9e59c640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda01454f0 0x7efda0145980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9e59c640 1 -- 172.21.15.60:0/1850393986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd94009400 con 0x7efda0150560 2023-11-18T20:43:18.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd9e59c640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150560 0x7efda01e3cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd9800c4d0 tx=0x7efd9800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:18.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.602+0000 7efd8effd640 1 -- 172.21.15.60:0/1850393986 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980074e0 con 0x7efda0150560 2023-11-18T20:43:18.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.603+0000 7efd8effd640 1 -- 172.21.15.60:0/1850393986 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd98007660 con 0x7efda0150560 2023-11-18T20:43:18.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.603+0000 7efda4f41640 1 -- 172.21.15.60:0/1850393986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda01e34c0 con 0x7efda0150560 2023-11-18T20:43:18.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.603+0000 7efd8effd640 1 -- 172.21.15.60:0/1850393986 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98004a20 con 0x7efda0150560 2023-11-18T20:43:18.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.603+0000 7efda4f41640 1 -- 172.21.15.60:0/1850393986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda0145ec0 con 0x7efda0150560 2023-11-18T20:43:18.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.604+0000 7efda4f41640 1 -- 172.21.15.60:0/1850393986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda01e3040 con 0x7efda0150560 2023-11-18T20:43:18.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.604+0000 7efd8effd640 1 -- 172.21.15.60:0/1850393986 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd98004b80 con 0x7efda0150560 2023-11-18T20:43:18.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.604+0000 7efd8effd640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efd8006ae40 0x7efd8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.605+0000 7efd8effd640 1 -- 172.21.15.60:0/1850393986 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efd98088bf0 con 0x7efda0150560 2023-11-18T20:43:18.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.605+0000 7efd9dd9b640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efd8006ae40 0x7efd8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.605+0000 7efd9dd9b640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efd8006ae40 0x7efd8006d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd94038000 tx=0x7efd94008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:18.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.609+0000 7efd8effd640 1 -- 172.21.15.60:0/1850393986 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efda01e3040 con 0x7efda0150560 2023-11-18T20:43:18.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.786+0000 7efda4f41640 1 -- 172.21.15.60:0/1850393986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efda01e3040 con 0x7efda0150560 2023-11-18T20:43:18.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.787+0000 7efd8effd640 1 -- 172.21.15.60:0/1850393986 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16683 (secure 0 0 0) 0x7efda01e3040 con 0x7efda0150560 2023-11-18T20:43:18.789 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:18.789 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:18.223105+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:18.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 -- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efd8006ae40 msgr2=0x7efd8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efd8006ae40 0x7efd8006d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd94038000 tx=0x7efd94008030 comp rx=0 tx=0).stop 2023-11-18T20:43:18.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 -- 172.21.15.60:0/1850393986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150560 msgr2=0x7efda01e3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150560 0x7efda01e3cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efd9800c4d0 tx=0x7efd9800c990 comp rx=0 tx=0).stop 2023-11-18T20:43:18.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 -- 172.21.15.60:0/1850393986 shutdown_connections 2023-11-18T20:43:18.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efd8006ae40 0x7efd8006d300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda01454f0 0x7efda0145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015dec0 0x7efda015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 --2- 172.21.15.60:0/1850393986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0150560 0x7efda01e3cc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 -- 172.21.15.60:0/1850393986 >> 172.21.15.60:0/1850393986 conn(0x7efda006a4d0 msgr2=0x7efda0071020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:18.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 -- 172.21.15.60:0/1850393986 shutdown_connections 2023-11-18T20:43:18.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.789+0000 7efda4f41640 1 -- 172.21.15.60:0/1850393986 wait complete. 2023-11-18T20:43:18.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:18.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:18.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.909+0000 7effadd61640 1 Processor -- start 2023-11-18T20:43:18.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.909+0000 7effadd61640 1 -- start start 2023-11-18T20:43:18.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.910+0000 7effadd61640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effa806ea50 0x7effa806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.910+0000 7effadd61640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7effa8160fb0 0x7effa8161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.910+0000 7effadd61640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa8161d20 con 0x7effa80b3ff0 2023-11-18T20:43:18.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.910+0000 7effadd61640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa80d1ee0 con 0x7effa806ea50 2023-11-18T20:43:18.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.910+0000 7effadd61640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa806f370 con 0x7effa8160fb0 2023-11-18T20:43:18.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.910+0000 7effa77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effa806ea50 0x7effa806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.910+0000 7effa77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effa806ea50 0x7effa806ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49364/0 (socket says 172.21.15.60:49364) 2023-11-18T20:43:18.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.910+0000 7effa77fe640 1 -- 172.21.15.60:0/544190519 learned_addr learned my addr 172.21.15.60:0/544190519 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:18.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.910+0000 7effa6ffd640 1 --2- 172.21.15.60:0/544190519 >> v2:172.21.15.100:3300/0 conn(0x7effa8160fb0 0x7effa8161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.911+0000 7effa77fe640 1 -- 172.21.15.60:0/544190519 >> v2:172.21.15.100:3300/0 conn(0x7effa8160fb0 msgr2=0x7effa8161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:18.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.911+0000 7effa77fe640 1 --2- 172.21.15.60:0/544190519 >> v2:172.21.15.100:3300/0 conn(0x7effa8160fb0 0x7effa8161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.911+0000 7effa77fe640 1 -- 172.21.15.60:0/544190519 >> v1:172.21.15.100:6789/0 conn(0x7effa80b3ff0 legacy=0x7effa80b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.911+0000 7effa77fe640 1 -- 172.21.15.60:0/544190519 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa815e760 con 0x7effa806ea50 2023-11-18T20:43:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.911+0000 7effa77fe640 1 --2- 172.21.15.60:0/544190519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa806ea50 0x7effa806ee30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7effa80b5b30 tx=0x7eff9802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5099a401eb80cfa server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.911+0000 7effa67fc640 1 -- 172.21.15.60:0/544190519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98031870 con 0x7effa806ea50 2023-11-18T20:43:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.912+0000 7effa67fc640 1 -- 172.21.15.60:0/544190519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff980319d0 con 0x7effa806ea50 2023-11-18T20:43:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.912+0000 7effa67fc640 1 -- 172.21.15.60:0/544190519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98031d60 con 0x7effa806ea50 2023-11-18T20:43:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.912+0000 7effadd61640 1 -- 172.21.15.60:0/544190519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa806ea50 msgr2=0x7effa806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.912+0000 7effadd61640 1 --2- 172.21.15.60:0/544190519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa806ea50 0x7effa806ee30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7effa80b5b30 tx=0x7eff9802b910 comp rx=0 tx=0).stop 2023-11-18T20:43:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.912+0000 7effadd61640 1 -- 172.21.15.60:0/544190519 shutdown_connections 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.912+0000 7effadd61640 1 --2- 172.21.15.60:0/544190519 >> v2:172.21.15.100:3300/0 conn(0x7effa8160fb0 0x7effa8161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.912+0000 7effadd61640 1 --2- 172.21.15.60:0/544190519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa806ea50 0x7effa806ee30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.912+0000 7effadd61640 1 -- 172.21.15.60:0/544190519 >> 172.21.15.60:0/544190519 conn(0x7effa806a6b0 msgr2=0x7effa806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.912+0000 7effadd61640 1 -- 172.21.15.60:0/544190519 shutdown_connections 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.912+0000 7effadd61640 1 -- 172.21.15.60:0/544190519 wait complete. 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.913+0000 7effadd61640 1 Processor -- start 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.913+0000 7effadd61640 1 -- start start 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effadd61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effa806ea50 0x7effa81e8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effadd61640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa80b3ff0 0x7effa813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effadd61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8160fb0 0x7effa8158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effadd61640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa81e70f0 con 0x7effa8160fb0 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effadd61640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa81e73d0 con 0x7effa80b3ff0 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effadd61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa81e76b0 con 0x7effa806ea50 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effa7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8160fb0 0x7effa8158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effa7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8160fb0 0x7effa8158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49372/0 (socket says 172.21.15.60:49372) 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effa7fff640 1 -- 172.21.15.60:0/4151658653 learned_addr learned my addr 172.21.15.60:0/4151658653 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effa77fe640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effa806ea50 0x7effa81e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.914+0000 7effa6ffd640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa80b3ff0 0x7effa813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7effa7fff640 1 -- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effa806ea50 msgr2=0x7effa81e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7effa7fff640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effa806ea50 0x7effa81e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7effa7fff640 1 -- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa80b3ff0 msgr2=0x7effa813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7effa7fff640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa80b3ff0 0x7effa813f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:18.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7effa7fff640 1 -- 172.21.15.60:0/4151658653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980094d0 con 0x7effa8160fb0 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7effa7fff640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8160fb0 0x7effa8158a60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7effa000c730 tx=0x7effa000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7eff8ffff640 1 -- 172.21.15.60:0/4151658653 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0003b70 con 0x7effa8160fb0 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7eff8ffff640 1 -- 172.21.15.60:0/4151658653 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa0003cf0 con 0x7effa8160fb0 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7effadd61640 1 -- 172.21.15.60:0/4151658653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa81e7990 con 0x7effa8160fb0 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7eff8ffff640 1 -- 172.21.15.60:0/4151658653 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0003ea0 con 0x7effa8160fb0 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.915+0000 7effadd61640 1 -- 172.21.15.60:0/4151658653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa813f930 con 0x7effa8160fb0 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.916+0000 7effadd61640 1 -- 172.21.15.60:0/4151658653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa8069400 con 0x7effa8160fb0 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.917+0000 7eff8ffff640 1 -- 172.21.15.60:0/4151658653 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effa0005520 con 0x7effa8160fb0 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.917+0000 7eff8ffff640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7eff9406ae40 0x7eff9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.917+0000 7eff8ffff640 1 -- 172.21.15.60:0/4151658653 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7effa00894f0 con 0x7effa8160fb0 2023-11-18T20:43:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.917+0000 7effa77fe640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7eff9406ae40 0x7eff9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.918+0000 7effa77fe640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7eff9406ae40 0x7eff9406d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7eff98036000 tx=0x7eff9803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:18.921+0000 7eff8ffff640 1 -- 172.21.15.60:0/4151658653 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7effa00545d0 con 0x7effa8160fb0 2023-11-18T20:43:19.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.096+0000 7effadd61640 1 -- 172.21.15.60:0/4151658653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7effa8070d30 con 0x7effa8160fb0 2023-11-18T20:43:19.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.096+0000 7eff8ffff640 1 -- 172.21.15.60:0/4151658653 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7effa0057bf0 con 0x7effa8160fb0 2023-11-18T20:43:19.098 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 -- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7eff9406ae40 msgr2=0x7eff9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7eff9406ae40 0x7eff9406d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7eff98036000 tx=0x7eff9803c040 comp rx=0 tx=0).stop 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 -- 172.21.15.60:0/4151658653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8160fb0 msgr2=0x7effa8158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8160fb0 0x7effa8158a60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7effa000c730 tx=0x7effa000cbf0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 -- 172.21.15.60:0/4151658653 shutdown_connections 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7eff9406ae40 0x7eff9406d300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effa8160fb0 0x7effa8158a60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effa80b3ff0 0x7effa813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 --2- 172.21.15.60:0/4151658653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effa806ea50 0x7effa81e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 -- 172.21.15.60:0/4151658653 >> 172.21.15.60:0/4151658653 conn(0x7effa806a6b0 msgr2=0x7effa815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 -- 172.21.15.60:0/4151658653 shutdown_connections 2023-11-18T20:43:19.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.098+0000 7effadd61640 1 -- 172.21.15.60:0/4151658653 wait complete. 2023-11-18T20:43:19.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:19.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.218+0000 7f569d826640 1 Processor -- start 2023-11-18T20:43:19.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.218+0000 7f569d826640 1 -- start start 2023-11-18T20:43:19.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.219+0000 7f569d826640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f569815d330 0x7f56980b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.219+0000 7f569d826640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56980d2910 0x7f56980d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.219+0000 7f569d826640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56980b2b10 con 0x7f56980d3730 2023-11-18T20:43:19.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.219+0000 7f569d826640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56980d3b50 con 0x7f569815d330 2023-11-18T20:43:19.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.219+0000 7f569d826640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56980d1590 con 0x7f56980d2910 2023-11-18T20:43:19.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.220+0000 7f5696ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f569815d330 0x7f56980b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.220+0000 7f56967fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56980d2910 0x7f56980d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.220+0000 7f5696ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f569815d330 0x7f56980b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:60916/0 (socket says 172.21.15.60:60916) 2023-11-18T20:43:19.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.220+0000 7f5696ffd640 1 -- 172.21.15.60:0/1372293426 learned_addr learned my addr 172.21.15.60:0/1372293426 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:19.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.220+0000 7f56977fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f56980d3730 0x7f56980b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:40118/0 (socket says 172.21.15.60:40118) 2023-11-18T20:43:19.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.220+0000 7f5696ffd640 1 -- 172.21.15.60:0/1372293426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56980d2910 msgr2=0x7f56980d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:19.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.220+0000 7f5696ffd640 1 --2- 172.21.15.60:0/1372293426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56980d2910 0x7f56980d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.220+0000 7f5696ffd640 1 -- 172.21.15.60:0/1372293426 >> v1:172.21.15.100:6789/0 conn(0x7f56980d3730 legacy=0x7f56980b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.220+0000 7f5696ffd640 1 -- 172.21.15.60:0/1372293426 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56980d1870 con 0x7f569815d330 2023-11-18T20:43:19.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f5696ffd640 1 --2- 172.21.15.60:0/1372293426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f569815d330 0x7f56980b8b80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5688009e50 tx=0x7f568802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93992a5d0af638d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:19.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f5695ffb640 1 -- 172.21.15.60:0/1372293426 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568802ca70 con 0x7f569815d330 2023-11-18T20:43:19.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f5695ffb640 1 -- 172.21.15.60:0/1372293426 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568802cbd0 con 0x7f569815d330 2023-11-18T20:43:19.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f5695ffb640 1 -- 172.21.15.60:0/1372293426 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56880343f0 con 0x7f569815d330 2023-11-18T20:43:19.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f569d826640 1 -- 172.21.15.60:0/1372293426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f569815d330 msgr2=0x7f56980b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f569d826640 1 --2- 172.21.15.60:0/1372293426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f569815d330 0x7f56980b8b80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5688009e50 tx=0x7f568802bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f569d826640 1 -- 172.21.15.60:0/1372293426 shutdown_connections 2023-11-18T20:43:19.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f569d826640 1 --2- 172.21.15.60:0/1372293426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56980d2910 0x7f56980d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f569d826640 1 --2- 172.21.15.60:0/1372293426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f569815d330 0x7f56980b8b80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f569d826640 1 -- 172.21.15.60:0/1372293426 >> 172.21.15.60:0/1372293426 conn(0x7f5698072020 msgr2=0x7f5698072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:19.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.221+0000 7f569d826640 1 -- 172.21.15.60:0/1372293426 shutdown_connections 2023-11-18T20:43:19.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.222+0000 7f569d826640 1 -- 172.21.15.60:0/1372293426 wait complete. 2023-11-18T20:43:19.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.223+0000 7f569d826640 1 Processor -- start 2023-11-18T20:43:19.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.223+0000 7f569d826640 1 -- start start 2023-11-18T20:43:19.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f569d826640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56980d2910 0x7f5698157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f569d826640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56980d3730 0x7f569815f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f569d826640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f569815d330 0x7f56980b8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f569d826640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569815fac0 con 0x7f569815d330 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f569d826640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56980c7670 con 0x7f56980d2910 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f569d826640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5698158160 con 0x7f56980d3730 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f56977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f569815d330 0x7f56980b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f56977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f569815d330 0x7f56980b8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49380/0 (socket says 172.21.15.60:49380) 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f56977fe640 1 -- 172.21.15.60:0/3790789793 learned_addr learned my addr 172.21.15.60:0/3790789793 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f56967fc640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56980d3730 0x7f569815f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f5696ffd640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56980d2910 0x7f5698157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f56977fe640 1 -- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56980d3730 msgr2=0x7f569815f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f56977fe640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56980d3730 0x7f569815f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f56977fe640 1 -- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56980d2910 msgr2=0x7f5698157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f56977fe640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56980d2910 0x7f5698157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f56977fe640 1 -- 172.21.15.60:0/3790789793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5688009b30 con 0x7f569815d330 2023-11-18T20:43:19.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f56977fe640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f569815d330 0x7f56980b8760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f569000c400 tx=0x7f569000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:19.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f567ffff640 1 -- 172.21.15.60:0/3790789793 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56900074e0 con 0x7f569815d330 2023-11-18T20:43:19.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f567ffff640 1 -- 172.21.15.60:0/3790789793 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5690007640 con 0x7f569815d330 2023-11-18T20:43:19.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f567ffff640 1 -- 172.21.15.60:0/3790789793 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56900049e0 con 0x7f569815d330 2023-11-18T20:43:19.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.224+0000 7f569d826640 1 -- 172.21.15.60:0/3790789793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56980c6d40 con 0x7f569815d330 2023-11-18T20:43:19.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.225+0000 7f569d826640 1 -- 172.21.15.60:0/3790789793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56980c7300 con 0x7f569815d330 2023-11-18T20:43:19.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.226+0000 7f569d826640 1 -- 172.21.15.60:0/3790789793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56980c77e0 con 0x7f569815d330 2023-11-18T20:43:19.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.226+0000 7f567ffff640 1 -- 172.21.15.60:0/3790789793 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5690004b40 con 0x7f569815d330 2023-11-18T20:43:19.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.226+0000 7f567ffff640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f568406ae70 0x7f568406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.226+0000 7f567ffff640 1 -- 172.21.15.60:0/3790789793 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5690088f10 con 0x7f569815d330 2023-11-18T20:43:19.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.227+0000 7f5696ffd640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f568406ae70 0x7f568406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.227+0000 7f5696ffd640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f568406ae70 0x7f568406d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5688035000 tx=0x7f5688030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:19.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.231+0000 7f567ffff640 1 -- 172.21.15.60:0/3790789793 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5690053ff0 con 0x7f569815d330 2023-11-18T20:43:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.410+0000 7f569d826640 1 -- 172.21.15.60:0/3790789793 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56980c7490 con 0x7f569815d330 2023-11-18T20:43:19.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.410+0000 7f567ffff640 1 -- 172.21.15.60:0/3790789793 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5690057610 con 0x7f569815d330 2023-11-18T20:43:19.411 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 -- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f568406ae70 msgr2=0x7f568406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f568406ae70 0x7f568406d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5688035000 tx=0x7f5688030000 comp rx=0 tx=0).stop 2023-11-18T20:43:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 -- 172.21.15.60:0/3790789793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f569815d330 msgr2=0x7f56980b8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f569815d330 0x7f56980b8760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f569000c400 tx=0x7f569000c8c0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 -- 172.21.15.60:0/3790789793 shutdown_connections 2023-11-18T20:43:19.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f568406ae70 0x7f568406d330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f569815d330 0x7f56980b8760 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56980d3730 0x7f569815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 --2- 172.21.15.60:0/3790789793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56980d2910 0x7f5698157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 -- 172.21.15.60:0/3790789793 >> 172.21.15.60:0/3790789793 conn(0x7f5698072020 msgr2=0x7f5698072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:19.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 -- 172.21.15.60:0/3790789793 shutdown_connections 2023-11-18T20:43:19.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.412+0000 7f569d826640 1 -- 172.21.15.60:0/3790789793 wait complete. 2023-11-18T20:43:19.424 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:19.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.532+0000 7fbb2be46640 1 Processor -- start 2023-11-18T20:43:19.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.532+0000 7fbb2be46640 1 -- start start 2023-11-18T20:43:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.533+0000 7fbb2be46640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb2415d330 0x7fbb240b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.533+0000 7fbb2be46640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 0x7fbb240d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.533+0000 7fbb2be46640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb240b2b10 con 0x7fbb240d3730 2023-11-18T20:43:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.533+0000 7fbb2be46640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb240d3b50 con 0x7fbb2415d330 2023-11-18T20:43:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.533+0000 7fbb2be46640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240d1590 con 0x7fbb240d2910 2023-11-18T20:43:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.533+0000 7fbb29be2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb2415d330 0x7fbb240b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.533+0000 7fbb29be2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb2415d330 0x7fbb240b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49384/0 (socket says 172.21.15.60:49384) 2023-11-18T20:43:19.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.533+0000 7fbb29be2640 1 -- 172.21.15.60:0/499998068 learned_addr learned my addr 172.21.15.60:0/499998068 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.534+0000 7fbb293e1640 1 --2- 172.21.15.60:0/499998068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 0x7fbb240d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.534+0000 7fbb29be2640 1 -- 172.21.15.60:0/499998068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 msgr2=0x7fbb240d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.534+0000 7fbb29be2640 1 --2- 172.21.15.60:0/499998068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 0x7fbb240d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.534+0000 7fbb29be2640 1 -- 172.21.15.60:0/499998068 >> v1:172.21.15.100:6789/0 conn(0x7fbb240d3730 legacy=0x7fbb240b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.534+0000 7fbb29be2640 1 -- 172.21.15.60:0/499998068 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240d1870 con 0x7fbb2415d330 2023-11-18T20:43:19.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.534+0000 7fbb29be2640 1 --2- 172.21.15.60:0/499998068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 0x7fbb240b8b80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009e50 tx=0x7fbb1402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86374349936a64f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:19.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb28be0640 1 -- 172.21.15.60:0/499998068 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1402ca70 con 0x7fbb2415d330 2023-11-18T20:43:19.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb28be0640 1 -- 172.21.15.60:0/499998068 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1402cbd0 con 0x7fbb2415d330 2023-11-18T20:43:19.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb28be0640 1 -- 172.21.15.60:0/499998068 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb140343f0 con 0x7fbb2415d330 2023-11-18T20:43:19.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb2be46640 1 -- 172.21.15.60:0/499998068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 msgr2=0x7fbb240b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb2be46640 1 --2- 172.21.15.60:0/499998068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 0x7fbb240b8b80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009e50 tx=0x7fbb1402bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb2be46640 1 -- 172.21.15.60:0/499998068 shutdown_connections 2023-11-18T20:43:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb2be46640 1 --2- 172.21.15.60:0/499998068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 0x7fbb240d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb2be46640 1 --2- 172.21.15.60:0/499998068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 0x7fbb240b8b80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb2be46640 1 -- 172.21.15.60:0/499998068 >> 172.21.15.60:0/499998068 conn(0x7fbb24072020 msgr2=0x7fbb24072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb2be46640 1 -- 172.21.15.60:0/499998068 shutdown_connections 2023-11-18T20:43:19.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.535+0000 7fbb2be46640 1 -- 172.21.15.60:0/499998068 wait complete. 2023-11-18T20:43:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.536+0000 7fbb2be46640 1 Processor -- start 2023-11-18T20:43:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.537+0000 7fbb2be46640 1 -- start start 2023-11-18T20:43:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.537+0000 7fbb2be46640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 0x7fbb241f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.537+0000 7fbb2be46640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb240d3730 0x7fbb241f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.537+0000 7fbb2be46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 0x7fbb2415b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.537+0000 7fbb2be46640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb24153fb0 con 0x7fbb2415d330 2023-11-18T20:43:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.537+0000 7fbb2be46640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb241576c0 con 0x7fbb240d3730 2023-11-18T20:43:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.537+0000 7fbb2be46640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb241584e0 con 0x7fbb240d2910 2023-11-18T20:43:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.537+0000 7fbb2a3e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 0x7fbb2415b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.537+0000 7fbb2a3e3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 0x7fbb2415b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49390/0 (socket says 172.21.15.60:49390) 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.537+0000 7fbb2a3e3640 1 -- 172.21.15.60:0/2615997 learned_addr learned my addr 172.21.15.60:0/2615997 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb293e1640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb240d3730 0x7fbb241f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb29be2640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 0x7fbb241f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb2a3e3640 1 -- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 msgr2=0x7fbb241f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb2a3e3640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 0x7fbb241f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb2a3e3640 1 -- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb240d3730 msgr2=0x7fbb241f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb2a3e3640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb240d3730 0x7fbb241f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb2a3e3640 1 -- 172.21.15.60:0/2615997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb14009b30 con 0x7fbb2415d330 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb29be2640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 0x7fbb241f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb2a3e3640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 0x7fbb2415b1e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00c390 tx=0x7fbb1c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb227fc640 1 -- 172.21.15.60:0/2615997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0052d0 con 0x7fbb2415d330 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb227fc640 1 -- 172.21.15.60:0/2615997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c005430 con 0x7fbb2415d330 2023-11-18T20:43:19.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb2be46640 1 -- 172.21.15.60:0/2615997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb241f5ea0 con 0x7fbb2415d330 2023-11-18T20:43:19.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb227fc640 1 -- 172.21.15.60:0/2615997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c003810 con 0x7fbb2415d330 2023-11-18T20:43:19.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.538+0000 7fbb2be46640 1 -- 172.21.15.60:0/2615997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb24157bb0 con 0x7fbb2415d330 2023-11-18T20:43:19.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.539+0000 7fbb2be46640 1 -- 172.21.15.60:0/2615997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb24069400 con 0x7fbb2415d330 2023-11-18T20:43:19.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.539+0000 7fbb227fc640 1 -- 172.21.15.60:0/2615997 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb1c003970 con 0x7fbb2415d330 2023-11-18T20:43:19.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.540+0000 7fbb227fc640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbb1006ae40 0x7fbb1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.540+0000 7fbb227fc640 1 -- 172.21.15.60:0/2615997 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fbb1c088ac0 con 0x7fbb2415d330 2023-11-18T20:43:19.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.540+0000 7fbb29be2640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbb1006ae40 0x7fbb1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.541+0000 7fbb29be2640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbb1006ae40 0x7fbb1006d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb14035000 tx=0x7fbb14030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:19.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.544+0000 7fbb227fc640 1 -- 172.21.15.60:0/2615997 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb1c053b80 con 0x7fbb2415d330 2023-11-18T20:43:19.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.720+0000 7fbb2be46640 1 -- 172.21.15.60:0/2615997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb2406a4d0 con 0x7fbb2415d330 2023-11-18T20:43:19.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.720+0000 7fbb227fc640 1 -- 172.21.15.60:0/2615997 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb1c088e60 con 0x7fbb2415d330 2023-11-18T20:43:19.721 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 -- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbb1006ae40 msgr2=0x7fbb1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbb1006ae40 0x7fbb1006d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb14035000 tx=0x7fbb14030000 comp rx=0 tx=0).stop 2023-11-18T20:43:19.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 -- 172.21.15.60:0/2615997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 msgr2=0x7fbb2415b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 0x7fbb2415b1e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00c390 tx=0x7fbb1c00c850 comp rx=0 tx=0).stop 2023-11-18T20:43:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 -- 172.21.15.60:0/2615997 shutdown_connections 2023-11-18T20:43:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbb1006ae40 0x7fbb1006d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb2415d330 0x7fbb2415b1e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb240d3730 0x7fbb241f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 --2- 172.21.15.60:0/2615997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb240d2910 0x7fbb241f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 -- 172.21.15.60:0/2615997 >> 172.21.15.60:0/2615997 conn(0x7fbb24072020 msgr2=0x7fbb240b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 -- 172.21.15.60:0/2615997 shutdown_connections 2023-11-18T20:43:19.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.722+0000 7fbb2be46640 1 -- 172.21.15.60:0/2615997 wait complete. 2023-11-18T20:43:19.734 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:19.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.841+0000 7fd152e7c640 1 Processor -- start 2023-11-18T20:43:19.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.841+0000 7fd152e7c640 1 -- start start 2023-11-18T20:43:19.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd152e7c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd14c0ceb70 0x7fd14c0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd152e7c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c071770 con 0x7fd14c0b4370 2023-11-18T20:43:19.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd152e7c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0ccdc0 con 0x7fd14c144980 2023-11-18T20:43:19.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd152e7c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c15ced0 con 0x7fd14c0ceb70 2023-11-18T20:43:19.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd14bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd14c0ceb70 0x7fd14c0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd151419640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd14c0b4370 0x7fd14c071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41374/0 (socket says 172.21.15.60:41374) 2023-11-18T20:43:19.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd151419640 1 -- 172.21.15.60:0/92812456 learned_addr learned my addr 172.21.15.60:0/92812456 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:19.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd14bfff640 1 -- 172.21.15.60:0/92812456 >> v1:172.21.15.100:6789/0 conn(0x7fd14c144980 legacy=0x7fd14c15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:19.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd14bfff640 1 -- 172.21.15.60:0/92812456 >> v1:172.21.15.60:6789/0 conn(0x7fd14c0b4370 legacy=0x7fd14c071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd14b7fe640 1 -- 172.21.15.60:0/92812456 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675217620 0 0) 0x7fd14c071770 con 0x7fd14c0b4370 2023-11-18T20:43:19.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.842+0000 7fd14bfff640 1 -- 172.21.15.60:0/92812456 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c0cd0a0 con 0x7fd14c0ceb70 2023-11-18T20:43:19.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.843+0000 7fd14bfff640 1 --2- 172.21.15.60:0/92812456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0ceb70 0x7fd14c0cc450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd1380097a0 tx=0x7fd13802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a0119d5d4a20eb server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:19.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.843+0000 7fd14b7fe640 1 -- 172.21.15.60:0/92812456 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138031830 con 0x7fd14c0ceb70 2023-11-18T20:43:19.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.843+0000 7fd14b7fe640 1 -- 172.21.15.60:0/92812456 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd138031990 con 0x7fd14c0ceb70 2023-11-18T20:43:19.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.843+0000 7fd14b7fe640 1 -- 172.21.15.60:0/92812456 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138031d20 con 0x7fd14c0ceb70 2023-11-18T20:43:19.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.844+0000 7fd152e7c640 1 -- 172.21.15.60:0/92812456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0ceb70 msgr2=0x7fd14c0cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.844+0000 7fd152e7c640 1 --2- 172.21.15.60:0/92812456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0ceb70 0x7fd14c0cc450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd1380097a0 tx=0x7fd13802b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.844+0000 7fd152e7c640 1 -- 172.21.15.60:0/92812456 shutdown_connections 2023-11-18T20:43:19.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.844+0000 7fd152e7c640 1 --2- 172.21.15.60:0/92812456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0ceb70 0x7fd14c0cc450 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.844+0000 7fd152e7c640 1 -- 172.21.15.60:0/92812456 >> 172.21.15.60:0/92812456 conn(0x7fd14c06a4d0 msgr2=0x7fd14c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:19.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.844+0000 7fd152e7c640 1 -- 172.21.15.60:0/92812456 shutdown_connections 2023-11-18T20:43:19.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.844+0000 7fd152e7c640 1 -- 172.21.15.60:0/92812456 wait complete. 2023-11-18T20:43:19.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.845+0000 7fd152e7c640 1 Processor -- start 2023-11-18T20:43:19.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.845+0000 7fd152e7c640 1 -- start start 2023-11-18T20:43:19.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd152e7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0b4370 0x7fd14c13b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd152e7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd14c0ceb70 0x7fd14c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd152e7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c144980 0x7fd14c13c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd152e7c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b37f0 con 0x7fd14c0b4370 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd152e7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b2be0 con 0x7fd14c144980 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd152e7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b2ec0 con 0x7fd14c0ceb70 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd150c18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0b4370 0x7fd14c13b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd150c18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0b4370 0x7fd14c13b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49416/0 (socket says 172.21.15.60:49416) 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd150c18640 1 -- 172.21.15.60:0/972565575 learned_addr learned my addr 172.21.15.60:0/972565575 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd151419640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c144980 0x7fd14c13c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd14bfff640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd14c0ceb70 0x7fd14c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd150c18640 1 -- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd14c0ceb70 msgr2=0x7fd14c0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd150c18640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd14c0ceb70 0x7fd14c0b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd150c18640 1 -- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c144980 msgr2=0x7fd14c13c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd150c18640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c144980 0x7fd14c13c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd150c18640 1 -- 172.21.15.60:0/972565575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd138009400 con 0x7fd14c0b4370 2023-11-18T20:43:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd151419640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c144980 0x7fd14c13c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.846+0000 7fd150c18640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0b4370 0x7fd14c13b770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd14000b3f0 tx=0x7fd14000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.847+0000 7fd1497fa640 1 -- 172.21.15.60:0/972565575 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140011830 con 0x7fd14c0b4370 2023-11-18T20:43:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.847+0000 7fd1497fa640 1 -- 172.21.15.60:0/972565575 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd140011990 con 0x7fd14c0b4370 2023-11-18T20:43:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.847+0000 7fd152e7c640 1 -- 172.21.15.60:0/972565575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c0b31a0 con 0x7fd14c0b4370 2023-11-18T20:43:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.847+0000 7fd1497fa640 1 -- 172.21.15.60:0/972565575 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140011d40 con 0x7fd14c0b4370 2023-11-18T20:43:19.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.847+0000 7fd152e7c640 1 -- 172.21.15.60:0/972565575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c13bcb0 con 0x7fd14c0b4370 2023-11-18T20:43:19.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.848+0000 7fd152e7c640 1 -- 172.21.15.60:0/972565575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c0b3960 con 0x7fd14c0b4370 2023-11-18T20:43:19.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.848+0000 7fd1497fa640 1 -- 172.21.15.60:0/972565575 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd14001a400 con 0x7fd14c0b4370 2023-11-18T20:43:19.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.849+0000 7fd1497fa640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd13006ae70 0x7fd13006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:19.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.849+0000 7fd1497fa640 1 -- 172.21.15.60:0/972565575 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd140089830 con 0x7fd14c0b4370 2023-11-18T20:43:19.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.849+0000 7fd14bfff640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd13006ae70 0x7fd13006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:19.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.849+0000 7fd14bfff640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd13006ae70 0x7fd13006d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd138036000 tx=0x7fd13803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:19.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:19.853+0000 7fd1497fa640 1 -- 172.21.15.60:0/972565575 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1400547d0 con 0x7fd14c0b4370 2023-11-18T20:43:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.042+0000 7fd152e7c640 1 -- 172.21.15.60:0/972565575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd14c069400 con 0x7fd14c0b4370 2023-11-18T20:43:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.042+0000 7fd1497fa640 1 -- 172.21.15.60:0/972565575 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd140015070 con 0x7fd14c0b4370 2023-11-18T20:43:20.043 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 -- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd13006ae70 msgr2=0x7fd13006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd13006ae70 0x7fd13006d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd138036000 tx=0x7fd13803c040 comp rx=0 tx=0).stop 2023-11-18T20:43:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 -- 172.21.15.60:0/972565575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0b4370 msgr2=0x7fd14c13b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0b4370 0x7fd14c13b770 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd14000b3f0 tx=0x7fd14000b8b0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 -- 172.21.15.60:0/972565575 shutdown_connections 2023-11-18T20:43:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd13006ae70 0x7fd13006d330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c144980 0x7fd14c13c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd14c0ceb70 0x7fd14c0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 --2- 172.21.15.60:0/972565575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd14c0b4370 0x7fd14c13b770 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 -- 172.21.15.60:0/972565575 >> 172.21.15.60:0/972565575 conn(0x7fd14c06a4d0 msgr2=0x7fd14c070290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 -- 172.21.15.60:0/972565575 shutdown_connections 2023-11-18T20:43:20.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.044+0000 7fd152e7c640 1 -- 172.21.15.60:0/972565575 wait complete. 2023-11-18T20:43:20.056 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi100.front.sepia.ceph.com... 2023-11-18T20:43:20.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:20.056 DEBUG:teuthology.orchestra.run.smithi100:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-18T20:43:20.088 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:43:20.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:43:20.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:20.089 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-18T20:43:20.147 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-18T20:43:20.147 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:43:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.254+0000 7fe19b561640 1 Processor -- start 2023-11-18T20:43:20.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.254+0000 7fe19b561640 1 -- start start 2023-11-18T20:43:20.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe19b561640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe194143490 0x7fe1940b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe19b561640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 0x7fe1940b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe19b561640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19415e2a0 con 0x7fe19415a4e0 2023-11-18T20:43:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe19b561640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19415b520 con 0x7fe194143490 2023-11-18T20:43:20.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe19b561640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe194143870 con 0x7fe19415e730 2023-11-18T20:43:20.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe1992fd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe194143490 0x7fe1940b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe199afe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe19415a4e0 0x7fe19415a970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41386/0 (socket says 172.21.15.60:41386) 2023-11-18T20:43:20.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe199afe640 1 -- 172.21.15.60:0/577917768 learned_addr learned my addr 172.21.15.60:0/577917768 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:20.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe1992fd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe194143490 0x7fe1940b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49432/0 (socket says 172.21.15.60:49432) 2023-11-18T20:43:20.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe198afc640 1 --2- 172.21.15.60:0/577917768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 0x7fe1940b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe1992fd640 1 -- 172.21.15.60:0/577917768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 msgr2=0x7fe1940b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe1992fd640 1 --2- 172.21.15.60:0/577917768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 0x7fe1940b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe1992fd640 1 -- 172.21.15.60:0/577917768 >> v1:172.21.15.60:6789/0 conn(0x7fe19415a4e0 legacy=0x7fe19415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe1992fd640 1 -- 172.21.15.60:0/577917768 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19415b240 con 0x7fe194143490 2023-11-18T20:43:20.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.255+0000 7fe18bfff640 1 -- 172.21.15.60:0/577917768 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236598741 0 0) 0x7fe19415e2a0 con 0x7fe19415a4e0 2023-11-18T20:43:20.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.256+0000 7fe1992fd640 1 --2- 172.21.15.60:0/577917768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 0x7fe1940b0cf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe1900097a0 tx=0x7fe19002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0979edca227698 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:20.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.256+0000 7fe18bfff640 1 -- 172.21.15.60:0/577917768 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190031830 con 0x7fe194143490 2023-11-18T20:43:20.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.256+0000 7fe18bfff640 1 -- 172.21.15.60:0/577917768 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe190031990 con 0x7fe194143490 2023-11-18T20:43:20.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.256+0000 7fe18bfff640 1 -- 172.21.15.60:0/577917768 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190031d20 con 0x7fe194143490 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.257+0000 7fe19b561640 1 -- 172.21.15.60:0/577917768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 msgr2=0x7fe1940b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.257+0000 7fe19b561640 1 --2- 172.21.15.60:0/577917768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 0x7fe1940b0cf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe1900097a0 tx=0x7fe19002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.257+0000 7fe19b561640 1 -- 172.21.15.60:0/577917768 shutdown_connections 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.257+0000 7fe19b561640 1 --2- 172.21.15.60:0/577917768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 0x7fe1940b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.257+0000 7fe19b561640 1 --2- 172.21.15.60:0/577917768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 0x7fe1940b0cf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.257+0000 7fe19b561640 1 -- 172.21.15.60:0/577917768 >> 172.21.15.60:0/577917768 conn(0x7fe19406a4d0 msgr2=0x7fe19406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.257+0000 7fe19b561640 1 -- 172.21.15.60:0/577917768 shutdown_connections 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.257+0000 7fe19b561640 1 -- 172.21.15.60:0/577917768 wait complete. 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.258+0000 7fe19b561640 1 Processor -- start 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.258+0000 7fe19b561640 1 -- start start 2023-11-18T20:43:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe19b561640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 0x7fe1941599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe19b561640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe19415a4e0 0x7fe1941e71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe19b561640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 0x7fe1941e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe19b561640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940ad2f0 con 0x7fe194143490 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe19b561640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe194159f10 con 0x7fe19415a4e0 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe19b561640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19415c7d0 con 0x7fe19415e730 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe1992fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 0x7fe1941599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe1992fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 0x7fe1941599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49446/0 (socket says 172.21.15.60:49446) 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe1992fd640 1 -- 172.21.15.60:0/4252481311 learned_addr learned my addr 172.21.15.60:0/4252481311 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe198afc640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe19415a4e0 0x7fe1941e71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe199afe640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 0x7fe1941e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe1992fd640 1 -- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 msgr2=0x7fe1941e51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe1992fd640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 0x7fe1941e51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe1992fd640 1 -- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe19415a4e0 msgr2=0x7fe1941e71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe1992fd640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe19415a4e0 0x7fe1941e71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe1992fd640 1 -- 172.21.15.60:0/4252481311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe190009400 con 0x7fe194143490 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe199afe640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 0x7fe1941e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe1992fd640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 0x7fe1941599d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe19002bee0 tx=0x7fe1900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.259+0000 7fe189ffb640 1 -- 172.21.15.60:0/4252481311 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19003ba70 con 0x7fe194143490 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.260+0000 7fe189ffb640 1 -- 172.21.15.60:0/4252481311 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19003bbf0 con 0x7fe194143490 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.260+0000 7fe19b561640 1 -- 172.21.15.60:0/4252481311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1940ad540 con 0x7fe194143490 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.260+0000 7fe189ffb640 1 -- 172.21.15.60:0/4252481311 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1900433f0 con 0x7fe194143490 2023-11-18T20:43:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.260+0000 7fe19b561640 1 -- 172.21.15.60:0/4252481311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1940adb00 con 0x7fe194143490 2023-11-18T20:43:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.261+0000 7fe19b561640 1 -- 172.21.15.60:0/4252481311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1940b4380 con 0x7fe194143490 2023-11-18T20:43:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.261+0000 7fe189ffb640 1 -- 172.21.15.60:0/4252481311 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe19004c3b0 con 0x7fe194143490 2023-11-18T20:43:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.262+0000 7fe189ffb640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe17806ae40 0x7fe17806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.262+0000 7fe189ffb640 1 -- 172.21.15.60:0/4252481311 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe1900ab5b0 con 0x7fe194143490 2023-11-18T20:43:20.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.262+0000 7fe198afc640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe17806ae40 0x7fe17806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.262+0000 7fe198afc640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe17806ae40 0x7fe17806d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe184006fd0 tx=0x7fe184008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:20.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.266+0000 7fe189ffb640 1 -- 172.21.15.60:0/4252481311 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe190076790 con 0x7fe194143490 2023-11-18T20:43:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.479+0000 7fe19b561640 1 -- 172.21.15.60:0/4252481311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1940b1e50 con 0x7fe194143490 2023-11-18T20:43:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.479+0000 7fe189ffb640 1 -- 172.21.15.60:0/4252481311 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fe190079db0 con 0x7fe194143490 2023-11-18T20:43:20.481 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:20.481 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1530496056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1530496056},{"type":"v1","addr":"172.21.15.60:6833","nonce":1530496056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3842389676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3842389676},{"type":"v1","addr":"172.21.15.60:6835","nonce":3842389676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4248907695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4248907695},{"type":"v1","addr":"172.21.15.100:6836","nonce":4248907695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:42:37.459556+0000","modified":"2023-11-18T20:42:39.458016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1851816776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1851816776},{"type":"v1","addr":"172.21.15.100:6837","nonce":1851816776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-18T20:43:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2023-11-18T20:43:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.481+0000 7fe19b561640 1 -- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe17806ae40 msgr2=0x7fe17806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.481+0000 7fe19b561640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe17806ae40 0x7fe17806d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe184006fd0 tx=0x7fe184008040 comp rx=0 tx=0).stop 2023-11-18T20:43:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.481+0000 7fe19b561640 1 -- 172.21.15.60:0/4252481311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 msgr2=0x7fe1941599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.481+0000 7fe19b561640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 0x7fe1941599d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe19002bee0 tx=0x7fe1900046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.481+0000 7fe19b561640 1 -- 172.21.15.60:0/4252481311 shutdown_connections 2023-11-18T20:43:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.481+0000 7fe19b561640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe17806ae40 0x7fe17806d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.481+0000 7fe19b561640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19415e730 0x7fe1941e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.483 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.481+0000 7fe19b561640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe19415a4e0 0x7fe1941e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.481+0000 7fe19b561640 1 --2- 172.21.15.60:0/4252481311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe194143490 0x7fe1941599d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.481+0000 7fe19b561640 1 -- 172.21.15.60:0/4252481311 >> 172.21.15.60:0/4252481311 conn(0x7fe19406a4d0 msgr2=0x7fe1940b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.482+0000 7fe19b561640 1 -- 172.21.15.60:0/4252481311 shutdown_connections 2023-11-18T20:43:20.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.482+0000 7fe19b561640 1 -- 172.21.15.60:0/4252481311 wait complete. 2023-11-18T20:43:20.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:20.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.599+0000 7faa90790640 1 Processor -- start 2023-11-18T20:43:20.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.599+0000 7faa90790640 1 -- start start 2023-11-18T20:43:20.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.600+0000 7faa90790640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa88143490 0x7faa880b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.600+0000 7faa90790640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa8815abf0 0x7faa880b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.600+0000 7faa90790640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8815e2a0 con 0x7faa8815e730 2023-11-18T20:43:20.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.600+0000 7faa90790640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8815b520 con 0x7faa8815abf0 2023-11-18T20:43:20.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.600+0000 7faa90790640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa88143870 con 0x7faa88143490 2023-11-18T20:43:20.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.600+0000 7faa8e52c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa88143490 0x7faa880b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.600+0000 7faa8ed2d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa8815abf0 0x7faa880b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.600+0000 7faa8e52c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa88143490 0x7faa880b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:45654/0 (socket says 172.21.15.60:45654) 2023-11-18T20:43:20.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.600+0000 7faa8e52c640 1 -- 172.21.15.60:0/1173819766 learned_addr learned my addr 172.21.15.60:0/1173819766 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:20.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.601+0000 7faa8ed2d640 1 -- 172.21.15.60:0/1173819766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa88143490 msgr2=0x7faa880b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:20.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.601+0000 7faa8ed2d640 1 --2- 172.21.15.60:0/1173819766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa88143490 0x7faa880b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.601+0000 7faa8ed2d640 1 -- 172.21.15.60:0/1173819766 >> v1:172.21.15.100:6789/0 conn(0x7faa8815e730 legacy=0x7faa8815a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.601+0000 7faa8ed2d640 1 -- 172.21.15.60:0/1173819766 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8815b240 con 0x7faa8815abf0 2023-11-18T20:43:20.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.601+0000 7faa8d52a640 1 -- 172.21.15.60:0/1173819766 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058763380 0 0) 0x7faa8815e2a0 con 0x7faa8815e730 2023-11-18T20:43:20.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.601+0000 7faa8e52c640 1 --2- 172.21.15.60:0/1173819766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa88143490 0x7faa880b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:20.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.601+0000 7faa8ed2d640 1 --2- 172.21.15.60:0/1173819766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8815abf0 0x7faa880b1230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa80009d90 tx=0x7faa8002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbcfb2055225b80d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:20.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa8d52a640 1 -- 172.21.15.60:0/1173819766 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8002cc50 con 0x7faa8815abf0 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa8d52a640 1 -- 172.21.15.60:0/1173819766 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8002cdb0 con 0x7faa8815abf0 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa8d52a640 1 -- 172.21.15.60:0/1173819766 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80034660 con 0x7faa8815abf0 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa90790640 1 -- 172.21.15.60:0/1173819766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8815abf0 msgr2=0x7faa880b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa90790640 1 --2- 172.21.15.60:0/1173819766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8815abf0 0x7faa880b1230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faa80009d90 tx=0x7faa8002bea0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa90790640 1 -- 172.21.15.60:0/1173819766 shutdown_connections 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa90790640 1 --2- 172.21.15.60:0/1173819766 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8815abf0 0x7faa880b1230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa90790640 1 --2- 172.21.15.60:0/1173819766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa88143490 0x7faa880b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa90790640 1 -- 172.21.15.60:0/1173819766 >> 172.21.15.60:0/1173819766 conn(0x7faa8806a4d0 msgr2=0x7faa8806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa90790640 1 -- 172.21.15.60:0/1173819766 shutdown_connections 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.602+0000 7faa90790640 1 -- 172.21.15.60:0/1173819766 wait complete. 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.603+0000 7faa90790640 1 Processor -- start 2023-11-18T20:43:20.605 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.604+0000 7faa90790640 1 -- start start 2023-11-18T20:43:20.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.604+0000 7faa90790640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa88143490 0x7faa8815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.604+0000 7faa90790640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8815abf0 0x7faa8813ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.604+0000 7faa90790640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8815e730 0x7faa880b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.604+0000 7faa90790640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa881f3030 con 0x7faa8815abf0 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.604+0000 7faa90790640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8813d0a0 con 0x7faa88143490 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.604+0000 7faa90790640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8815beb0 con 0x7faa8815e730 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.604+0000 7faa8dd2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8815abf0 0x7faa8813ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.604+0000 7faa8dd2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8815abf0 0x7faa8813ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49776/0 (socket says 172.21.15.60:49776) 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.604+0000 7faa8dd2b640 1 -- 172.21.15.60:0/2048047815 learned_addr learned my addr 172.21.15.60:0/2048047815 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa8e52c640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa88143490 0x7faa8815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa8ed2d640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8815e730 0x7faa880b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa8dd2b640 1 -- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8815e730 msgr2=0x7faa880b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa8dd2b640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8815e730 0x7faa880b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa8dd2b640 1 -- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa88143490 msgr2=0x7faa8815b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa8dd2b640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa88143490 0x7faa8815b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa8dd2b640 1 -- 172.21.15.60:0/2048047815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa840094d0 con 0x7faa8815abf0 2023-11-18T20:43:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa8e52c640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa88143490 0x7faa8815b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa8dd2b640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8815abf0 0x7faa8813ac70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c410 tx=0x7faa7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa7effd640 1 -- 172.21.15.60:0/2048047815 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa780035f0 con 0x7faa8815abf0 2023-11-18T20:43:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa7effd640 1 -- 172.21.15.60:0/2048047815 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa78003770 con 0x7faa8815abf0 2023-11-18T20:43:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa7effd640 1 -- 172.21.15.60:0/2048047815 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78003b20 con 0x7faa8815abf0 2023-11-18T20:43:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa90790640 1 -- 172.21.15.60:0/2048047815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa80009a70 con 0x7faa8815abf0 2023-11-18T20:43:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.605+0000 7faa90790640 1 -- 172.21.15.60:0/2048047815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8815c470 con 0x7faa8815abf0 2023-11-18T20:43:20.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.606+0000 7faa90790640 1 -- 172.21.15.60:0/2048047815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa88069400 con 0x7faa8815abf0 2023-11-18T20:43:20.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.607+0000 7faa7effd640 1 -- 172.21.15.60:0/2048047815 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa78003c80 con 0x7faa8815abf0 2023-11-18T20:43:20.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.607+0000 7faa7effd640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa6c06ae40 0x7faa6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.607+0000 7faa7effd640 1 -- 172.21.15.60:0/2048047815 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7faa78088cb0 con 0x7faa8815abf0 2023-11-18T20:43:20.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.607+0000 7faa8e52c640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa6c06ae40 0x7faa6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.608+0000 7faa8e52c640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa6c06ae40 0x7faa6c06d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faa84009db0 tx=0x7faa84008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:20.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.612+0000 7faa7effd640 1 -- 172.21.15.60:0/2048047815 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa78053ce0 con 0x7faa8815abf0 2023-11-18T20:43:20.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.789+0000 7faa90790640 1 -- 172.21.15.60:0/2048047815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa880cbbe0 con 0x7faa8815abf0 2023-11-18T20:43:20.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.790+0000 7faa7effd640 1 -- 172.21.15.60:0/2048047815 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16683 (secure 0 0 0) 0x7faa78017020 con 0x7faa8815abf0 2023-11-18T20:43:20.792 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:20.792 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:19.231773+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:20.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 -- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa6c06ae40 msgr2=0x7faa6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa6c06ae40 0x7faa6c06d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faa84009db0 tx=0x7faa84008040 comp rx=0 tx=0).stop 2023-11-18T20:43:20.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 -- 172.21.15.60:0/2048047815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8815abf0 msgr2=0x7faa8813ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8815abf0 0x7faa8813ac70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c410 tx=0x7faa7800c8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 -- 172.21.15.60:0/2048047815 shutdown_connections 2023-11-18T20:43:20.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa6c06ae40 0x7faa6c06d300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8815e730 0x7faa880b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8815abf0 0x7faa8813ac70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 --2- 172.21.15.60:0/2048047815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa88143490 0x7faa8815b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 -- 172.21.15.60:0/2048047815 >> 172.21.15.60:0/2048047815 conn(0x7faa8806a4d0 msgr2=0x7faa880b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:20.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 -- 172.21.15.60:0/2048047815 shutdown_connections 2023-11-18T20:43:20.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.792+0000 7faa90790640 1 -- 172.21.15.60:0/2048047815 wait complete. 2023-11-18T20:43:20.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:42:35.573014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:42:36.443031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:20.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:20.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.915+0000 7fce534ec640 1 Processor -- start 2023-11-18T20:43:20.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.915+0000 7fce534ec640 1 -- start start 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce534ec640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fce4c143490 0x7fce4c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce534ec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4c15e730 0x7fce4c0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce534ec640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c15e2a0 con 0x7fce4c15a4e0 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce534ec640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c15b520 con 0x7fce4c143490 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce534ec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c143870 con 0x7fce4c15e730 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce51a89640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fce4c15a4e0 0x7fce4c15a970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45314/0 (socket says 172.21.15.60:45314) 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce51a89640 1 -- 172.21.15.60:0/1284827592 learned_addr learned my addr 172.21.15.60:0/1284827592 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce50a87640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4c15e730 0x7fce4c0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce51288640 1 --2- 172.21.15.60:0/1284827592 >> v2:172.21.15.100:3300/0 conn(0x7fce4c143490 0x7fce4c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766256792 0 0) 0x7fce4c15e2a0 con 0x7fce4c15a4e0 2023-11-18T20:43:20.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce38005730 con 0x7fce4c15a4e0 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 511270321 0 0) 0x7fce38005730 con 0x7fce4c15a4e0 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4c15e730 msgr2=0x7fce4c0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4c15e730 0x7fce4c0b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 >> v2:172.21.15.100:3300/0 conn(0x7fce4c143490 msgr2=0x7fce4c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 --2- 172.21.15.60:0/1284827592 >> v2:172.21.15.100:3300/0 conn(0x7fce4c143490 0x7fce4c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c15b240 con 0x7fce4c15a4e0 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fce440019f0 con 0x7fce4c15a4e0 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 >> v1:172.21.15.60:6789/0 conn(0x7fce4c15a4e0 legacy=0x7fce4c15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce38005b80 0x7fce38005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce380069d0 0x7fce38006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4c15e730 0x7fce38007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c15b520 con 0x7fce380069d0 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c143870 con 0x7fce38005b80 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3800db90 con 0x7fce4c15e730 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.916+0000 7fce50a87640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce380069d0 0x7fce38006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.917+0000 7fce51288640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce38005b80 0x7fce38005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.917+0000 7fce51a89640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4c15e730 0x7fce38007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.917+0000 7fce50a87640 1 -- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4c15e730 msgr2=0x7fce38007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.917+0000 7fce50a87640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce4c15e730 0x7fce38007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.917+0000 7fce50a87640 1 -- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce38005b80 msgr2=0x7fce38005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.917+0000 7fce50a87640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce38005b80 0x7fce38005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.917+0000 7fce50a87640 1 -- 172.21.15.60:0/1284827592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38005730 con 0x7fce380069d0 2023-11-18T20:43:20.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.917+0000 7fce50a87640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce380069d0 0x7fce38006e50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fce48009ac0 tx=0x7fce38007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638a73b69f3d849a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:20.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.918+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4802c6b0 con 0x7fce380069d0 2023-11-18T20:43:20.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.918+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4802c810 con 0x7fce380069d0 2023-11-18T20:43:20.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.918+0000 7fce3ffff640 1 -- 172.21.15.60:0/1284827592 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4802cba0 con 0x7fce380069d0 2023-11-18T20:43:20.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.918+0000 7fce534ec640 1 -- 172.21.15.60:0/1284827592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce380069d0 msgr2=0x7fce38006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.918+0000 7fce534ec640 1 --2- 172.21.15.60:0/1284827592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce380069d0 0x7fce38006e50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fce48009ac0 tx=0x7fce38007a00 comp rx=0 tx=0).stop 2023-11-18T20:43:20.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.918+0000 7fce51a89640 1 -- 172.21.15.60:0/1284827592 reap_dead start 2023-11-18T20:43:20.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.918+0000 7fce534ec640 1 -- 172.21.15.60:0/1284827592 shutdown_connections 2023-11-18T20:43:20.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.918+0000 7fce534ec640 1 -- 172.21.15.60:0/1284827592 >> 172.21.15.60:0/1284827592 conn(0x7fce4c06a4d0 msgr2=0x7fce4c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:20.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.918+0000 7fce534ec640 1 -- 172.21.15.60:0/1284827592 shutdown_connections 2023-11-18T20:43:20.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.918+0000 7fce534ec640 1 -- 172.21.15.60:0/1284827592 wait complete. 2023-11-18T20:43:20.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.919+0000 7fce534ec640 1 Processor -- start 2023-11-18T20:43:20.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.920+0000 7fce534ec640 1 -- start start 2023-11-18T20:43:20.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.920+0000 7fce534ec640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce380069d0 0x7fce4c13f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.920+0000 7fce534ec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce4c13be40 0x7fce4c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.920+0000 7fce534ec640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4c13c2c0 0x7fce4c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.920+0000 7fce534ec640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c13fc10 con 0x7fce4c13c2c0 2023-11-18T20:43:20.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.920+0000 7fce534ec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c13fef0 con 0x7fce4c13be40 2023-11-18T20:43:20.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.920+0000 7fce534ec640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c1401d0 con 0x7fce380069d0 2023-11-18T20:43:20.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.920+0000 7fce51a89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4c13c2c0 0x7fce4c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.920+0000 7fce51a89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4c13c2c0 0x7fce4c0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49794/0 (socket says 172.21.15.60:49794) 2023-11-18T20:43:20.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.920+0000 7fce51a89640 1 -- 172.21.15.60:0/2853987387 learned_addr learned my addr 172.21.15.60:0/2853987387 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:20.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce50a87640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce4c13be40 0x7fce4c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce51288640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce380069d0 0x7fce4c13f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce51a89640 1 -- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce380069d0 msgr2=0x7fce4c13f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce51a89640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce380069d0 0x7fce4c13f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce51a89640 1 -- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce4c13be40 msgr2=0x7fce4c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:20.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce51a89640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce4c13be40 0x7fce4c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:20.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce51a89640 1 -- 172.21.15.60:0/2853987387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c13dc60 con 0x7fce4c13c2c0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce51a89640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4c13c2c0 0x7fce4c0b3950 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0cef60 tx=0x7fce44007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce3dffb640 1 -- 172.21.15.60:0/2853987387 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4400ac80 con 0x7fce4c13c2c0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce3dffb640 1 -- 172.21.15.60:0/2853987387 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4400ade0 con 0x7fce4c13c2c0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce534ec640 1 -- 172.21.15.60:0/2853987387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c13f900 con 0x7fce4c13c2c0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce3dffb640 1 -- 172.21.15.60:0/2853987387 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44013630 con 0x7fce4c13c2c0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.921+0000 7fce534ec640 1 -- 172.21.15.60:0/2853987387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c13e360 con 0x7fce4c13c2c0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.922+0000 7fce534ec640 1 -- 172.21.15.60:0/2853987387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c13dac0 con 0x7fce4c13c2c0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.923+0000 7fce3dffb640 1 -- 172.21.15.60:0/2853987387 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce44013790 con 0x7fce4c13c2c0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.923+0000 7fce3dffb640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fce3006ae70 0x7fce3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.923+0000 7fce3dffb640 1 -- 172.21.15.60:0/2853987387 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fce44089df0 con 0x7fce4c13c2c0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.923+0000 7fce51288640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fce3006ae70 0x7fce3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:20.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.924+0000 7fce51288640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fce3006ae70 0x7fce3006d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fce40009b80 tx=0x7fce40008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:20.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:20.927+0000 7fce3dffb640 1 -- 172.21.15.60:0/2853987387 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce4c13dac0 con 0x7fce4c13c2c0 2023-11-18T20:43:21.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.100+0000 7fce534ec640 1 -- 172.21.15.60:0/2853987387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce4c13dac0 con 0x7fce4c13c2c0 2023-11-18T20:43:21.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.101+0000 7fce3dffb640 1 -- 172.21.15.60:0/2853987387 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fce4c13dac0 con 0x7fce4c13c2c0 2023-11-18T20:43:21.102 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:21.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 -- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fce3006ae70 msgr2=0x7fce3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fce3006ae70 0x7fce3006d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fce40009b80 tx=0x7fce40008040 comp rx=0 tx=0).stop 2023-11-18T20:43:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 -- 172.21.15.60:0/2853987387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4c13c2c0 msgr2=0x7fce4c0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4c13c2c0 0x7fce4c0b3950 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0cef60 tx=0x7fce44007bd0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 -- 172.21.15.60:0/2853987387 shutdown_connections 2023-11-18T20:43:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fce3006ae70 0x7fce3006d330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce4c13c2c0 0x7fce4c0b3950 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce4c13be40 0x7fce4c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 --2- 172.21.15.60:0/2853987387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce380069d0 0x7fce4c13f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 -- 172.21.15.60:0/2853987387 >> 172.21.15.60:0/2853987387 conn(0x7fce4c06a4d0 msgr2=0x7fce4c071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 -- 172.21.15.60:0/2853987387 shutdown_connections 2023-11-18T20:43:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.103+0000 7fce534ec640 1 -- 172.21.15.60:0/2853987387 wait complete. 2023-11-18T20:43:21.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.223+0000 7f52836f2640 1 Processor -- start 2023-11-18T20:43:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.223+0000 7f52836f2640 1 -- start start 2023-11-18T20:43:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.224+0000 7f52836f2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f527c140310 0x7f527c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.224+0000 7f52836f2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c06abe0 0x7f527c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.224+0000 7f52836f2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527c1406f0 con 0x7f527c1409d0 2023-11-18T20:43:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.224+0000 7f52836f2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527c161e20 con 0x7f527c140310 2023-11-18T20:43:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.224+0000 7f52836f2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c143960 con 0x7f527c06abe0 2023-11-18T20:43:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.225+0000 7f5281c8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c06abe0 0x7f527c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.225+0000 7f528148e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f527c140310 0x7f527c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.225+0000 7f528148e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f527c140310 0x7f527c1618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:48774/0 (socket says 172.21.15.60:48774) 2023-11-18T20:43:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.225+0000 7f5280c8d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f527c1409d0 0x7f527c06a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:46882/0 (socket says 172.21.15.60:46882) 2023-11-18T20:43:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.225+0000 7f5281c8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c06abe0 0x7f527c06efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:45714/0 (socket says 172.21.15.60:45714) 2023-11-18T20:43:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.225+0000 7f528148e640 1 -- 172.21.15.60:0/2638734105 learned_addr learned my addr 172.21.15.60:0/2638734105 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.225+0000 7f528148e640 1 -- 172.21.15.60:0/2638734105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c06abe0 msgr2=0x7f527c06efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.225+0000 7f528148e640 1 --2- 172.21.15.60:0/2638734105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c06abe0 0x7f527c06efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.225+0000 7f528148e640 1 -- 172.21.15.60:0/2638734105 >> v1:172.21.15.100:6789/0 conn(0x7f527c1409d0 legacy=0x7f527c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.225+0000 7f528148e640 1 -- 172.21.15.60:0/2638734105 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c142de0 con 0x7f527c140310 2023-11-18T20:43:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f528148e640 1 --2- 172.21.15.60:0/2638734105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c140310 0x7f527c1618e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5274009870 tx=0x7f527402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a5a8ab2f3271405 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f527bfff640 1 -- 172.21.15.60:0/2638734105 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274031640 con 0x7f527c140310 2023-11-18T20:43:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f527bfff640 1 -- 172.21.15.60:0/2638734105 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52740317a0 con 0x7f527c140310 2023-11-18T20:43:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f527bfff640 1 -- 172.21.15.60:0/2638734105 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274031b30 con 0x7f527c140310 2023-11-18T20:43:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f52836f2640 1 -- 172.21.15.60:0/2638734105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c140310 msgr2=0x7f527c1618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f52836f2640 1 --2- 172.21.15.60:0/2638734105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c140310 0x7f527c1618e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5274009870 tx=0x7f527402b8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f52836f2640 1 -- 172.21.15.60:0/2638734105 shutdown_connections 2023-11-18T20:43:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f52836f2640 1 --2- 172.21.15.60:0/2638734105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c06abe0 0x7f527c06efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f52836f2640 1 --2- 172.21.15.60:0/2638734105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c140310 0x7f527c1618e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f52836f2640 1 -- 172.21.15.60:0/2638734105 >> 172.21.15.60:0/2638734105 conn(0x7f527c0b1e50 msgr2=0x7f527c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f52836f2640 1 -- 172.21.15.60:0/2638734105 shutdown_connections 2023-11-18T20:43:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.226+0000 7f52836f2640 1 -- 172.21.15.60:0/2638734105 wait complete. 2023-11-18T20:43:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.228+0000 7f52836f2640 1 Processor -- start 2023-11-18T20:43:21.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.228+0000 7f52836f2640 1 -- start start 2023-11-18T20:43:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.228+0000 7f52836f2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f527c06abe0 0x7f527c13c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.228+0000 7f52836f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c140310 0x7f527c13dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.228+0000 7f52836f2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c1409d0 0x7f527c15d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.228+0000 7f52836f2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c145770 con 0x7f527c06abe0 2023-11-18T20:43:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.228+0000 7f52836f2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c144e20 con 0x7f527c140310 2023-11-18T20:43:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.228+0000 7f52836f2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c13cf00 con 0x7f527c1409d0 2023-11-18T20:43:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.228+0000 7f528148e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f527c06abe0 0x7f527c13c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f528148e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f527c06abe0 0x7f527c13c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49806/0 (socket says 172.21.15.60:49806) 2023-11-18T20:43:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f528148e640 1 -- 172.21.15.60:0/2328595964 learned_addr learned my addr 172.21.15.60:0/2328595964 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f5280c8d640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c140310 0x7f527c13dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f5281c8f640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c1409d0 0x7f527c15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f528148e640 1 -- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c1409d0 msgr2=0x7f527c15d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f528148e640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c1409d0 0x7f527c15d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f528148e640 1 -- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c140310 msgr2=0x7f527c13dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f528148e640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c140310 0x7f527c13dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f528148e640 1 -- 172.21.15.60:0/2328595964 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740094d0 con 0x7f527c06abe0 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f5281c8f640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c1409d0 0x7f527c15d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f528148e640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f527c06abe0 0x7f527c13c9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5274000c00 tx=0x7f52740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f5279ffb640 1 -- 172.21.15.60:0/2328595964 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527403a930 con 0x7f527c06abe0 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f5279ffb640 1 -- 172.21.15.60:0/2328595964 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527403aa90 con 0x7f527c06abe0 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f52836f2640 1 -- 172.21.15.60:0/2328595964 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c1f4040 con 0x7f527c06abe0 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f5279ffb640 1 -- 172.21.15.60:0/2328595964 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527403ae40 con 0x7f527c06abe0 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.229+0000 7f52836f2640 1 -- 172.21.15.60:0/2328595964 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c0aeff0 con 0x7f527c06abe0 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.231+0000 7f52836f2640 1 -- 172.21.15.60:0/2328595964 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c1458b0 con 0x7f527c06abe0 2023-11-18T20:43:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.231+0000 7f5279ffb640 1 -- 172.21.15.60:0/2328595964 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f527402f050 con 0x7f527c06abe0 2023-11-18T20:43:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.231+0000 7f5279ffb640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f526806ae40 0x7f526806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.231+0000 7f5279ffb640 1 -- 172.21.15.60:0/2328595964 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f52740ab3e0 con 0x7f527c06abe0 2023-11-18T20:43:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.232+0000 7f5280c8d640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f526806ae40 0x7f526806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.232+0000 7f5280c8d640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f526806ae40 0x7f526806d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f527c071dd0 tx=0x7f5270000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:21.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.236+0000 7f5279ffb640 1 -- 172.21.15.60:0/2328595964 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f527c1458b0 con 0x7f527c06abe0 2023-11-18T20:43:21.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.415+0000 7f52836f2640 1 -- 172.21.15.60:0/2328595964 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f527c1458b0 con 0x7f527c06abe0 2023-11-18T20:43:21.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.415+0000 7f5279ffb640 1 -- 172.21.15.60:0/2328595964 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5274076420 con 0x7f527c06abe0 2023-11-18T20:43:21.417 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.417+0000 7f52836f2640 1 -- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f526806ae40 msgr2=0x7f526806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.417+0000 7f52836f2640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f526806ae40 0x7f526806d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f527c071dd0 tx=0x7f5270000ed0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.417+0000 7f52836f2640 1 -- 172.21.15.60:0/2328595964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f527c06abe0 msgr2=0x7f527c13c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.417+0000 7f52836f2640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f527c06abe0 0x7f527c13c9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5274000c00 tx=0x7f52740046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.417+0000 7f52836f2640 1 -- 172.21.15.60:0/2328595964 shutdown_connections 2023-11-18T20:43:21.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.417+0000 7f52836f2640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f526806ae40 0x7f526806d300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.417+0000 7f52836f2640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f527c1409d0 0x7f527c15d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.417+0000 7f52836f2640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f527c140310 0x7f527c13dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.417+0000 7f52836f2640 1 --2- 172.21.15.60:0/2328595964 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f527c06abe0 0x7f527c13c9c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.417+0000 7f52836f2640 1 -- 172.21.15.60:0/2328595964 >> 172.21.15.60:0/2328595964 conn(0x7f527c0b1e50 msgr2=0x7f527c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:21.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.418+0000 7f52836f2640 1 -- 172.21.15.60:0/2328595964 shutdown_connections 2023-11-18T20:43:21.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.418+0000 7f52836f2640 1 -- 172.21.15.60:0/2328595964 wait complete. 2023-11-18T20:43:21.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.538+0000 7f125264b640 1 Processor -- start 2023-11-18T20:43:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.538+0000 7f125264b640 1 -- start start 2023-11-18T20:43:21.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.539+0000 7f125264b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c144a00 0x7f124c06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.539+0000 7f125264b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f124c144440 0x7f124c06ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.539+0000 7f125264b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0b2ef0 con 0x7f124c1415f0 2023-11-18T20:43:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.539+0000 7f125264b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c0b3b70 con 0x7f124c144440 2023-11-18T20:43:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.539+0000 7f125264b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c142510 con 0x7f124c144a00 2023-11-18T20:43:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.539+0000 7f124bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f124c1415f0 0x7f124c141e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45326/0 (socket says 172.21.15.60:45326) 2023-11-18T20:43:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.539+0000 7f124bfff640 1 -- 172.21.15.60:0/3889520290 learned_addr learned my addr 172.21.15.60:0/3889520290 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.539+0000 7f1250be8640 1 --2- 172.21.15.60:0/3889520290 >> v2:172.21.15.100:3300/0 conn(0x7f124c144440 0x7f124c06ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.539+0000 7f124b7fe640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c144a00 0x7f124c06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359941575 0 0) 0x7f124c0b2ef0 con 0x7f124c1415f0 2023-11-18T20:43:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1238003520 con 0x7f124c1415f0 2023-11-18T20:43:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2683240160 0 0) 0x7f1238003520 con 0x7f124c1415f0 2023-11-18T20:43:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c144a00 msgr2=0x7f124c06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c144a00 0x7f124c06b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 >> v2:172.21.15.100:3300/0 conn(0x7f124c144440 msgr2=0x7f124c06ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:21.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 --2- 172.21.15.60:0/3889520290 >> v2:172.21.15.100:3300/0 conn(0x7f124c144440 0x7f124c06ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0b31d0 con 0x7f124c1415f0 2023-11-18T20:43:21.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f1234001670 con 0x7f124c1415f0 2023-11-18T20:43:21.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 >> v1:172.21.15.60:6789/0 conn(0x7f124c1415f0 legacy=0x7f124c141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238003970 0x7f1238003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c144a00 0x7f12380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1238004d00 0x7f1238005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0b3b70 con 0x7f1238004d00 2023-11-18T20:43:21.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c142510 con 0x7f1238003970 2023-11-18T20:43:21.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123800b980 con 0x7f124c144a00 2023-11-18T20:43:21.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f1250be8640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1238004d00 0x7f1238005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124bfff640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238003970 0x7f1238003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f124b7fe640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c144a00 0x7f12380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f1250be8640 1 -- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c144a00 msgr2=0x7f12380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f1250be8640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c144a00 0x7f12380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f1250be8640 1 -- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238003970 msgr2=0x7f1238003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f1250be8640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238003970 0x7f1238003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.540+0000 7f1250be8640 1 -- 172.21.15.60:0/3889520290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1238003520 con 0x7f1238004d00 2023-11-18T20:43:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.541+0000 7f1250be8640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1238004d00 0x7f1238005180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1244009fc0 tx=0x7f12380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa94fb74abf6774 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:21.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.541+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124402cc60 con 0x7f1238004d00 2023-11-18T20:43:21.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.541+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124400f040 con 0x7f1238004d00 2023-11-18T20:43:21.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.541+0000 7f124affd640 1 -- 172.21.15.60:0/3889520290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244034440 con 0x7f1238004d00 2023-11-18T20:43:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.542+0000 7f125264b640 1 -- 172.21.15.60:0/3889520290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1238004d00 msgr2=0x7f1238005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.542+0000 7f125264b640 1 --2- 172.21.15.60:0/3889520290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1238004d00 0x7f1238005180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1244009fc0 tx=0x7f12380057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.542+0000 7f1250be8640 1 -- 172.21.15.60:0/3889520290 reap_dead start 2023-11-18T20:43:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.542+0000 7f125264b640 1 -- 172.21.15.60:0/3889520290 shutdown_connections 2023-11-18T20:43:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.542+0000 7f125264b640 1 -- 172.21.15.60:0/3889520290 >> 172.21.15.60:0/3889520290 conn(0x7f124c070e70 msgr2=0x7f124c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.542+0000 7f125264b640 1 -- 172.21.15.60:0/3889520290 shutdown_connections 2023-11-18T20:43:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.542+0000 7f125264b640 1 -- 172.21.15.60:0/3889520290 wait complete. 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.543+0000 7f125264b640 1 Processor -- start 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.543+0000 7f125264b640 1 -- start start 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f125264b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f124c156420 0x7f124c06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f125264b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c1593f0 0x7f124c15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f125264b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c156800 0x7f124c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f125264b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c159920 con 0x7f124c1593f0 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f125264b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c159c00 con 0x7f124c156420 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f125264b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c159ee0 con 0x7f124c156800 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f124b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c1593f0 0x7f124c15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f124b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c1593f0 0x7f124c15b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49822/0 (socket says 172.21.15.60:49822) 2023-11-18T20:43:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f124b7fe640 1 -- 172.21.15.60:0/3431486736 learned_addr learned my addr 172.21.15.60:0/3431486736 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f124b7fe640 1 -- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c156800 msgr2=0x7f124c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f124bfff640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f124c156420 0x7f124c06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f1250be8640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c156800 0x7f124c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f124b7fe640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c156800 0x7f124c15be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f124b7fe640 1 -- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f124c156420 msgr2=0x7f124c06e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f124b7fe640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f124c156420 0x7f124c06e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f124b7fe640 1 -- 172.21.15.60:0/3431486736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c15a1c0 con 0x7f124c1593f0 2023-11-18T20:43:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.544+0000 7f124bfff640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f124c156420 0x7f124c06e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.545+0000 7f124b7fe640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c1593f0 0x7f124c15b8c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f123c011910 tx=0x7f123c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:21.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.545+0000 7f1248ff9640 1 -- 172.21.15.60:0/3431486736 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c009590 con 0x7f124c1593f0 2023-11-18T20:43:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.545+0000 7f1248ff9640 1 -- 172.21.15.60:0/3431486736 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c004240 con 0x7f124c1593f0 2023-11-18T20:43:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.545+0000 7f125264b640 1 -- 172.21.15.60:0/3431486736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c15a4a0 con 0x7f124c1593f0 2023-11-18T20:43:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.545+0000 7f1248ff9640 1 -- 172.21.15.60:0/3431486736 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00f8e0 con 0x7f124c1593f0 2023-11-18T20:43:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.545+0000 7f125264b640 1 -- 172.21.15.60:0/3431486736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c06f0e0 con 0x7f124c1593f0 2023-11-18T20:43:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.546+0000 7f125264b640 1 -- 172.21.15.60:0/3431486736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c069400 con 0x7f124c1593f0 2023-11-18T20:43:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.546+0000 7f1248ff9640 1 -- 172.21.15.60:0/3431486736 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f123c00fa40 con 0x7f124c1593f0 2023-11-18T20:43:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.547+0000 7f1248ff9640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f122c06ae70 0x7f122c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.547+0000 7f1248ff9640 1 -- 172.21.15.60:0/3431486736 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f123c089ce0 con 0x7f124c1593f0 2023-11-18T20:43:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.547+0000 7f124bfff640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f122c06ae70 0x7f122c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.547+0000 7f124bfff640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f122c06ae70 0x7f122c06d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1234004840 tx=0x7f1234009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.551+0000 7f1248ff9640 1 -- 172.21.15.60:0/3431486736 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f123c0587a0 con 0x7f124c1593f0 2023-11-18T20:43:21.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.728+0000 7f125264b640 1 -- 172.21.15.60:0/3431486736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f124c066760 con 0x7f124c1593f0 2023-11-18T20:43:21.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.728+0000 7f1248ff9640 1 -- 172.21.15.60:0/3431486736 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f123c0230a0 con 0x7f124c1593f0 2023-11-18T20:43:21.729 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:21.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 -- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f122c06ae70 msgr2=0x7f122c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f122c06ae70 0x7f122c06d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1234004840 tx=0x7f1234009040 comp rx=0 tx=0).stop 2023-11-18T20:43:21.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 -- 172.21.15.60:0/3431486736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c1593f0 msgr2=0x7f124c15b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c1593f0 0x7f124c15b8c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f123c011910 tx=0x7f123c011dd0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 -- 172.21.15.60:0/3431486736 shutdown_connections 2023-11-18T20:43:21.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f122c06ae70 0x7f122c06d330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f124c156800 0x7f124c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f124c1593f0 0x7f124c15b8c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 --2- 172.21.15.60:0/3431486736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f124c156420 0x7f124c06e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 -- 172.21.15.60:0/3431486736 >> 172.21.15.60:0/3431486736 conn(0x7f124c070e70 msgr2=0x7f124c15f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:21.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 -- 172.21.15.60:0/3431486736 shutdown_connections 2023-11-18T20:43:21.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.730+0000 7f125264b640 1 -- 172.21.15.60:0/3431486736 wait complete. 2023-11-18T20:43:21.742 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:43:21.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.848+0000 7f9fc92da640 1 Processor -- start 2023-11-18T20:43:21.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.848+0000 7f9fc92da640 1 -- start start 2023-11-18T20:43:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.848+0000 7f9fc92da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc415e730 0x7f9fc415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.848+0000 7f9fc92da640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc415e2a0 con 0x7f9fc40b1400 2023-11-18T20:43:21.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.848+0000 7f9fc92da640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc415b240 con 0x7f9fc4143490 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.848+0000 7f9fc92da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc415b520 con 0x7f9fc415e730 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.849+0000 7f9fbbfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9fc40b1400 0x7f9fc415a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45334/0 (socket says 172.21.15.60:45334) 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.849+0000 7f9fbbfff640 1 -- 172.21.15.60:0/924883517 learned_addr learned my addr 172.21.15.60:0/924883517 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.849+0000 7f9fc37fe640 1 --2- 172.21.15.60:0/924883517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc415e730 0x7f9fc415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.849+0000 7f9fc27fc640 1 -- 172.21.15.60:0/924883517 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018480585 0 0) 0x7f9fc415b240 con 0x7f9fc4143490 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.849+0000 7f9fc27fc640 1 -- 172.21.15.60:0/924883517 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb0003520 con 0x7f9fc4143490 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.849+0000 7f9fc27fc640 1 -- 172.21.15.60:0/924883517 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609223280 0 0) 0x7f9fc415e2a0 con 0x7f9fc40b1400 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.849+0000 7f9fc27fc640 1 -- 172.21.15.60:0/924883517 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc415b240 con 0x7f9fc40b1400 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.849+0000 7f9fc37fe640 1 -- 172.21.15.60:0/924883517 >> v1:172.21.15.100:6789/0 conn(0x7f9fc4143490 legacy=0x7f9fc40b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.849+0000 7f9fc37fe640 1 -- 172.21.15.60:0/924883517 >> v1:172.21.15.60:6789/0 conn(0x7f9fc40b1400 legacy=0x7f9fc415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.849+0000 7f9fc37fe640 1 -- 172.21.15.60:0/924883517 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc415dc50 con 0x7f9fc415e730 2023-11-18T20:43:21.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.850+0000 7f9fc37fe640 1 --2- 172.21.15.60:0/924883517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc415e730 0x7f9fc415abf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009dc0 tx=0x7f9fbc02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69fcf43514176b48 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.850+0000 7f9fc27fc640 1 -- 172.21.15.60:0/924883517 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc02ccd0 con 0x7f9fc415e730 2023-11-18T20:43:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.850+0000 7f9fc27fc640 1 -- 172.21.15.60:0/924883517 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fbc02ce30 con 0x7f9fc415e730 2023-11-18T20:43:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.850+0000 7f9fc27fc640 1 -- 172.21.15.60:0/924883517 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc034660 con 0x7f9fc415e730 2023-11-18T20:43:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.851+0000 7f9fc92da640 1 -- 172.21.15.60:0/924883517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc415e730 msgr2=0x7f9fc415abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.851+0000 7f9fc92da640 1 --2- 172.21.15.60:0/924883517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc415e730 0x7f9fc415abf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009dc0 tx=0x7f9fbc02bf20 comp rx=0 tx=0).stop 2023-11-18T20:43:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.851+0000 7f9fc92da640 1 -- 172.21.15.60:0/924883517 shutdown_connections 2023-11-18T20:43:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.851+0000 7f9fc92da640 1 --2- 172.21.15.60:0/924883517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc415e730 0x7f9fc415abf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.851+0000 7f9fc92da640 1 -- 172.21.15.60:0/924883517 >> 172.21.15.60:0/924883517 conn(0x7f9fc406a4d0 msgr2=0x7f9fc406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.851+0000 7f9fc92da640 1 -- 172.21.15.60:0/924883517 shutdown_connections 2023-11-18T20:43:21.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.851+0000 7f9fc92da640 1 -- 172.21.15.60:0/924883517 wait complete. 2023-11-18T20:43:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.852+0000 7f9fc92da640 1 Processor -- start 2023-11-18T20:43:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.852+0000 7f9fc92da640 1 -- start start 2023-11-18T20:43:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc92da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fc40b1400 0x7f9fc4159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc92da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc4143490 0x7f9fc415f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc92da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc415e730 0x7f9fc4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc92da640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc415a050 con 0x7f9fc40b1400 2023-11-18T20:43:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc92da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc413f2a0 con 0x7f9fc415e730 2023-11-18T20:43:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc92da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40ad5e0 con 0x7f9fc4143490 2023-11-18T20:43:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fc40b1400 0x7f9fc4159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fc40b1400 0x7f9fc4159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49826/0 (socket says 172.21.15.60:49826) 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc2ffd640 1 -- 172.21.15.60:0/643401721 learned_addr learned my addr 172.21.15.60:0/643401721 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fbbfff640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc4143490 0x7f9fc415f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc37fe640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc415e730 0x7f9fc4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc2ffd640 1 -- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc4143490 msgr2=0x7f9fc415f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc2ffd640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc4143490 0x7f9fc415f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc2ffd640 1 -- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc415e730 msgr2=0x7f9fc4070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc2ffd640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc415e730 0x7f9fc4070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc2ffd640 1 -- 172.21.15.60:0/643401721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc009a70 con 0x7f9fc40b1400 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc37fe640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc415e730 0x7f9fc4070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fbbfff640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc4143490 0x7f9fc415f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc2ffd640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fc40b1400 0x7f9fc4159b10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4001eb0 tx=0x7f9fb400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fbb7fe640 1 -- 172.21.15.60:0/643401721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40052d0 con 0x7f9fc40b1400 2023-11-18T20:43:21.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fbb7fe640 1 -- 172.21.15.60:0/643401721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb4005430 con 0x7f9fc40b1400 2023-11-18T20:43:21.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc92da640 1 -- 172.21.15.60:0/643401721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc40ad8c0 con 0x7f9fc40b1400 2023-11-18T20:43:21.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fbb7fe640 1 -- 172.21.15.60:0/643401721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4012870 con 0x7f9fc40b1400 2023-11-18T20:43:21.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.853+0000 7f9fc92da640 1 -- 172.21.15.60:0/643401721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc40ade80 con 0x7f9fc40b1400 2023-11-18T20:43:21.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.854+0000 7f9fc92da640 1 -- 172.21.15.60:0/643401721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc4069400 con 0x7f9fc40b1400 2023-11-18T20:43:21.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.855+0000 7f9fbb7fe640 1 -- 172.21.15.60:0/643401721 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9fb40035f0 con 0x7f9fc40b1400 2023-11-18T20:43:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.855+0000 7f9fbb7fe640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9fa406ae40 0x7f9fa406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.855+0000 7f9fbb7fe640 1 -- 172.21.15.60:0/643401721 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9fb4088bb0 con 0x7f9fc40b1400 2023-11-18T20:43:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.856+0000 7f9fbbfff640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9fa406ae40 0x7f9fa406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:21.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.856+0000 7f9fbbfff640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9fa406ae40 0x7f9fa406d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40b0920 tx=0x7f9fac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:21.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:21.859+0000 7f9fbb7fe640 1 -- 172.21.15.60:0/643401721 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9fb40577c0 con 0x7f9fc40b1400 2023-11-18T20:43:22.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.071+0000 7f9fc92da640 1 -- 172.21.15.60:0/643401721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fc40ae960 con 0x7f9fc40b1400 2023-11-18T20:43:22.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.072+0000 7f9fbb7fe640 1 -- 172.21.15.60:0/643401721 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f9fb4017090 con 0x7f9fc40b1400 2023-11-18T20:43:22.073 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:22.074 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1530496056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1530496056},{"type":"v1","addr":"172.21.15.60:6833","nonce":1530496056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3842389676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3842389676},{"type":"v1","addr":"172.21.15.60:6835","nonce":3842389676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4248907695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4248907695},{"type":"v1","addr":"172.21.15.100:6836","nonce":4248907695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:42:37.459556+0000","modified":"2023-11-18T20:42:39.458016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1851816776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1851816776},{"type":"v1","addr":"172.21.15.100:6837","nonce":1851816776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-18T20:43:22.074 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2023-11-18T20:43:22.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 -- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9fa406ae40 msgr2=0x7f9fa406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9fa406ae40 0x7f9fa406d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40b0920 tx=0x7f9fac009040 comp rx=0 tx=0).stop 2023-11-18T20:43:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 -- 172.21.15.60:0/643401721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fc40b1400 msgr2=0x7f9fc4159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fc40b1400 0x7f9fc4159b10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4001eb0 tx=0x7f9fb400c960 comp rx=0 tx=0).stop 2023-11-18T20:43:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 -- 172.21.15.60:0/643401721 shutdown_connections 2023-11-18T20:43:22.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9fa406ae40 0x7f9fa406d300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fc415e730 0x7f9fc4070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fc4143490 0x7f9fc415f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 --2- 172.21.15.60:0/643401721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fc40b1400 0x7f9fc4159b10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 -- 172.21.15.60:0/643401721 >> 172.21.15.60:0/643401721 conn(0x7f9fc406a4d0 msgr2=0x7f9fc40cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 -- 172.21.15.60:0/643401721 shutdown_connections 2023-11-18T20:43:22.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.074+0000 7f9fc92da640 1 -- 172.21.15.60:0/643401721 wait complete. 2023-11-18T20:43:22.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:22.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.192+0000 7f87c20bb640 1 Processor -- start 2023-11-18T20:43:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.192+0000 7f87c20bb640 1 -- start start 2023-11-18T20:43:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87c20bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 0x7f87bc06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87c20bb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87bc0aeb80 0x7f87bc06ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87c20bb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87bc0adae0 con 0x7f87bc0ad640 2023-11-18T20:43:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87c20bb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87bc0addc0 con 0x7f87bc0aeb80 2023-11-18T20:43:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87c20bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0ae0a0 con 0x7f87bc15d1d0 2023-11-18T20:43:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87bbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87bc0aeb80 0x7f87bc06ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87baffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f87bc0ad640 0x7f87bc160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45340/0 (socket says 172.21.15.60:45340) 2023-11-18T20:43:22.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87baffd640 1 -- 172.21.15.60:0/3644363039 learned_addr learned my addr 172.21.15.60:0/3644363039 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87bb7fe640 1 --2- 172.21.15.60:0/3644363039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 0x7f87bc06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87bbfff640 1 -- 172.21.15.60:0/3644363039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 msgr2=0x7f87bc06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87bbfff640 1 --2- 172.21.15.60:0/3644363039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 0x7f87bc06b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87bbfff640 1 -- 172.21.15.60:0/3644363039 >> v1:172.21.15.60:6789/0 conn(0x7f87bc0ad640 legacy=0x7f87bc160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87bbfff640 1 -- 172.21.15.60:0/3644363039 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0ae380 con 0x7f87bc0aeb80 2023-11-18T20:43:22.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.193+0000 7f87ba7fc640 1 -- 172.21.15.60:0/3644363039 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393407730 0 0) 0x7f87bc0adae0 con 0x7f87bc0ad640 2023-11-18T20:43:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.194+0000 7f87bbfff640 1 --2- 172.21.15.60:0/3644363039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aeb80 0x7f87bc06ba20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009d90 tx=0x7f87b402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b902966dcdbc90c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.194+0000 7f87ba7fc640 1 -- 172.21.15.60:0/3644363039 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b402cc50 con 0x7f87bc0aeb80 2023-11-18T20:43:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.194+0000 7f87ba7fc640 1 -- 172.21.15.60:0/3644363039 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b402cdb0 con 0x7f87bc0aeb80 2023-11-18T20:43:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.194+0000 7f87ba7fc640 1 -- 172.21.15.60:0/3644363039 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4034660 con 0x7f87bc0aeb80 2023-11-18T20:43:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.195+0000 7f87c20bb640 1 -- 172.21.15.60:0/3644363039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aeb80 msgr2=0x7f87bc06ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.195+0000 7f87c20bb640 1 --2- 172.21.15.60:0/3644363039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aeb80 0x7f87bc06ba20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009d90 tx=0x7f87b402bea0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.195+0000 7f87c20bb640 1 -- 172.21.15.60:0/3644363039 shutdown_connections 2023-11-18T20:43:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.195+0000 7f87c20bb640 1 --2- 172.21.15.60:0/3644363039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0aeb80 0x7f87bc06ba20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.195+0000 7f87c20bb640 1 --2- 172.21.15.60:0/3644363039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 0x7f87bc06b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.195+0000 7f87c20bb640 1 -- 172.21.15.60:0/3644363039 >> 172.21.15.60:0/3644363039 conn(0x7f87bc070e70 msgr2=0x7f87bc071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.195+0000 7f87c20bb640 1 -- 172.21.15.60:0/3644363039 shutdown_connections 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.195+0000 7f87c20bb640 1 -- 172.21.15.60:0/3644363039 wait complete. 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.196+0000 7f87c20bb640 1 Processor -- start 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.196+0000 7f87c20bb640 1 -- start start 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87c20bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad640 0x7f87bc141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87c20bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc0aeb80 0x7f87bc15f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87c20bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 0x7f87bc15f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87c20bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc15fd90 con 0x7f87bc0ad640 2023-11-18T20:43:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87c20bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc160290 con 0x7f87bc0aeb80 2023-11-18T20:43:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87c20bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc15ffb0 con 0x7f87bc15d1d0 2023-11-18T20:43:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad640 0x7f87bc141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad640 0x7f87bc141460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49844/0 (socket says 172.21.15.60:49844) 2023-11-18T20:43:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bb7fe640 1 -- 172.21.15.60:0/2602753576 learned_addr learned my addr 172.21.15.60:0/2602753576 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87baffd640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc0aeb80 0x7f87bc15f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bbfff640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 0x7f87bc15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bb7fe640 1 -- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 msgr2=0x7f87bc15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bb7fe640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 0x7f87bc15f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bb7fe640 1 -- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc0aeb80 msgr2=0x7f87bc15f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bb7fe640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc0aeb80 0x7f87bc15f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bb7fe640 1 -- 172.21.15.60:0/2602753576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b4009a70 con 0x7f87bc0ad640 2023-11-18T20:43:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bbfff640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 0x7f87bc15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87bb7fe640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad640 0x7f87bc141460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00d5f0 tx=0x7f87ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87a3fff640 1 -- 172.21.15.60:0/2602753576 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac0125f0 con 0x7f87bc0ad640 2023-11-18T20:43:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87a3fff640 1 -- 172.21.15.60:0/2602753576 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac012750 con 0x7f87bc0ad640 2023-11-18T20:43:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87a3fff640 1 -- 172.21.15.60:0/2602753576 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac012b00 con 0x7f87bc0ad640 2023-11-18T20:43:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.197+0000 7f87c20bb640 1 -- 172.21.15.60:0/2602753576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc1572e0 con 0x7f87bc0ad640 2023-11-18T20:43:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.198+0000 7f87c20bb640 1 -- 172.21.15.60:0/2602753576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc1578a0 con 0x7f87bc0ad640 2023-11-18T20:43:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.199+0000 7f87c20bb640 1 -- 172.21.15.60:0/2602753576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc069400 con 0x7f87bc0ad640 2023-11-18T20:43:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.199+0000 7f87a3fff640 1 -- 172.21.15.60:0/2602753576 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87ac012c60 con 0x7f87bc0ad640 2023-11-18T20:43:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.199+0000 7f87a3fff640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f87a806ae40 0x7f87a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.200+0000 7f87a3fff640 1 -- 172.21.15.60:0/2602753576 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f87ac089ae0 con 0x7f87bc0ad640 2023-11-18T20:43:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.200+0000 7f87baffd640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f87a806ae40 0x7f87a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.200+0000 7f87baffd640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f87a806ae40 0x7f87a806d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f87bc150bb0 tx=0x7f87b0007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:22.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.204+0000 7f87a3fff640 1 -- 172.21.15.60:0/2602753576 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f87ac054a90 con 0x7f87bc0ad640 2023-11-18T20:43:22.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.383+0000 7f87c20bb640 1 -- 172.21.15.60:0/2602753576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87bc0b4d90 con 0x7f87bc0ad640 2023-11-18T20:43:22.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.385+0000 7f87a3fff640 1 -- 172.21.15.60:0/2602753576 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16683 (secure 0 0 0) 0x7f87ac0580b0 con 0x7f87bc0ad640 2023-11-18T20:43:22.386 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:22.386 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:19.231773+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:22.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.386+0000 7f87c20bb640 1 -- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f87a806ae40 msgr2=0x7f87a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.386+0000 7f87c20bb640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f87a806ae40 0x7f87a806d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f87bc150bb0 tx=0x7f87b0007d50 comp rx=0 tx=0).stop 2023-11-18T20:43:22.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.386+0000 7f87c20bb640 1 -- 172.21.15.60:0/2602753576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad640 msgr2=0x7f87bc141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.386+0000 7f87c20bb640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad640 0x7f87bc141460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00d5f0 tx=0x7f87ac00dab0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.386+0000 7f87c20bb640 1 -- 172.21.15.60:0/2602753576 shutdown_connections 2023-11-18T20:43:22.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.386+0000 7f87c20bb640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f87a806ae40 0x7f87a806d300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.386+0000 7f87c20bb640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc15d1d0 0x7f87bc15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.386+0000 7f87c20bb640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc0aeb80 0x7f87bc15f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.386+0000 7f87c20bb640 1 --2- 172.21.15.60:0/2602753576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad640 0x7f87bc141460 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.387+0000 7f87c20bb640 1 -- 172.21.15.60:0/2602753576 >> 172.21.15.60:0/2602753576 conn(0x7f87bc070e70 msgr2=0x7f87bc0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:22.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.387+0000 7f87c20bb640 1 -- 172.21.15.60:0/2602753576 shutdown_connections 2023-11-18T20:43:22.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.387+0000 7f87c20bb640 1 -- 172.21.15.60:0/2602753576 wait complete. 2023-11-18T20:43:22.400 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-18T20:43:22.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:43:22.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.505+0000 7f76ce0da640 1 Processor -- start 2023-11-18T20:43:22.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.505+0000 7f76ce0da640 1 -- start start 2023-11-18T20:43:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76ce0da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80b4370 0x7f76c80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76ce0da640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c8071770 con 0x7f76c8144980 2023-11-18T20:43:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76ce0da640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c80ccb60 con 0x7f76c80ceb70 2023-11-18T20:43:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76ce0da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c815ced0 con 0x7f76c80b4370 2023-11-18T20:43:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76c77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f76c8144980 0x7f76c815c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45350/0 (socket says 172.21.15.60:45350) 2023-11-18T20:43:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76c77fe640 1 -- 172.21.15.60:0/2096736362 learned_addr learned my addr 172.21.15.60:0/2096736362 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2096736362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80b4370 0x7f76c80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76c67fc640 1 -- 172.21.15.60:0/2096736362 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768726852 0 0) 0x7f76c8071770 con 0x7f76c8144980 2023-11-18T20:43:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76c67fc640 1 -- 172.21.15.60:0/2096736362 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76b0003520 con 0x7f76c8144980 2023-11-18T20:43:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76c67fc640 1 -- 172.21.15.60:0/2096736362 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142981983 0 0) 0x7f76c80ccb60 con 0x7f76c80ceb70 2023-11-18T20:43:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76c67fc640 1 -- 172.21.15.60:0/2096736362 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c8071770 con 0x7f76c80ceb70 2023-11-18T20:43:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76c7fff640 1 -- 172.21.15.60:0/2096736362 >> v1:172.21.15.100:6789/0 conn(0x7f76c80ceb70 legacy=0x7f76c80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76c7fff640 1 -- 172.21.15.60:0/2096736362 >> v1:172.21.15.60:6789/0 conn(0x7f76c8144980 legacy=0x7f76c815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.506+0000 7f76c7fff640 1 -- 172.21.15.60:0/2096736362 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c80b4ce0 con 0x7f76c80b4370 2023-11-18T20:43:22.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.507+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2096736362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80b4370 0x7f76c80b47a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76c80b4e70 tx=0x7f76c002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7e33e15d077ac1d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:22.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.507+0000 7f76c67fc640 1 -- 172.21.15.60:0/2096736362 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c002ccd0 con 0x7f76c80b4370 2023-11-18T20:43:22.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.507+0000 7f76c67fc640 1 -- 172.21.15.60:0/2096736362 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c002ce30 con 0x7f76c80b4370 2023-11-18T20:43:22.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.507+0000 7f76c67fc640 1 -- 172.21.15.60:0/2096736362 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0034660 con 0x7f76c80b4370 2023-11-18T20:43:22.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.508+0000 7f76ce0da640 1 -- 172.21.15.60:0/2096736362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80b4370 msgr2=0x7f76c80b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.508+0000 7f76ce0da640 1 --2- 172.21.15.60:0/2096736362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80b4370 0x7f76c80b47a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76c80b4e70 tx=0x7f76c002bf20 comp rx=0 tx=0).stop 2023-11-18T20:43:22.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.508+0000 7f76ce0da640 1 -- 172.21.15.60:0/2096736362 shutdown_connections 2023-11-18T20:43:22.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.508+0000 7f76ce0da640 1 --2- 172.21.15.60:0/2096736362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80b4370 0x7f76c80b47a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.508+0000 7f76ce0da640 1 -- 172.21.15.60:0/2096736362 >> 172.21.15.60:0/2096736362 conn(0x7f76c806a4d0 msgr2=0x7f76c806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:22.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.508+0000 7f76ce0da640 1 -- 172.21.15.60:0/2096736362 shutdown_connections 2023-11-18T20:43:22.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.508+0000 7f76ce0da640 1 -- 172.21.15.60:0/2096736362 wait complete. 2023-11-18T20:43:22.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.509+0000 7f76ce0da640 1 Processor -- start 2023-11-18T20:43:22.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.509+0000 7f76ce0da640 1 -- start start 2023-11-18T20:43:22.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76ce0da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80b4370 0x7f76c81eff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76ce0da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80ceb70 0x7f76c8157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76ce0da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8144980 0x7f76c80a8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76ce0da640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c80af4c0 con 0x7f76c8144980 2023-11-18T20:43:22.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76ce0da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c81f04c0 con 0x7f76c80b4370 2023-11-18T20:43:22.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76ce0da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c8157f70 con 0x7f76c80ceb70 2023-11-18T20:43:22.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8144980 0x7f76c80a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8144980 0x7f76c80a8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49850/0 (socket says 172.21.15.60:49850) 2023-11-18T20:43:22.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76c7fff640 1 -- 172.21.15.60:0/2266439819 learned_addr learned my addr 172.21.15.60:0/2266439819 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76c77fe640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80b4370 0x7f76c81eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.510+0000 7f76c6ffd640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80ceb70 0x7f76c8157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76c7fff640 1 -- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80ceb70 msgr2=0x7f76c8157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80ceb70 0x7f76c8157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76c7fff640 1 -- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80b4370 msgr2=0x7f76c81eff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80b4370 0x7f76c81eff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76c7fff640 1 -- 172.21.15.60:0/2266439819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c0009a70 con 0x7f76c8144980 2023-11-18T20:43:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76c6ffd640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80ceb70 0x7f76c8157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76c7fff640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8144980 0x7f76c80a8690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76c0037040 tx=0x7f76c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76affff640 1 -- 172.21.15.60:0/2266439819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c003d600 con 0x7f76c8144980 2023-11-18T20:43:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76affff640 1 -- 172.21.15.60:0/2266439819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c003d780 con 0x7f76c8144980 2023-11-18T20:43:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76ce0da640 1 -- 172.21.15.60:0/2266439819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c8158250 con 0x7f76c8144980 2023-11-18T20:43:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76affff640 1 -- 172.21.15.60:0/2266439819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c003db30 con 0x7f76c8144980 2023-11-18T20:43:22.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.511+0000 7f76ce0da640 1 -- 172.21.15.60:0/2266439819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c80af760 con 0x7f76c8144980 2023-11-18T20:43:22.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.512+0000 7f76ce0da640 1 -- 172.21.15.60:0/2266439819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c80af950 con 0x7f76c8144980 2023-11-18T20:43:22.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.513+0000 7f76affff640 1 -- 172.21.15.60:0/2266439819 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76c003b040 con 0x7f76c8144980 2023-11-18T20:43:22.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.513+0000 7f76affff640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76b406ae40 0x7f76b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.513+0000 7f76affff640 1 -- 172.21.15.60:0/2266439819 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f76c00ab6b0 con 0x7f76c8144980 2023-11-18T20:43:22.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.513+0000 7f76c77fe640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76b406ae40 0x7f76b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.514+0000 7f76c77fe640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76b406ae40 0x7f76b406d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f76b8000f60 tx=0x7f76b8007d70 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.517+0000 7f76affff640 1 -- 172.21.15.60:0/2266439819 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76c0076940 con 0x7f76c8144980 2023-11-18T20:43:22.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.729+0000 7f76ce0da640 1 -- 172.21.15.60:0/2266439819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76c8069400 con 0x7f76c8144980 2023-11-18T20:43:22.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.729+0000 7f76affff640 1 -- 172.21.15.60:0/2266439819 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f76c0009450 con 0x7f76c8144980 2023-11-18T20:43:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:22.731 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1530496056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1530496056},{"type":"v1","addr":"172.21.15.60:6833","nonce":1530496056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3842389676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3842389676},{"type":"v1","addr":"172.21.15.60:6835","nonce":3842389676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4248907695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4248907695},{"type":"v1","addr":"172.21.15.100:6836","nonce":4248907695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:42:37.459556+0000","modified":"2023-11-18T20:42:39.458016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1851816776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1851816776},{"type":"v1","addr":"172.21.15.100:6837","nonce":1851816776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-18T20:43:22.731 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2023-11-18T20:43:22.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 -- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76b406ae40 msgr2=0x7f76b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76b406ae40 0x7f76b406d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f76b8000f60 tx=0x7f76b8007d70 comp rx=0 tx=0).stop 2023-11-18T20:43:22.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 -- 172.21.15.60:0/2266439819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8144980 msgr2=0x7f76c80a8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8144980 0x7f76c80a8690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76c0037040 tx=0x7f76c0004a40 comp rx=0 tx=0).stop 2023-11-18T20:43:22.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 -- 172.21.15.60:0/2266439819 shutdown_connections 2023-11-18T20:43:22.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76b406ae40 0x7f76b406d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c8144980 0x7f76c80a8690 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c80ceb70 0x7f76c8157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 --2- 172.21.15.60:0/2266439819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c80b4370 0x7f76c81eff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 -- 172.21.15.60:0/2266439819 >> 172.21.15.60:0/2266439819 conn(0x7f76c806a4d0 msgr2=0x7f76c8143c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:22.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 -- 172.21.15.60:0/2266439819 shutdown_connections 2023-11-18T20:43:22.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.731+0000 7f76ce0da640 1 -- 172.21.15.60:0/2266439819 wait complete. 2023-11-18T20:43:22.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:43:22.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.849+0000 7f3f5ad83640 1 Processor -- start 2023-11-18T20:43:22.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.849+0000 7f3f5ad83640 1 -- start start 2023-11-18T20:43:22.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.850+0000 7f3f5ad83640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415d1d0 0x7f3f5406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.850+0000 7f3f5ad83640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f54161ce0 con 0x7f3f540aeb80 2023-11-18T20:43:22.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.850+0000 7f3f5ad83640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f540adae0 con 0x7f3f540ad640 2023-11-18T20:43:22.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.850+0000 7f3f5ad83640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540addc0 con 0x7f3f5415d1d0 2023-11-18T20:43:22.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.850+0000 7f3f59320640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3f540aeb80 0x7f3f541615d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45354/0 (socket says 172.21.15.60:45354) 2023-11-18T20:43:22.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.850+0000 7f3f59320640 1 -- 172.21.15.60:0/3713373920 learned_addr learned my addr 172.21.15.60:0/3713373920 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:22.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.850+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793628828 0 0) 0x7f3f54161ce0 con 0x7f3f540aeb80 2023-11-18T20:43:22.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.850+0000 7f3f58b1f640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415d1d0 0x7f3f5406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.850+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f44003520 con 0x7f3f540aeb80 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3938479831 0 0) 0x7f3f44003520 con 0x7f3f540aeb80 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415d1d0 msgr2=0x7f3f5406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415d1d0 0x7f3f5406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 >> v1:172.21.15.100:6789/0 conn(0x7f3f540ad640 legacy=0x7f3f54160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540ae0a0 con 0x7f3f540aeb80 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f3f4c0019f0 con 0x7f3f540aeb80 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 >> v1:172.21.15.60:6789/0 conn(0x7f3f540aeb80 legacy=0x7f3f541615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f440039a0 0x7f3f44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 0x7f3f44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415d1d0 0x7f3f44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540addc0 con 0x7f3f440047c0 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4400ba30 con 0x7f3f440039a0 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4400bbf0 con 0x7f3f5415d1d0 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f53fff640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 0x7f3f44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f58b1f640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f440039a0 0x7f3f44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f59320640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415d1d0 0x7f3f44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f53fff640 1 -- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415d1d0 msgr2=0x7f3f44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f53fff640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415d1d0 0x7f3f44005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f53fff640 1 -- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f440039a0 msgr2=0x7f3f44003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f53fff640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f440039a0 0x7f3f44003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.851+0000 7f3f53fff640 1 -- 172.21.15.60:0/3713373920 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f44003520 con 0x7f3f440047c0 2023-11-18T20:43:22.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.852+0000 7f3f53fff640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 0x7f3f44004c40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3f40001720 tx=0x7f3f440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88820e2782d8a57c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:22.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.852+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400052d0 con 0x7f3f440047c0 2023-11-18T20:43:22.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.852+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4002e050 con 0x7f3f440047c0 2023-11-18T20:43:22.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.852+0000 7f3f537fe640 1 -- 172.21.15.60:0/3713373920 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400035f0 con 0x7f3f440047c0 2023-11-18T20:43:22.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.853+0000 7f3f5ad83640 1 -- 172.21.15.60:0/3713373920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 msgr2=0x7f3f44004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.853+0000 7f3f5ad83640 1 --2- 172.21.15.60:0/3713373920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 0x7f3f44004c40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3f40001720 tx=0x7f3f440057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.853+0000 7f3f59320640 1 -- 172.21.15.60:0/3713373920 reap_dead start 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.853+0000 7f3f5ad83640 1 -- 172.21.15.60:0/3713373920 shutdown_connections 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.853+0000 7f3f5ad83640 1 -- 172.21.15.60:0/3713373920 >> 172.21.15.60:0/3713373920 conn(0x7f3f54070e70 msgr2=0x7f3f54071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.853+0000 7f3f5ad83640 1 -- 172.21.15.60:0/3713373920 shutdown_connections 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.853+0000 7f3f5ad83640 1 -- 172.21.15.60:0/3713373920 wait complete. 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.854+0000 7f3f5ad83640 1 Processor -- start 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.854+0000 7f3f5ad83640 1 -- start start 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f5ad83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 0x7f3f5415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f5ad83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f54151da0 0x7f3f54141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f5ad83640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f54140a70 0x7f3f541433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f5ad83640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5415e480 con 0x7f3f440047c0 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f5ad83640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541517f0 con 0x7f3f54151da0 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f5ad83640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54152220 con 0x7f3f54140a70 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f58b1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 0x7f3f5415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f58b1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 0x7f3f5415bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49876/0 (socket says 172.21.15.60:49876) 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f58b1f640 1 -- 172.21.15.60:0/2592852360 learned_addr learned my addr 172.21.15.60:0/2592852360 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f53fff640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f54151da0 0x7f3f54141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f59320640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f54140a70 0x7f3f541433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f58b1f640 1 -- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f54140a70 msgr2=0x7f3f541433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f58b1f640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f54140a70 0x7f3f541433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f58b1f640 1 -- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f54151da0 msgr2=0x7f3f54141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f58b1f640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f54151da0 0x7f3f54141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:22.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f58b1f640 1 -- 172.21.15.60:0/2592852360 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5415dae0 con 0x7f3f440047c0 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f53fff640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f54151da0 0x7f3f54141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f59320640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f54140a70 0x7f3f541433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f58b1f640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 0x7f3f5415bb10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3f480118e0 tx=0x7f3f48011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f517fa640 1 -- 172.21.15.60:0/2592852360 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800fa90 con 0x7f3f440047c0 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.855+0000 7f3f517fa640 1 -- 172.21.15.60:0/2592852360 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4800fbf0 con 0x7f3f440047c0 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.856+0000 7f3f5ad83640 1 -- 172.21.15.60:0/2592852360 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5415ddc0 con 0x7f3f440047c0 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.856+0000 7f3f517fa640 1 -- 172.21.15.60:0/2592852360 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48014440 con 0x7f3f440047c0 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.856+0000 7f3f5ad83640 1 -- 172.21.15.60:0/2592852360 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f541418c0 con 0x7f3f440047c0 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.857+0000 7f3f5ad83640 1 -- 172.21.15.60:0/2592852360 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5415e6f0 con 0x7f3f440047c0 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.857+0000 7f3f517fa640 1 -- 172.21.15.60:0/2592852360 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f480145a0 con 0x7f3f440047c0 2023-11-18T20:43:22.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.857+0000 7f3f517fa640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3806ae40 0x7f3f3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:22.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.858+0000 7f3f517fa640 1 -- 172.21.15.60:0/2592852360 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3f48089f00 con 0x7f3f440047c0 2023-11-18T20:43:22.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.858+0000 7f3f53fff640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3806ae40 0x7f3f3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:22.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.858+0000 7f3f53fff640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3806ae40 0x7f3f3806d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3f40033000 tx=0x7f3f40009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:22.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:22.862+0000 7f3f517fa640 1 -- 172.21.15.60:0/2592852360 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f48054f30 con 0x7f3f440047c0 2023-11-18T20:43:23.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.074+0000 7f3f5ad83640 1 -- 172.21.15.60:0/2592852360 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f54069400 con 0x7f3f440047c0 2023-11-18T20:43:23.075 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.074+0000 7f3f517fa640 1 -- 172.21.15.60:0/2592852360 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f3f48058550 con 0x7f3f440047c0 2023-11-18T20:43:23.076 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:23.076 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1530496056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1530496056},{"type":"v1","addr":"172.21.15.60:6833","nonce":1530496056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3842389676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3842389676},{"type":"v1","addr":"172.21.15.60:6835","nonce":3842389676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4248907695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4248907695},{"type":"v1","addr":"172.21.15.100:6836","nonce":4248907695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:42:37.459556+0000","modified":"2023-11-18T20:42:39.458016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1851816776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1851816776},{"type":"v1","addr":"172.21.15.100:6837","nonce":1851816776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-18T20:43:23.077 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2023-11-18T20:43:23.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 -- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3806ae40 msgr2=0x7f3f3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3806ae40 0x7f3f3806d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3f40033000 tx=0x7f3f40009030 comp rx=0 tx=0).stop 2023-11-18T20:43:23.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 -- 172.21.15.60:0/2592852360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 msgr2=0x7f3f5415bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 0x7f3f5415bb10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3f480118e0 tx=0x7f3f48011da0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 -- 172.21.15.60:0/2592852360 shutdown_connections 2023-11-18T20:43:23.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3806ae40 0x7f3f3806d300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f54140a70 0x7f3f541433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f54151da0 0x7f3f54141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 --2- 172.21.15.60:0/2592852360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f440047c0 0x7f3f5415bb10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 -- 172.21.15.60:0/2592852360 >> 172.21.15.60:0/2592852360 conn(0x7f3f54070e70 msgr2=0x7f3f5406d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:23.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 -- 172.21.15.60:0/2592852360 shutdown_connections 2023-11-18T20:43:23.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.076+0000 7f3f5ad83640 1 -- 172.21.15.60:0/2592852360 wait complete. 2023-11-18T20:43:23.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:23.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.194+0000 7f98f2748640 1 Processor -- start 2023-11-18T20:43:23.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.194+0000 7f98f2748640 1 -- start start 2023-11-18T20:43:23.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.195+0000 7f98f2748640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f98ec144980 0x7f98ec15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.195+0000 7f98f2748640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98ec0ceb70 0x7f98ec0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.195+0000 7f98f2748640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ec071770 con 0x7f98ec0b4370 2023-11-18T20:43:23.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.195+0000 7f98f2748640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ec0ccdc0 con 0x7f98ec0ceb70 2023-11-18T20:43:23.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.195+0000 7f98f2748640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ec0cd0a0 con 0x7f98ec144980 2023-11-18T20:43:23.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.195+0000 7f98f0ce5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f98ec0b4370 0x7f98ec071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45358/0 (socket says 172.21.15.60:45358) 2023-11-18T20:43:23.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.195+0000 7f98eb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98ec0ceb70 0x7f98ec0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.195+0000 7f98f0ce5640 1 -- 172.21.15.60:0/863570238 learned_addr learned my addr 172.21.15.60:0/863570238 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:23.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.195+0000 7f98ebfff640 1 --2- 172.21.15.60:0/863570238 >> v2:172.21.15.100:3300/0 conn(0x7f98ec144980 0x7f98ec15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.195+0000 7f98eaffd640 1 -- 172.21.15.60:0/863570238 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193544283 0 0) 0x7f98ec071770 con 0x7f98ec0b4370 2023-11-18T20:43:23.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.196+0000 7f98eaffd640 1 -- 172.21.15.60:0/863570238 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98d8003520 con 0x7f98ec0b4370 2023-11-18T20:43:23.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.196+0000 7f98eb7fe640 1 -- 172.21.15.60:0/863570238 >> v2:172.21.15.100:3300/0 conn(0x7f98ec144980 msgr2=0x7f98ec15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:23.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.196+0000 7f98eb7fe640 1 --2- 172.21.15.60:0/863570238 >> v2:172.21.15.100:3300/0 conn(0x7f98ec144980 0x7f98ec15c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.196+0000 7f98eb7fe640 1 -- 172.21.15.60:0/863570238 >> v1:172.21.15.60:6789/0 conn(0x7f98ec0b4370 legacy=0x7f98ec071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.196+0000 7f98eb7fe640 1 -- 172.21.15.60:0/863570238 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec0cd380 con 0x7f98ec0ceb70 2023-11-18T20:43:23.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.196+0000 7f98eb7fe640 1 --2- 172.21.15.60:0/863570238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec0ceb70 0x7f98ec0cc450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0b4ec0 tx=0x7f98d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7d93849cca77df server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:23.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.196+0000 7f98eaffd640 1 -- 172.21.15.60:0/863570238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4031830 con 0x7f98ec0ceb70 2023-11-18T20:43:23.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.197+0000 7f98eaffd640 1 -- 172.21.15.60:0/863570238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d4031990 con 0x7f98ec0ceb70 2023-11-18T20:43:23.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.197+0000 7f98eaffd640 1 -- 172.21.15.60:0/863570238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4031d20 con 0x7f98ec0ceb70 2023-11-18T20:43:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.197+0000 7f98f2748640 1 -- 172.21.15.60:0/863570238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec0ceb70 msgr2=0x7f98ec0cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.197+0000 7f98f2748640 1 --2- 172.21.15.60:0/863570238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec0ceb70 0x7f98ec0cc450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0b4ec0 tx=0x7f98d402b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.197+0000 7f98f2748640 1 -- 172.21.15.60:0/863570238 shutdown_connections 2023-11-18T20:43:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.197+0000 7f98f2748640 1 --2- 172.21.15.60:0/863570238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec0ceb70 0x7f98ec0cc450 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.197+0000 7f98f2748640 1 --2- 172.21.15.60:0/863570238 >> v2:172.21.15.100:3300/0 conn(0x7f98ec144980 0x7f98ec15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.197+0000 7f98f2748640 1 -- 172.21.15.60:0/863570238 >> 172.21.15.60:0/863570238 conn(0x7f98ec06a4d0 msgr2=0x7f98ec06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:23.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.197+0000 7f98f2748640 1 -- 172.21.15.60:0/863570238 shutdown_connections 2023-11-18T20:43:23.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.197+0000 7f98f2748640 1 -- 172.21.15.60:0/863570238 wait complete. 2023-11-18T20:43:23.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.198+0000 7f98f2748640 1 Processor -- start 2023-11-18T20:43:23.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.198+0000 7f98f2748640 1 -- start start 2023-11-18T20:43:23.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.199+0000 7f98f2748640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec0b4370 0x7f98ec1f14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.199+0000 7f98f2748640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec0ceb70 0x7f98ec13ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.199+0000 7f98f2748640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec144980 0x7f98ec1eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.199+0000 7f98f2748640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec13b090 con 0x7f98ec144980 2023-11-18T20:43:23.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.199+0000 7f98f2748640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec13f580 con 0x7f98ec0ceb70 2023-11-18T20:43:23.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.199+0000 7f98f2748640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec13e100 con 0x7f98ec0b4370 2023-11-18T20:43:23.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.199+0000 7f98f0ce5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec144980 0x7f98ec1eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.199+0000 7f98f0ce5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec144980 0x7f98ec1eddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49898/0 (socket says 172.21.15.60:49898) 2023-11-18T20:43:23.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.199+0000 7f98f0ce5640 1 -- 172.21.15.60:0/3895591289 learned_addr learned my addr 172.21.15.60:0/3895591289 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:23.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98f0ce5640 1 -- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec0b4370 msgr2=0x7f98ec1f14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98f0ce5640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec0b4370 0x7f98ec1f14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98f0ce5640 1 -- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec0ceb70 msgr2=0x7f98ec13ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98f0ce5640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec0ceb70 0x7f98ec13ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98f0ce5640 1 -- 172.21.15.60:0/3895591289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4009400 con 0x7f98ec144980 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98f0ce5640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec144980 0x7f98ec1eddd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f98e400cb40 tx=0x7f98e400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98e8ff9640 1 -- 172.21.15.60:0/3895591289 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e400aca0 con 0x7f98ec144980 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98e8ff9640 1 -- 172.21.15.60:0/3895591289 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e400ae00 con 0x7f98ec144980 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98f2748640 1 -- 172.21.15.60:0/3895591289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ec13db40 con 0x7f98ec144980 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98e8ff9640 1 -- 172.21.15.60:0/3895591289 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4012620 con 0x7f98ec144980 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.200+0000 7f98f2748640 1 -- 172.21.15.60:0/3895591289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ec13f800 con 0x7f98ec144980 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.201+0000 7f98f2748640 1 -- 172.21.15.60:0/3895591289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ec069400 con 0x7f98ec144980 2023-11-18T20:43:23.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.201+0000 7f98e8ff9640 1 -- 172.21.15.60:0/3895591289 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98e4007070 con 0x7f98ec144980 2023-11-18T20:43:23.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.202+0000 7f98e8ff9640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98cc06ae70 0x7f98cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.202+0000 7f98e8ff9640 1 -- 172.21.15.60:0/3895591289 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f98e4089780 con 0x7f98ec144980 2023-11-18T20:43:23.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.202+0000 7f98ebfff640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98cc06ae70 0x7f98cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.203+0000 7f98ebfff640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98cc06ae70 0x7f98cc06d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98dc006a90 tx=0x7f98dc006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:23.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.206+0000 7f98e8ff9640 1 -- 172.21.15.60:0/3895591289 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98e4054710 con 0x7f98ec144980 2023-11-18T20:43:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.384+0000 7f98f2748640 1 -- 172.21.15.60:0/3895591289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98ec13b2c0 con 0x7f98ec144980 2023-11-18T20:43:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.385+0000 7f98e8ff9640 1 -- 172.21.15.60:0/3895591289 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16683 (secure 0 0 0) 0x7f98e4057d30 con 0x7f98ec144980 2023-11-18T20:43:23.387 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:23.387 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:19.231773+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:42:35.573014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-18T20:42:36.443031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:23.388 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 -- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98cc06ae70 msgr2=0x7f98cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98cc06ae70 0x7f98cc06d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98dc006a90 tx=0x7f98dc006a20 comp rx=0 tx=0).stop 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 -- 172.21.15.60:0/3895591289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec144980 msgr2=0x7f98ec1eddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec144980 0x7f98ec1eddd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f98e400cb40 tx=0x7f98e400a060 comp rx=0 tx=0).stop 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 -- 172.21.15.60:0/3895591289 shutdown_connections 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f98cc06ae70 0x7f98cc06d330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98ec144980 0x7f98ec1eddd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec0ceb70 0x7f98ec13ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 --2- 172.21.15.60:0/3895591289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98ec0b4370 0x7f98ec1f14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 -- 172.21.15.60:0/3895591289 >> 172.21.15.60:0/3895591289 conn(0x7f98ec06a4d0 msgr2=0x7f98ec156140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 -- 172.21.15.60:0/3895591289 shutdown_connections 2023-11-18T20:43:23.389 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.387+0000 7f98f2748640 1 -- 172.21.15.60:0/3895591289 wait complete. 2023-11-18T20:43:23.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-18T20:43:23.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.505+0000 7f58e11f0640 1 Processor -- start 2023-11-18T20:43:23.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.505+0000 7f58e11f0640 1 -- start start 2023-11-18T20:43:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58e11f0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f58dc0ceb70 0x7f58dc0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58e11f0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc071770 con 0x7f58dc0b4370 2023-11-18T20:43:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58e11f0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc0ccdc0 con 0x7f58dc144980 2023-11-18T20:43:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58e11f0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc15ced0 con 0x7f58dc0ceb70 2023-11-18T20:43:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58db59e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f58dc0b4370 0x7f58dc071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45370/0 (socket says 172.21.15.60:45370) 2023-11-18T20:43:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58db59e640 1 -- 172.21.15.60:0/3029678103 learned_addr learned my addr 172.21.15.60:0/3029678103 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58d3fff640 1 --2- 172.21.15.60:0/3029678103 >> v2:172.21.15.100:3300/0 conn(0x7f58dc0ceb70 0x7f58dc0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246851453 0 0) 0x7f58dc071770 con 0x7f58dc0b4370 2023-11-18T20:43:23.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58c8003520 con 0x7f58dc0b4370 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1585739361 0 0) 0x7f58c8003520 con 0x7f58dc0b4370 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 >> v2:172.21.15.100:3300/0 conn(0x7f58dc0ceb70 msgr2=0x7f58dc0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58da59c640 1 --2- 172.21.15.60:0/3029678103 >> v2:172.21.15.100:3300/0 conn(0x7f58dc0ceb70 0x7f58dc0cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 >> v1:172.21.15.100:6789/0 conn(0x7f58dc144980 legacy=0x7f58dc15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.506+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc0cd0a0 con 0x7f58dc0b4370 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f58d40019f0 con 0x7f58dc0b4370 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 >> v1:172.21.15.60:6789/0 conn(0x7f58dc0b4370 legacy=0x7f58dc071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58da59c640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58c8003970 0x7f58c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58da59c640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c80047c0 0x7f58c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58da59c640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58c8005180 0x7f58c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc15ced0 con 0x7f58c8003970 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c800c020 con 0x7f58c80047c0 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c800c1e0 con 0x7f58c8005180 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58dad9d640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58c8003970 0x7f58c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58d3fff640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c80047c0 0x7f58c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58db59e640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58c8005180 0x7f58c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58dad9d640 1 -- 172.21.15.60:0/3029678103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58c8005180 msgr2=0x7f58c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58dad9d640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58c8005180 0x7f58c800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58dad9d640 1 -- 172.21.15.60:0/3029678103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c80047c0 msgr2=0x7f58c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58dad9d640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c80047c0 0x7f58c8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58dad9d640 1 -- 172.21.15.60:0/3029678103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8003520 con 0x7f58c8003970 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58d3fff640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c80047c0 0x7f58c8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58db59e640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58c8005180 0x7f58c800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.507+0000 7f58db59e640 1 -- 172.21.15.60:0/3029678103 reap_dead start 2023-11-18T20:43:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.508+0000 7f58dad9d640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58c8003970 0x7f58c8003d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f58cc001750 tx=0x7f58c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb31aa0f8664eac7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.508+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0052d0 con 0x7f58c8003970 2023-11-18T20:43:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.508+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc02e050 con 0x7f58c8003970 2023-11-18T20:43:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.508+0000 7f58da59c640 1 -- 172.21.15.60:0/3029678103 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc003830 con 0x7f58c8003970 2023-11-18T20:43:23.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.508+0000 7f58e11f0640 1 -- 172.21.15.60:0/3029678103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58c8003970 msgr2=0x7f58c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.508+0000 7f58e11f0640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58c8003970 0x7f58c8003d90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f58cc001750 tx=0x7f58c80057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.508+0000 7f58e11f0640 1 -- 172.21.15.60:0/3029678103 shutdown_connections 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.508+0000 7f58e11f0640 1 --2- 172.21.15.60:0/3029678103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58c8003970 0x7f58c8003d90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.508+0000 7f58e11f0640 1 -- 172.21.15.60:0/3029678103 >> 172.21.15.60:0/3029678103 conn(0x7f58dc06a4d0 msgr2=0x7f58dc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.508+0000 7f58e11f0640 1 -- 172.21.15.60:0/3029678103 shutdown_connections 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.509+0000 7f58e11f0640 1 -- 172.21.15.60:0/3029678103 wait complete. 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.510+0000 7f58e11f0640 1 Processor -- start 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.510+0000 7f58e11f0640 1 -- start start 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58e11f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c8003970 0x7f58dc143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58e11f0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58dc0c91f0 0x7f58dc1f0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58e11f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc0c9650 0x7f58dc1f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58e11f0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0c8dd0 con 0x7f58dc0c9650 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58e11f0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0c8150 con 0x7f58c8003970 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58e11f0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0c8430 con 0x7f58dc0c91f0 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58db59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc0c9650 0x7f58dc1f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58db59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc0c9650 0x7f58dc1f09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49916/0 (socket says 172.21.15.60:49916) 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58db59e640 1 -- 172.21.15.60:0/3985255271 learned_addr learned my addr 172.21.15.60:0/3985255271 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58dad9d640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c8003970 0x7f58dc143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58d3fff640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58dc0c91f0 0x7f58dc1f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58db59e640 1 -- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58dc0c91f0 msgr2=0x7f58dc1f0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58db59e640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58dc0c91f0 0x7f58dc1f0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58db59e640 1 -- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c8003970 msgr2=0x7f58dc143260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58db59e640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c8003970 0x7f58dc143260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58db59e640 1 -- 172.21.15.60:0/3985255271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc0c8710 con 0x7f58dc0c9650 2023-11-18T20:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58db59e640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc0c9650 0x7f58dc1f09d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f58d4002010 tx=0x7f58d400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58d37fe640 1 -- 172.21.15.60:0/3985255271 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400eb90 con 0x7f58dc0c9650 2023-11-18T20:43:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58d37fe640 1 -- 172.21.15.60:0/3985255271 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d400ecf0 con 0x7f58dc0c9650 2023-11-18T20:43:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58e11f0640 1 -- 172.21.15.60:0/3985255271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc0c89f0 con 0x7f58dc0c9650 2023-11-18T20:43:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58d37fe640 1 -- 172.21.15.60:0/3985255271 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4013660 con 0x7f58dc0c9650 2023-11-18T20:43:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.511+0000 7f58e11f0640 1 -- 172.21.15.60:0/3985255271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc0ca760 con 0x7f58dc0c9650 2023-11-18T20:43:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.513+0000 7f58e11f0640 1 -- 172.21.15.60:0/3985255271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc0c8f10 con 0x7f58dc0c9650 2023-11-18T20:43:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.513+0000 7f58d37fe640 1 -- 172.21.15.60:0/3985255271 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58d40137c0 con 0x7f58dc0c9650 2023-11-18T20:43:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.513+0000 7f58d37fe640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58bc06ae40 0x7f58bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.513+0000 7f58d37fe640 1 -- 172.21.15.60:0/3985255271 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f58d40898c0 con 0x7f58dc0c9650 2023-11-18T20:43:23.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.514+0000 7f58dad9d640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58bc06ae40 0x7f58bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.514+0000 7f58dad9d640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58bc06ae40 0x7f58bc06d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f58cc033000 tx=0x7f58cc009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:23.519 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.518+0000 7f58d37fe640 1 -- 172.21.15.60:0/3985255271 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58dc0c8f10 con 0x7f58dc0c9650 2023-11-18T20:43:23.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.733+0000 7f58e11f0640 1 -- 172.21.15.60:0/3985255271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f58dc0c8f10 con 0x7f58dc0c9650 2023-11-18T20:43:23.734 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-11-18T20:43:23.733+0000 7fe66ea9a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-18T20:43:23.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.735+0000 7f58d37fe640 1 -- 172.21.15.60:0/3985255271 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f58dc0c8f10 con 0x7f58dc0c9650 2023-11-18T20:43:23.736 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-18T20:43:23.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.737+0000 7f58e11f0640 1 -- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58bc06ae40 msgr2=0x7f58bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.737+0000 7f58e11f0640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58bc06ae40 0x7f58bc06d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f58cc033000 tx=0x7f58cc009030 comp rx=0 tx=0).stop 2023-11-18T20:43:23.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.737+0000 7f58e11f0640 1 -- 172.21.15.60:0/3985255271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc0c9650 msgr2=0x7f58dc1f09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.737+0000 7f58e11f0640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc0c9650 0x7f58dc1f09d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f58d4002010 tx=0x7f58d400e530 comp rx=0 tx=0).stop 2023-11-18T20:43:23.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.737+0000 7f58e11f0640 1 -- 172.21.15.60:0/3985255271 shutdown_connections 2023-11-18T20:43:23.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.737+0000 7f58e11f0640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58bc06ae40 0x7f58bc06d300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.737+0000 7f58e11f0640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58dc0c9650 0x7f58dc1f09d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.737+0000 7f58e11f0640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58dc0c91f0 0x7f58dc1f0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.737+0000 7f58e11f0640 1 --2- 172.21.15.60:0/3985255271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c8003970 0x7f58dc143260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.737+0000 7f58e11f0640 1 -- 172.21.15.60:0/3985255271 >> 172.21.15.60:0/3985255271 conn(0x7f58dc06a4d0 msgr2=0x7f58dc1590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:23.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.738+0000 7f58e11f0640 1 -- 172.21.15.60:0/3985255271 shutdown_connections 2023-11-18T20:43:23.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.738+0000 7f58e11f0640 1 -- 172.21.15.60:0/3985255271 wait complete. 2023-11-18T20:43:23.743 INFO:tasks.ceph.mds.b.smithi100.stderr: -2698> 2023-11-18T20:42:42.091+0000 7f7b09c15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-18T20:43:23.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-18T20:43:23.772 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2023-11-18T20:43:23.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.857+0000 7f67d4cc9640 1 Processor -- start 2023-11-18T20:43:23.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.857+0000 7f67d4cc9640 1 -- start start 2023-11-18T20:43:23.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.858+0000 7f67d4cc9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67d00d31c0 0x7f67d00d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.858+0000 7f67d4cc9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d00d2e80 con 0x7f67d00ce640 2023-11-18T20:43:23.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.858+0000 7f67d4cc9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d00d28a0 con 0x7f67d00d19b0 2023-11-18T20:43:23.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.858+0000 7f67d4cc9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d00d3ae0 con 0x7f67d00d31c0 2023-11-18T20:43:23.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.858+0000 7f67ced9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f67d00ce640 0x7f67d00b6470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45378/0 (socket says 172.21.15.60:45378) 2023-11-18T20:43:23.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.858+0000 7f67ced9d640 1 -- 172.21.15.60:0/4107606292 learned_addr learned my addr 172.21.15.60:0/4107606292 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:23.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67ce59c640 1 --2- 172.21.15.60:0/4107606292 >> v2:172.21.15.100:3300/0 conn(0x7f67d00d31c0 0x7f67d00d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149884843 0 0) 0x7f67d00d2e80 con 0x7f67d00ce640 2023-11-18T20:43:23.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67bc003520 con 0x7f67d00ce640 2023-11-18T20:43:23.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963019189 0 0) 0x7f67d00d28a0 con 0x7f67d00d19b0 2023-11-18T20:43:23.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d00d2e80 con 0x7f67d00d19b0 2023-11-18T20:43:23.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3073859299 0 0) 0x7f67bc003520 con 0x7f67d00ce640 2023-11-18T20:43:23.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 >> v2:172.21.15.100:3300/0 conn(0x7f67d00d31c0 msgr2=0x7f67d00d35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:23.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 --2- 172.21.15.60:0/4107606292 >> v2:172.21.15.100:3300/0 conn(0x7f67d00d31c0 0x7f67d00d35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 >> v1:172.21.15.100:6789/0 conn(0x7f67d00d19b0 legacy=0x7f67d00cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d00cdc50 con 0x7f67d00ce640 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f67c00019f0 con 0x7f67d00ce640 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 >> v1:172.21.15.60:6789/0 conn(0x7f67d00ce640 legacy=0x7f67d00b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc003b00 0x7f67bc003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0047c0 0x7f67bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc005180 0x7f67bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00d3ae0 con 0x7f67bc003b00 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc00c000 con 0x7f67bc005180 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc00c1c0 con 0x7f67bc0047c0 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67ce59c640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc003b00 0x7f67bc003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67ced9d640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc005180 0x7f67bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.859+0000 7f67cdd9b640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0047c0 0x7f67bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.860+0000 7f67ce59c640 1 -- 172.21.15.60:0/4107606292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0047c0 msgr2=0x7f67bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.860+0000 7f67ce59c640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0047c0 0x7f67bc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.860+0000 7f67ce59c640 1 -- 172.21.15.60:0/4107606292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc005180 msgr2=0x7f67bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.860+0000 7f67ce59c640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc005180 0x7f67bc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.860+0000 7f67ce59c640 1 -- 172.21.15.60:0/4107606292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc003520 con 0x7f67bc003b00 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.860+0000 7f67ced9d640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc005180 0x7f67bc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.860+0000 7f67ced9d640 1 -- 172.21.15.60:0/4107606292 reap_dead start 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.860+0000 7f67ce59c640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc003b00 0x7f67bc003f00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009ac0 tx=0x7f67bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c90120e3ae46bb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.861+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c802c6b0 con 0x7f67bc003b00 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.861+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c802c810 con 0x7f67bc003b00 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.861+0000 7f67cd59a640 1 -- 172.21.15.60:0/4107606292 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c802cba0 con 0x7f67bc003b00 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.861+0000 7f67d4cc9640 1 -- 172.21.15.60:0/4107606292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc003b00 msgr2=0x7f67bc003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.861+0000 7f67d4cc9640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc003b00 0x7f67bc003f00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009ac0 tx=0x7f67bc0057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.861+0000 7f67d4cc9640 1 -- 172.21.15.60:0/4107606292 shutdown_connections 2023-11-18T20:43:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.861+0000 7f67d4cc9640 1 --2- 172.21.15.60:0/4107606292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc003b00 0x7f67bc003f00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.861+0000 7f67d4cc9640 1 -- 172.21.15.60:0/4107606292 >> 172.21.15.60:0/4107606292 conn(0x7f67d0072020 msgr2=0x7f67d0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.861+0000 7f67d4cc9640 1 -- 172.21.15.60:0/4107606292 shutdown_connections 2023-11-18T20:43:23.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.861+0000 7f67d4cc9640 1 -- 172.21.15.60:0/4107606292 wait complete. 2023-11-18T20:43:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.862+0000 7f67d4cc9640 1 Processor -- start 2023-11-18T20:43:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67d4cc9640 1 -- start start 2023-11-18T20:43:23.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67d4cc9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc003b00 0x7f67d00c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67d4cc9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d01501d0 0x7f67d01f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67d4cc9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d00cad80 0x7f67d00cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67d4cc9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00cb740 con 0x7f67d01501d0 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67d4cc9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00c8bc0 con 0x7f67d00cad80 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67d4cc9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d0150650 con 0x7f67bc003b00 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67cdd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d01501d0 0x7f67d01f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67cdd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d01501d0 0x7f67d01f5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49922/0 (socket says 172.21.15.60:49922) 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67cdd9b640 1 -- 172.21.15.60:0/966775167 learned_addr learned my addr 172.21.15.60:0/966775167 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67ce59c640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc003b00 0x7f67d00c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.863+0000 7f67ced9d640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d00cad80 0x7f67d00cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67cdd9b640 1 -- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc003b00 msgr2=0x7f67d00c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67cdd9b640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc003b00 0x7f67d00c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67cdd9b640 1 -- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d00cad80 msgr2=0x7f67d00cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67cdd9b640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d00cad80 0x7f67d00cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67cdd9b640 1 -- 172.21.15.60:0/966775167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d00c7fe0 con 0x7f67d01501d0 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67cdd9b640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d01501d0 0x7f67d01f5500 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67b800f7a0 tx=0x7f67b800fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67c6ffd640 1 -- 172.21.15.60:0/966775167 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80050d0 con 0x7f67d01501d0 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67c6ffd640 1 -- 172.21.15.60:0/966775167 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b8005250 con 0x7f67d01501d0 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67c6ffd640 1 -- 172.21.15.60:0/966775167 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b800d730 con 0x7f67d01501d0 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67d4cc9640 1 -- 172.21.15.60:0/966775167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d00c82c0 con 0x7f67d01501d0 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.864+0000 7f67d4cc9640 1 -- 172.21.15.60:0/966775167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d00c8880 con 0x7f67d01501d0 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.865+0000 7f67d4cc9640 1 -- 172.21.15.60:0/966775167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d0069400 con 0x7f67d01501d0 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.866+0000 7f67c6ffd640 1 -- 172.21.15.60:0/966775167 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f67b800d890 con 0x7f67d01501d0 2023-11-18T20:43:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.866+0000 7f67c6ffd640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f67b006ae40 0x7f67b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.866+0000 7f67c6ffd640 1 -- 172.21.15.60:0/966775167 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f67b8089520 con 0x7f67d01501d0 2023-11-18T20:43:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.866+0000 7f67ce59c640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f67b006ae40 0x7f67b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.867+0000 7f67ce59c640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f67b006ae40 0x7f67b006d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f67c8036000 tx=0x7f67c8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:23.870+0000 7f67c6ffd640 1 -- 172.21.15.60:0/966775167 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67b8054590 con 0x7f67d01501d0 2023-11-18T20:43:24.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.082+0000 7f67d4cc9640 1 -- 172.21.15.60:0/966775167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f67d006a4d0 con 0x7f67d01501d0 2023-11-18T20:43:24.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.736+0000 7f67c6ffd640 1 -- 172.21.15.60:0/966775167 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f67b8057bb0 con 0x7f67d01501d0 2023-11-18T20:43:24.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 -- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f67b006ae40 msgr2=0x7f67b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:24.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f67b006ae40 0x7f67b006d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f67c8036000 tx=0x7f67c8030000 comp rx=0 tx=0).stop 2023-11-18T20:43:24.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 -- 172.21.15.60:0/966775167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d01501d0 msgr2=0x7f67d01f5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:24.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d01501d0 0x7f67d01f5500 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67b800f7a0 tx=0x7f67b800fba0 comp rx=0 tx=0).stop 2023-11-18T20:43:24.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 -- 172.21.15.60:0/966775167 shutdown_connections 2023-11-18T20:43:24.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f67b006ae40 0x7f67b006d300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:24.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67d00cad80 0x7f67d00cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:24.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67d01501d0 0x7f67d01f5500 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:24.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 --2- 172.21.15.60:0/966775167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc003b00 0x7f67d00c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:24.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 -- 172.21.15.60:0/966775167 >> 172.21.15.60:0/966775167 conn(0x7f67d0072020 msgr2=0x7f67d00b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:24.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 -- 172.21.15.60:0/966775167 shutdown_connections 2023-11-18T20:43:24.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.739+0000 7f67d4cc9640 1 -- 172.21.15.60:0/966775167 wait complete. 2023-11-18T20:43:24.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-18T20:43:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.869+0000 7f3de9d05640 1 Processor -- start 2023-11-18T20:43:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.869+0000 7f3de9d05640 1 -- start start 2023-11-18T20:43:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.870+0000 7f3de9d05640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3de40d2910 0x7f3de40d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.870+0000 7f3de9d05640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3de40d3730 0x7f3de40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.870+0000 7f3de9d05640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de40b2b10 con 0x7f3de415d330 2023-11-18T20:43:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.870+0000 7f3de9d05640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de40d3bc0 con 0x7f3de40d3730 2023-11-18T20:43:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.870+0000 7f3de9d05640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de40d1590 con 0x7f3de40d2910 2023-11-18T20:43:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.870+0000 7f3de3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3de40d3730 0x7f3de40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.870+0000 7f3de37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3de415d330 0x7f3de40b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45392/0 (socket says 172.21.15.60:45392) 2023-11-18T20:43:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.870+0000 7f3de37fe640 1 -- 172.21.15.60:0/1628450639 learned_addr learned my addr 172.21.15.60:0/1628450639 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.871+0000 7f3de2ffd640 1 --2- 172.21.15.60:0/1628450639 >> v2:172.21.15.100:3300/0 conn(0x7f3de40d2910 0x7f3de40d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.871+0000 7f3de3fff640 1 -- 172.21.15.60:0/1628450639 >> v2:172.21.15.100:3300/0 conn(0x7f3de40d2910 msgr2=0x7f3de40d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.871+0000 7f3de3fff640 1 --2- 172.21.15.60:0/1628450639 >> v2:172.21.15.100:3300/0 conn(0x7f3de40d2910 0x7f3de40d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.871+0000 7f3de3fff640 1 -- 172.21.15.60:0/1628450639 >> v1:172.21.15.60:6789/0 conn(0x7f3de415d330 legacy=0x7f3de40b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.871+0000 7f3de27fc640 1 -- 172.21.15.60:0/1628450639 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533696997 0 0) 0x7f3de40b2b10 con 0x7f3de415d330 2023-11-18T20:43:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.871+0000 7f3de3fff640 1 -- 172.21.15.60:0/1628450639 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40d1870 con 0x7f3de40d3730 2023-11-18T20:43:24.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de3fff640 1 --2- 172.21.15.60:0/1628450639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d3730 0x7f3de40b6390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3de40d1540 tx=0x7f3ddc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c061c759a617ae server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de27fc640 1 -- 172.21.15.60:0/1628450639 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc02cbf0 con 0x7f3de40d3730 2023-11-18T20:43:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de27fc640 1 -- 172.21.15.60:0/1628450639 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc02cd50 con 0x7f3de40d3730 2023-11-18T20:43:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de27fc640 1 -- 172.21.15.60:0/1628450639 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc034660 con 0x7f3de40d3730 2023-11-18T20:43:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de9d05640 1 -- 172.21.15.60:0/1628450639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d3730 msgr2=0x7f3de40b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de9d05640 1 --2- 172.21.15.60:0/1628450639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d3730 0x7f3de40b6390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3de40d1540 tx=0x7f3ddc02be40 comp rx=0 tx=0).stop 2023-11-18T20:43:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de9d05640 1 -- 172.21.15.60:0/1628450639 shutdown_connections 2023-11-18T20:43:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de9d05640 1 --2- 172.21.15.60:0/1628450639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d3730 0x7f3de40b6390 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de9d05640 1 --2- 172.21.15.60:0/1628450639 >> v2:172.21.15.100:3300/0 conn(0x7f3de40d2910 0x7f3de40d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de9d05640 1 -- 172.21.15.60:0/1628450639 >> 172.21.15.60:0/1628450639 conn(0x7f3de4072020 msgr2=0x7f3de4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de9d05640 1 -- 172.21.15.60:0/1628450639 shutdown_connections 2023-11-18T20:43:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.872+0000 7f3de9d05640 1 -- 172.21.15.60:0/1628450639 wait complete. 2023-11-18T20:43:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.874+0000 7f3de9d05640 1 Processor -- start 2023-11-18T20:43:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.874+0000 7f3de9d05640 1 -- start start 2023-11-18T20:43:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.874+0000 7f3de9d05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d2910 0x7f3de4153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.874+0000 7f3de9d05640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de40d3730 0x7f3de40c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.874+0000 7f3de9d05640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3de415d330 0x7f3de40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.874+0000 7f3de9d05640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40c8760 con 0x7f3de40d2910 2023-11-18T20:43:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.874+0000 7f3de9d05640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40c8a40 con 0x7f3de40d3730 2023-11-18T20:43:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de9d05640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40c8d20 con 0x7f3de415d330 2023-11-18T20:43:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d2910 0x7f3de4153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d2910 0x7f3de4153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49942/0 (socket says 172.21.15.60:49942) 2023-11-18T20:43:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de37fe640 1 -- 172.21.15.60:0/390473158 learned_addr learned my addr 172.21.15.60:0/390473158 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de2ffd640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de40d3730 0x7f3de40c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de3fff640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3de415d330 0x7f3de40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de37fe640 1 -- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3de415d330 msgr2=0x7f3de40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de37fe640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3de415d330 0x7f3de40cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de37fe640 1 -- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de40d3730 msgr2=0x7f3de40c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de37fe640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de40d3730 0x7f3de40c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de37fe640 1 -- 172.21.15.60:0/390473158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc009a70 con 0x7f3de40d2910 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de3fff640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3de415d330 0x7f3de40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de37fe640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d2910 0x7f3de4153be0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400c430 tx=0x7f3dd400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3dcbfff640 1 -- 172.21.15.60:0/390473158 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40074e0 con 0x7f3de40d2910 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3dcbfff640 1 -- 172.21.15.60:0/390473158 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd4007660 con 0x7f3de40d2910 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de9d05640 1 -- 172.21.15.60:0/390473158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de40c9000 con 0x7f3de40d2910 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3dcbfff640 1 -- 172.21.15.60:0/390473158 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4004a20 con 0x7f3de40d2910 2023-11-18T20:43:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.875+0000 7f3de9d05640 1 -- 172.21.15.60:0/390473158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de40c8370 con 0x7f3de40d2910 2023-11-18T20:43:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.876+0000 7f3de9d05640 1 -- 172.21.15.60:0/390473158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de4069400 con 0x7f3de40d2910 2023-11-18T20:43:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.877+0000 7f3dcbfff640 1 -- 172.21.15.60:0/390473158 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3dd4004b80 con 0x7f3de40d2910 2023-11-18T20:43:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.877+0000 7f3dcbfff640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3dd006ae40 0x7f3dd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.877+0000 7f3dcbfff640 1 -- 172.21.15.60:0/390473158 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f3dd40894e0 con 0x7f3de40d2910 2023-11-18T20:43:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.878+0000 7f3de2ffd640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3dd006ae40 0x7f3dd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.878+0000 7f3de2ffd640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3dd006ae40 0x7f3dd006d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3de40c7d80 tx=0x7f3dd8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:24.883+0000 7f3dcbfff640 1 -- 172.21.15.60:0/390473158 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3dd40544c0 con 0x7f3de40d2910 2023-11-18T20:43:25.060 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.059+0000 7f3de9d05640 1 -- 172.21.15.60:0/390473158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3de4159e60 con 0x7f3de40d2910 2023-11-18T20:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.738+0000 7f3dcbfff640 1 -- 172.21.15.60:0/390473158 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3dd4057ae0 con 0x7f3de40d2910 2023-11-18T20:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-11-18T20:43:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 -- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3dd006ae40 msgr2=0x7f3dd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:25.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3dd006ae40 0x7f3dd006d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3de40c7d80 tx=0x7f3dd8008040 comp rx=0 tx=0).stop 2023-11-18T20:43:25.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 -- 172.21.15.60:0/390473158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d2910 msgr2=0x7f3de4153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:25.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d2910 0x7f3de4153be0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400c430 tx=0x7f3dd400c8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 -- 172.21.15.60:0/390473158 shutdown_connections 2023-11-18T20:43:25.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3dd006ae40 0x7f3dd006d300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3de415d330 0x7f3de40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3de40d3730 0x7f3de40c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 --2- 172.21.15.60:0/390473158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3de40d2910 0x7f3de4153be0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 -- 172.21.15.60:0/390473158 >> 172.21.15.60:0/390473158 conn(0x7f3de4072020 msgr2=0x7f3de4072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:25.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 -- 172.21.15.60:0/390473158 shutdown_connections 2023-11-18T20:43:25.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.741+0000 7f3de9d05640 1 -- 172.21.15.60:0/390473158 wait complete. 2023-11-18T20:43:25.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-18T20:43:25.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.875+0000 7ff92007c640 1 Processor -- start 2023-11-18T20:43:25.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.875+0000 7ff92007c640 1 -- start start 2023-11-18T20:43:25.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.876+0000 7ff92007c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff91815b130 0x7ff9180d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.876+0000 7ff92007c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9180b3f10 con 0x7ff9180cf320 2023-11-18T20:43:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.876+0000 7ff92007c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9180d0a50 con 0x7ff91815f760 2023-11-18T20:43:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.876+0000 7ff92007c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91815fd20 con 0x7ff91815b130 2023-11-18T20:43:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.876+0000 7ff91e619640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff9180cf320 0x7ff9180cd310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45406/0 (socket says 172.21.15.60:45406) 2023-11-18T20:43:25.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.876+0000 7ff91e619640 1 -- 172.21.15.60:0/309628202 learned_addr learned my addr 172.21.15.60:0/309628202 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91de18640 1 --2- 172.21.15.60:0/309628202 >> v2:172.21.15.100:3300/0 conn(0x7ff91815b130 0x7ff9180d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298923403 0 0) 0x7ff9180b3f10 con 0x7ff9180cf320 2023-11-18T20:43:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff900003520 con 0x7ff9180cf320 2023-11-18T20:43:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1118435582 0 0) 0x7ff900003520 con 0x7ff9180cf320 2023-11-18T20:43:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 >> v2:172.21.15.100:3300/0 conn(0x7ff91815b130 msgr2=0x7ff9180d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:25.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 --2- 172.21.15.60:0/309628202 >> v2:172.21.15.100:3300/0 conn(0x7ff91815b130 0x7ff9180d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 >> v1:172.21.15.100:6789/0 conn(0x7ff91815f760 legacy=0x7ff9180cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:25.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180d0d30 con 0x7ff9180cf320 2023-11-18T20:43:25.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7ff910001670 con 0x7ff9180cf320 2023-11-18T20:43:25.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 >> v1:172.21.15.60:6789/0 conn(0x7ff9180cf320 legacy=0x7ff9180cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:25.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff900003970 0x7ff900003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:25.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9000047c0 0x7ff900004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:25.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff90000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:25.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91815fd20 con 0x7ff900005180 2023-11-18T20:43:25.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90000c120 con 0x7ff9000047c0 2023-11-18T20:43:25.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90000c2e0 con 0x7ff900003970 2023-11-18T20:43:25.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91e619640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff90000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:25.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91d617640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9000047c0 0x7ff900004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:25.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.877+0000 7ff91de18640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff900003970 0x7ff900003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:25.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.878+0000 7ff91e619640 1 -- 172.21.15.60:0/309628202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff900003970 msgr2=0x7ff900003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:25.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.878+0000 7ff91e619640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff900003970 0x7ff900003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.878+0000 7ff91e619640 1 -- 172.21.15.60:0/309628202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9000047c0 msgr2=0x7ff900004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:25.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.878+0000 7ff91e619640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9000047c0 0x7ff900004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.878+0000 7ff91e619640 1 -- 172.21.15.60:0/309628202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff900003520 con 0x7ff900005180 2023-11-18T20:43:25.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.878+0000 7ff91e619640 1 -- 172.21.15.60:0/309628202 reap_dead start 2023-11-18T20:43:25.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.878+0000 7ff91e619640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff90000b980 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff91000c190 tx=0x7ff9000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04ea0c333010367 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:25.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.878+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910005ad0 con 0x7ff900005180 2023-11-18T20:43:25.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.878+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff910005c70 con 0x7ff900005180 2023-11-18T20:43:25.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.878+0000 7ff91ce16640 1 -- 172.21.15.60:0/309628202 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91002c8a0 con 0x7ff900005180 2023-11-18T20:43:25.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.879+0000 7ff92007c640 1 -- 172.21.15.60:0/309628202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 msgr2=0x7ff90000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:25.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.879+0000 7ff92007c640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff90000b980 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff91000c190 tx=0x7ff9000057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.879+0000 7ff92007c640 1 -- 172.21.15.60:0/309628202 shutdown_connections 2023-11-18T20:43:25.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.879+0000 7ff92007c640 1 --2- 172.21.15.60:0/309628202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff90000b980 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.879+0000 7ff92007c640 1 -- 172.21.15.60:0/309628202 >> 172.21.15.60:0/309628202 conn(0x7ff9180b2c10 msgr2=0x7ff9180b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:25.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.879+0000 7ff92007c640 1 -- 172.21.15.60:0/309628202 shutdown_connections 2023-11-18T20:43:25.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.879+0000 7ff92007c640 1 -- 172.21.15.60:0/309628202 wait complete. 2023-11-18T20:43:25.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.880+0000 7ff92007c640 1 Processor -- start 2023-11-18T20:43:25.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.880+0000 7ff92007c640 1 -- start start 2023-11-18T20:43:25.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff92007c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff9180c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:25.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff92007c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff91815e530 0x7ff9180c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:25.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff92007c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180c4960 0x7ff9180b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:25.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff92007c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180c5ec0 con 0x7ff900005180 2023-11-18T20:43:25.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff92007c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180c52b0 con 0x7ff9180c4960 2023-11-18T20:43:25.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff92007c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180c5590 con 0x7ff91815e530 2023-11-18T20:43:25.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff91de18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff9180c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:25.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff91de18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff9180c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49964/0 (socket says 172.21.15.60:49964) 2023-11-18T20:43:25.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff91de18640 1 -- 172.21.15.60:0/3242721030 learned_addr learned my addr 172.21.15.60:0/3242721030 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:25.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff91d617640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff91815e530 0x7ff9180c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:25.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.881+0000 7ff91e619640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180c4960 0x7ff9180b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:25.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff91de18640 1 -- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff91815e530 msgr2=0x7ff9180c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:25.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff91de18640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff91815e530 0x7ff9180c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff91de18640 1 -- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180c4960 msgr2=0x7ff9180b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:25.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff91de18640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180c4960 0x7ff9180b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:25.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff91de18640 1 -- 172.21.15.60:0/3242721030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180c5870 con 0x7ff900005180 2023-11-18T20:43:25.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff91d617640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff91815e530 0x7ff9180c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:25.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff91de18640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff9180c6350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff908009f90 tx=0x7ff90800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:25.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff9167fc640 1 -- 172.21.15.60:0/3242721030 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90800e040 con 0x7ff900005180 2023-11-18T20:43:25.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff9167fc640 1 -- 172.21.15.60:0/3242721030 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90800f040 con 0x7ff900005180 2023-11-18T20:43:25.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff92007c640 1 -- 172.21.15.60:0/3242721030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9180c5b50 con 0x7ff900005180 2023-11-18T20:43:25.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff9167fc640 1 -- 172.21.15.60:0/3242721030 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908015870 con 0x7ff900005180 2023-11-18T20:43:25.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.882+0000 7ff92007c640 1 -- 172.21.15.60:0/3242721030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9180c4e10 con 0x7ff900005180 2023-11-18T20:43:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.883+0000 7ff92007c640 1 -- 172.21.15.60:0/3242721030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9180c4580 con 0x7ff900005180 2023-11-18T20:43:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.883+0000 7ff9167fc640 1 -- 172.21.15.60:0/3242721030 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9080159d0 con 0x7ff900005180 2023-11-18T20:43:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.884+0000 7ff9167fc640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff90406ae40 0x7ff90406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.884+0000 7ff9167fc640 1 -- 172.21.15.60:0/3242721030 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5473+0+0 (secure 0 0 0) 0x7ff90808a3b0 con 0x7ff900005180 2023-11-18T20:43:25.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.884+0000 7ff91d617640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff90406ae40 0x7ff90406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:25.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.885+0000 7ff91d617640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff90406ae40 0x7ff90406d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff9180c8000 tx=0x7ff90c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:25.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:25.888+0000 7ff9167fc640 1 -- 172.21.15.60:0/3242721030 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9080554b0 con 0x7ff900005180 2023-11-18T20:43:26.068 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.067+0000 7ff92007c640 1 -- 172.21.15.60:0/3242721030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff918067fc0 con 0x7ff900005180 2023-11-18T20:43:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.739+0000 7ff9167fc640 1 -- 172.21.15.60:0/3242721030 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff90800fea0 con 0x7ff900005180 2023-11-18T20:43:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-11-18T20:43:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.745+0000 7ff92007c640 1 -- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff90406ae40 msgr2=0x7ff90406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.745+0000 7ff92007c640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff90406ae40 0x7ff90406d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff9180c8000 tx=0x7ff90c009040 comp rx=0 tx=0).stop 2023-11-18T20:43:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.745+0000 7ff92007c640 1 -- 172.21.15.60:0/3242721030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 msgr2=0x7ff9180c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.745+0000 7ff92007c640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff9180c6350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff908009f90 tx=0x7ff90800f580 comp rx=0 tx=0).stop 2023-11-18T20:43:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.746+0000 7ff92007c640 1 -- 172.21.15.60:0/3242721030 shutdown_connections 2023-11-18T20:43:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.746+0000 7ff92007c640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff90406ae40 0x7ff90406d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.746+0000 7ff92007c640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9180c4960 0x7ff9180b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.746+0000 7ff92007c640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff91815e530 0x7ff9180c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.746+0000 7ff92007c640 1 --2- 172.21.15.60:0/3242721030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff900005180 0x7ff9180c6350 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.746+0000 7ff92007c640 1 -- 172.21.15.60:0/3242721030 >> 172.21.15.60:0/3242721030 conn(0x7ff9180b2c10 msgr2=0x7ff9180ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.746+0000 7ff92007c640 1 -- 172.21.15.60:0/3242721030 shutdown_connections 2023-11-18T20:43:26.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.746+0000 7ff92007c640 1 -- 172.21.15.60:0/3242721030 wait complete. 2023-11-18T20:43:26.765 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-18T20:43:26.765 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-18T20:43:26.765 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-18T20:43:26.766 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:43:26.766 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-18T20:43:26.766 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-18T20:43:26.766 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-18T20:43:26.766 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:43:26.766 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-18T20:43:26.766 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-18T20:43:26.767 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-18T20:43:26.767 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:43:26.767 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-18T20:43:26.767 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-18T20:43:26.767 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-18T20:43:26.767 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:43:26.776 INFO:tasks.ceph.mds.a:Stopped 2023-11-18T20:43:26.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-18T20:43:26.777 INFO:tasks.ceph.mds.c:Stopped 2023-11-18T20:43:26.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-18T20:43:26.779 INFO:tasks.ceph.mds.b:Stopped 2023-11-18T20:43:26.779 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-18T20:43:26.780 INFO:tasks.ceph.mds.d:Stopped 2023-11-18T20:43:26.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-18T20:43:26.820 INFO:tasks.ceph.mds.a:Started 2023-11-18T20:43:26.821 DEBUG:teuthology.parallel:result is None 2023-11-18T20:43:26.822 INFO:tasks.ceph.mds.b:Started 2023-11-18T20:43:26.822 INFO:tasks.ceph.mds.c:Started 2023-11-18T20:43:26.822 DEBUG:teuthology.parallel:result is None 2023-11-18T20:43:26.822 DEBUG:teuthology.parallel:result is None 2023-11-18T20:43:26.823 INFO:tasks.ceph.mds.d:Started 2023-11-18T20:43:26.823 DEBUG:teuthology.parallel:result is None 2023-11-18T20:43:26.823 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-18T20:43:26.901 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2023-11-18T20:43:26.901 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2023-11-18T20:43:26.908 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-11-18T20:43:26.909 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-11-18T20:43:26.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.938+0000 7ffacb0da640 1 Processor -- start 2023-11-18T20:43:26.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.938+0000 7ffacb0da640 1 -- start start 2023-11-18T20:43:26.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.939+0000 7ffacb0da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac415d330 0x7ffac40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:26.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.939+0000 7ffacb0da640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffac40d2910 0x7ffac40d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:26.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.939+0000 7ffacb0da640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffac40d2d90 0x7ffac40d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:26.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.939+0000 7ffacb0da640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac40b2b10 con 0x7ffac40d2d90 2023-11-18T20:43:26.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.939+0000 7ffacb0da640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac40d1590 con 0x7ffac40d2910 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.939+0000 7ffacb0da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40d1870 con 0x7ffac415d330 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.939+0000 7ffac9677640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffac40d2d90 0x7ffac40d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.939+0000 7ffac9677640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffac40d2d90 0x7ffac40d3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49994/0 (socket says 172.21.15.60:49994) 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.939+0000 7ffac9677640 1 -- 172.21.15.60:0/4275960451 learned_addr learned my addr 172.21.15.60:0/4275960451 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.940+0000 7ffac8e76640 1 --2- 172.21.15.60:0/4275960451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac415d330 0x7ffac40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.940+0000 7ffac3fff640 1 --2- 172.21.15.60:0/4275960451 >> v2:172.21.15.100:3300/0 conn(0x7ffac40d2910 0x7ffac40d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.940+0000 7ffac9677640 1 -- 172.21.15.60:0/4275960451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac415d330 msgr2=0x7ffac40b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.940+0000 7ffac9677640 1 --2- 172.21.15.60:0/4275960451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac415d330 0x7ffac40b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.940+0000 7ffac9677640 1 -- 172.21.15.60:0/4275960451 >> v2:172.21.15.100:3300/0 conn(0x7ffac40d2910 msgr2=0x7ffac40d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.940+0000 7ffac9677640 1 --2- 172.21.15.60:0/4275960451 >> v2:172.21.15.100:3300/0 conn(0x7ffac40d2910 0x7ffac40d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.940+0000 7ffac9677640 1 -- 172.21.15.60:0/4275960451 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac40d1b50 con 0x7ffac40d2d90 2023-11-18T20:43:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffac9677640 1 --2- 172.21.15.60:0/4275960451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac40d2d90 0x7ffac40d3730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffac40d4360 tx=0x7ffabc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24145186d3980caf server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffac37fe640 1 -- 172.21.15.60:0/4275960451 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc02c9e0 con 0x7ffac40d2d90 2023-11-18T20:43:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffac37fe640 1 -- 172.21.15.60:0/4275960451 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc02cb40 con 0x7ffac40d2d90 2023-11-18T20:43:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffac37fe640 1 -- 172.21.15.60:0/4275960451 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0343f0 con 0x7ffac40d2d90 2023-11-18T20:43:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffacb0da640 1 -- 172.21.15.60:0/4275960451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac40d2d90 msgr2=0x7ffac40d3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffacb0da640 1 --2- 172.21.15.60:0/4275960451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac40d2d90 0x7ffac40d3730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffac40d4360 tx=0x7ffabc02be40 comp rx=0 tx=0).stop 2023-11-18T20:43:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffacb0da640 1 -- 172.21.15.60:0/4275960451 shutdown_connections 2023-11-18T20:43:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffacb0da640 1 --2- 172.21.15.60:0/4275960451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac40d2d90 0x7ffac40d3730 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffacb0da640 1 --2- 172.21.15.60:0/4275960451 >> v2:172.21.15.100:3300/0 conn(0x7ffac40d2910 0x7ffac40d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffacb0da640 1 --2- 172.21.15.60:0/4275960451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac415d330 0x7ffac40b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffacb0da640 1 -- 172.21.15.60:0/4275960451 >> 172.21.15.60:0/4275960451 conn(0x7ffac4072020 msgr2=0x7ffac4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffacb0da640 1 -- 172.21.15.60:0/4275960451 shutdown_connections 2023-11-18T20:43:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.941+0000 7ffacb0da640 1 -- 172.21.15.60:0/4275960451 wait complete. 2023-11-18T20:43:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.943+0000 7ffacb0da640 1 Processor -- start 2023-11-18T20:43:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.943+0000 7ffacb0da640 1 -- start start 2023-11-18T20:43:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.943+0000 7ffacb0da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac40d2910 0x7ffac40ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.943+0000 7ffacb0da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac40d2d90 0x7ffac41f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.943+0000 7ffacb0da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac415d330 0x7ffac41f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.943+0000 7ffacb0da640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40cc0e0 con 0x7ffac40d2910 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.943+0000 7ffacb0da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac41f2e70 con 0x7ffac415d330 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.943+0000 7ffacb0da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac41f6b80 con 0x7ffac40d2d90 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac3fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac40d2d90 0x7ffac41f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac8e76640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac40d2910 0x7ffac40ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac9677640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac415d330 0x7ffac41f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac9677640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac415d330 0x7ffac41f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:48998/0 (socket says 172.21.15.60:48998) 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac9677640 1 -- 172.21.15.60:0/437680206 learned_addr learned my addr 172.21.15.60:0/437680206 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac3fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac40d2d90 0x7ffac41f6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:46006/0 (socket says 172.21.15.60:46006) 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac8e76640 1 -- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac40d2d90 msgr2=0x7ffac41f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac8e76640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac40d2d90 0x7ffac41f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac8e76640 1 -- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac415d330 msgr2=0x7ffac41f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac8e76640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac415d330 0x7ffac41f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac8e76640 1 -- 172.21.15.60:0/437680206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc009b30 con 0x7ffac40d2910 2023-11-18T20:43:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac9677640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac415d330 0x7ffac41f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac8e76640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac40d2910 0x7ffac40ccc10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffab800e8a0 tx=0x7ffab800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac17fa640 1 -- 172.21.15.60:0/437680206 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8009e60 con 0x7ffac40d2910 2023-11-18T20:43:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac17fa640 1 -- 172.21.15.60:0/437680206 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab800c940 con 0x7ffac40d2910 2023-11-18T20:43:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffacb0da640 1 -- 172.21.15.60:0/437680206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac41f6e60 con 0x7ffac40d2910 2023-11-18T20:43:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.944+0000 7ffac17fa640 1 -- 172.21.15.60:0/437680206 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab800ccd0 con 0x7ffac40d2910 2023-11-18T20:43:26.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.945+0000 7ffacb0da640 1 -- 172.21.15.60:0/437680206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac41f6890 con 0x7ffac40d2910 2023-11-18T20:43:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.946+0000 7ffacb0da640 1 -- 172.21.15.60:0/437680206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac4069400 con 0x7ffac40d2910 2023-11-18T20:43:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.946+0000 7ffac17fa640 1 -- 172.21.15.60:0/437680206 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffab8013470 con 0x7ffac40d2910 2023-11-18T20:43:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.946+0000 7ffac17fa640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ffaa806ae40 0x7ffaa806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:26.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.946+0000 7ffac17fa640 1 -- 172.21.15.60:0/437680206 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffab8089320 con 0x7ffac40d2910 2023-11-18T20:43:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.947+0000 7ffac3fff640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ffaa806ae40 0x7ffaa806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.947+0000 7ffac3fff640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ffaa806ae40 0x7ffaa806d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffac40ca490 tx=0x7ffab0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:26.951+0000 7ffac17fa640 1 -- 172.21.15.60:0/437680206 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffab80546e0 con 0x7ffac40d2910 2023-11-18T20:43:27.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.140+0000 7ffacb0da640 1 -- 172.21.15.60:0/437680206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffac40cdae0 con 0x7ffac40d2910 2023-11-18T20:43:27.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.761+0000 7ffac17fa640 1 -- 172.21.15.60:0/437680206 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffab8057d00 con 0x7ffac40d2910 2023-11-18T20:43:27.763 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-11-18T20:43:27.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.764+0000 7ffacb0da640 1 -- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ffaa806ae40 msgr2=0x7ffaa806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:27.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.764+0000 7ffacb0da640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ffaa806ae40 0x7ffaa806d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffac40ca490 tx=0x7ffab0009190 comp rx=0 tx=0).stop 2023-11-18T20:43:27.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.764+0000 7ffacb0da640 1 -- 172.21.15.60:0/437680206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac40d2910 msgr2=0x7ffac40ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:27.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.764+0000 7ffacb0da640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac40d2910 0x7ffac40ccc10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffab800e8a0 tx=0x7ffab800ed60 comp rx=0 tx=0).stop 2023-11-18T20:43:27.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.765+0000 7ffacb0da640 1 -- 172.21.15.60:0/437680206 shutdown_connections 2023-11-18T20:43:27.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.765+0000 7ffacb0da640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ffaa806ae40 0x7ffaa806d300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:27.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.765+0000 7ffacb0da640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac415d330 0x7ffac41f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.765+0000 7ffacb0da640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffac40d2d90 0x7ffac41f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.765+0000 7ffacb0da640 1 --2- 172.21.15.60:0/437680206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac40d2910 0x7ffac40ccc10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.765+0000 7ffacb0da640 1 -- 172.21.15.60:0/437680206 >> 172.21.15.60:0/437680206 conn(0x7ffac4072020 msgr2=0x7ffac40b8780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.765+0000 7ffacb0da640 1 -- 172.21.15.60:0/437680206 shutdown_connections 2023-11-18T20:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.765+0000 7ffacb0da640 1 -- 172.21.15.60:0/437680206 wait complete. 2023-11-18T20:43:27.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-18T20:43:27.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.894+0000 7f871e17c640 1 Processor -- start 2023-11-18T20:43:27.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.894+0000 7f871e17c640 1 -- start start 2023-11-18T20:43:27.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.895+0000 7f871e17c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8718144980 0x7f871815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:27.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.895+0000 7f871e17c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8718071770 con 0x7f87180b4370 2023-11-18T20:43:27.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.895+0000 7f871e17c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87180ccb60 con 0x7f87180ceb70 2023-11-18T20:43:27.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.895+0000 7f871e17c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87180ccdc0 con 0x7f8718144980 2023-11-18T20:43:27.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.895+0000 7f87177fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8718144980 0x7f871815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:27.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.895+0000 7f8717fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f87180b4370 0x7f8718071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45438/0 (socket says 172.21.15.60:45438) 2023-11-18T20:43:27.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.895+0000 7f8717fff640 1 -- 172.21.15.60:0/899254143 learned_addr learned my addr 172.21.15.60:0/899254143 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.896+0000 7f87177fe640 1 -- 172.21.15.60:0/899254143 >> v1:172.21.15.100:6789/0 conn(0x7f87180ceb70 legacy=0x7f87180cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.896+0000 7f87177fe640 1 -- 172.21.15.60:0/899254143 >> v1:172.21.15.60:6789/0 conn(0x7f87180b4370 legacy=0x7f8718071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.896+0000 7f87167fc640 1 -- 172.21.15.60:0/899254143 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010481352 0 0) 0x7f8718071770 con 0x7f87180b4370 2023-11-18T20:43:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.896+0000 7f87177fe640 1 -- 172.21.15.60:0/899254143 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87180cd0a0 con 0x7f8718144980 2023-11-18T20:43:27.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f87177fe640 1 --2- 172.21.15.60:0/899254143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8718144980 0x7f871815c7c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f87080097a0 tx=0x7f870802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89522862bc998e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:27.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f87167fc640 1 -- 172.21.15.60:0/899254143 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708031830 con 0x7f8718144980 2023-11-18T20:43:27.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f87167fc640 1 -- 172.21.15.60:0/899254143 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8708031990 con 0x7f8718144980 2023-11-18T20:43:27.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f87167fc640 1 -- 172.21.15.60:0/899254143 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708031d20 con 0x7f8718144980 2023-11-18T20:43:27.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f871e17c640 1 -- 172.21.15.60:0/899254143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8718144980 msgr2=0x7f871815c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:27.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f871e17c640 1 --2- 172.21.15.60:0/899254143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8718144980 0x7f871815c7c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f87080097a0 tx=0x7f870802b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:27.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f871e17c640 1 -- 172.21.15.60:0/899254143 shutdown_connections 2023-11-18T20:43:27.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f871e17c640 1 --2- 172.21.15.60:0/899254143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8718144980 0x7f871815c7c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:27.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f871e17c640 1 -- 172.21.15.60:0/899254143 >> 172.21.15.60:0/899254143 conn(0x7f871806a4d0 msgr2=0x7f871806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:27.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f871e17c640 1 -- 172.21.15.60:0/899254143 shutdown_connections 2023-11-18T20:43:27.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.897+0000 7f871e17c640 1 -- 172.21.15.60:0/899254143 wait complete. 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.899+0000 7f871e17c640 1 Processor -- start 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.899+0000 7f871e17c640 1 -- start start 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.899+0000 7f871e17c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87180b4370 0x7f8718137060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.899+0000 7f871e17c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87180ceb70 0x7f871813a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.899+0000 7f871e17c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8718144980 0x7f87180adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f871e17c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8718134400 con 0x7f87180ceb70 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f871e17c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871813a920 con 0x7f8718144980 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f871e17c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87181346a0 con 0x7f87180b4370 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f8716ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87180ceb70 0x7f871813a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f8716ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87180ceb70 0x7f871813a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50020/0 (socket says 172.21.15.60:50020) 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f8716ffd640 1 -- 172.21.15.60:0/3604995568 learned_addr learned my addr 172.21.15.60:0/3604995568 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f87177fe640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87180b4370 0x7f8718137060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f8717fff640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8718144980 0x7f87180adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f8716ffd640 1 -- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87180b4370 msgr2=0x7f8718137060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f8716ffd640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87180b4370 0x7f8718137060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f8716ffd640 1 -- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8718144980 msgr2=0x7f87180adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f8716ffd640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8718144980 0x7f87180adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f8716ffd640 1 -- 172.21.15.60:0/3604995568 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8708009400 con 0x7f87180ceb70 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f8716ffd640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87180ceb70 0x7f871813a3e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c430 tx=0x7f870c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f86fffff640 1 -- 172.21.15.60:0/3604995568 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c011790 con 0x7f87180ceb70 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f86fffff640 1 -- 172.21.15.60:0/3604995568 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c0118f0 con 0x7f87180ceb70 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f871e17c640 1 -- 172.21.15.60:0/3604995568 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8718134980 con 0x7f87180ceb70 2023-11-18T20:43:27.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f86fffff640 1 -- 172.21.15.60:0/3604995568 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c011ce0 con 0x7f87180ceb70 2023-11-18T20:43:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.900+0000 7f871e17c640 1 -- 172.21.15.60:0/3604995568 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8718134f40 con 0x7f87180ceb70 2023-11-18T20:43:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.901+0000 7f871e17c640 1 -- 172.21.15.60:0/3604995568 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87180cb8b0 con 0x7f87180ceb70 2023-11-18T20:43:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.902+0000 7f86fffff640 1 -- 172.21.15.60:0/3604995568 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f870c01c460 con 0x7f87180ceb70 2023-11-18T20:43:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.902+0000 7f86fffff640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f870406ae40 0x7f870406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.902+0000 7f86fffff640 1 -- 172.21.15.60:0/3604995568 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f870c089170 con 0x7f87180ceb70 2023-11-18T20:43:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.902+0000 7f87177fe640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f870406ae40 0x7f870406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.903+0000 7f87177fe640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f870406ae40 0x7f870406d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8708036000 tx=0x7f870803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:27.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:27.906+0000 7f86fffff640 1 -- 172.21.15.60:0/3604995568 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f870c0545a0 con 0x7f87180ceb70 2023-11-18T20:43:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.150+0000 7f871e17c640 1 -- 172.21.15.60:0/3604995568 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8718069400 con 0x7f87180ceb70 2023-11-18T20:43:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.150+0000 7f86fffff640 1 -- 172.21.15.60:0/3604995568 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f870c057bc0 con 0x7f87180ceb70 2023-11-18T20:43:28.152 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:28.153 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:43:28.153 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-11-18T20:43:28.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.153 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-11-18T20:43:28.153 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllC9UnFhAACm0q2kkeuVd7s/HDJ4HxsQ==", 2023-11-18T20:43:28.153 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.153 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:43:28.153 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-11-18T20:43:28.153 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-11-18T20:43:28.153 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVlluYuEIxAAQk5LaZvlSTcwf9UNsO5a9Q==", 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllaDI6HhAARs+hK3PNr6Pw7YXSHrks1g==", 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.155 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-11-18T20:43:28.156 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllFO7dKhAAXNdgA3EauG4YWKi99qLLRg==", 2023-11-18T20:43:28.156 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.156 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:43:28.156 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-11-18T20:43:28.156 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-11-18T20:43:28.156 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.156 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.156 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCHIVlli+6+GBAAF9u74MMa50Abf4XBRswFpQ==", 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCKIVlln59wFxAAiqokbWoD901iRvF5ESqEbw==", 2023-11-18T20:43:28.157 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.158 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:43:28.158 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:43:28.158 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.158 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.158 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-11-18T20:43:28.158 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNIVll3XquFhAA1zESpR80pRgZBgAC9U3aDw==", 2023-11-18T20:43:28.158 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.158 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQIVllMbZDFRAAVEOBFdVCedzteYM1w/enOA==", 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.159 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWIVllnl4LFxAAbiNZODS0LQROAz7MfPIo8A==", 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.160 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCZIVllTPeyFRAAB/IGUS8d7KCVaEqyKA6e3g==", 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCcIVllMIJ/FBAA7Wn5sn/6BBl+68DOSpAR5A==", 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCfIVllIAfgEhAAcOZ3Z5ZmYzIzqao5jmGb7A==", 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:43:28.162 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA==", 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllyxjzNhAAYt+5RXeMr8fnEZeiWX3b/w==", 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-11-18T20:43:28.164 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVlltRRZMxAALjGvfOl1LnaZj/6RaATH4w==", 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCGIVllI90eARAAOKYZObsxr760sxu2J+xy0Q==", 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.165 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCEIVll68DQMhAAnNh3Zk4de+9NNtg+zs4Hmg==", 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-11-18T20:43:28.166 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-mds", 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQClIVlle2NLExAAayc6sUzhM34t5n+G/Fqd1Q==", 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-mds" 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.167 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-mgr", 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQClIVllC2xLExAARjIDwgffkZylYlctauszHg==", 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-osd", 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQClIVlld3NLExAAFxyqxBXKF23aUBhxBFZSzQ==", 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-osd" 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rbd", 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQClIVll+XpLExAAXJEa7xqS4eBbAfUAbnBJPA==", 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQClIVlluoJLExAA13IZzi0vRTuH1knwhdYQIA==", 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.169 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rgw", 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQClIVlli4pLExAAZBCo5BKCge6QDZlP50RbwQ==", 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-11-18T20:43:28.170 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVlljlFBEhAAQDo8Xz2m+fCZpazdjrBOfQ==", 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllkJ9gDRAAiqXLdSkKHV4SjIPBOA85YA==", 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-11-18T20:43:28.171 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-11-18T20:43:28.172 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:43:28.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:28.172 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-11-18T20:43:28.172 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:43:28.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.152+0000 7f871e17c640 1 -- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f870406ae40 msgr2=0x7f870406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.152+0000 7f871e17c640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f870406ae40 0x7f870406d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8708036000 tx=0x7f870803d040 comp rx=0 tx=0).stop 2023-11-18T20:43:28.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.152+0000 7f871e17c640 1 -- 172.21.15.60:0/3604995568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87180ceb70 msgr2=0x7f871813a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.152+0000 7f871e17c640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87180ceb70 0x7f871813a3e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c430 tx=0x7f870c00c8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.152+0000 7f871e17c640 1 -- 172.21.15.60:0/3604995568 shutdown_connections 2023-11-18T20:43:28.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.152+0000 7f871e17c640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f870406ae40 0x7f870406d300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.152+0000 7f871e17c640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8718144980 0x7f87180adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.152+0000 7f871e17c640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87180ceb70 0x7f871813a3e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.152+0000 7f871e17c640 1 --2- 172.21.15.60:0/3604995568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87180b4370 0x7f8718137060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.152+0000 7f871e17c640 1 -- 172.21.15.60:0/3604995568 >> 172.21.15.60:0/3604995568 conn(0x7f871806a4d0 msgr2=0x7f8718158420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:28.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.153+0000 7f871e17c640 1 -- 172.21.15.60:0/3604995568 shutdown_connections 2023-11-18T20:43:28.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.153+0000 7f871e17c640 1 -- 172.21.15.60:0/3604995568 wait complete. 2023-11-18T20:43:28.175 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-18T20:43:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.281+0000 7fb58c0b4640 1 Processor -- start 2023-11-18T20:43:28.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.281+0000 7fb58c0b4640 1 -- start start 2023-11-18T20:43:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.282+0000 7fb58c0b4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5840d2910 0x7fb5840d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.282+0000 7fb58c0b4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb5840d3730 0x7fb5840b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.282+0000 7fb58c0b4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5840b2b10 con 0x7fb58415d330 2023-11-18T20:43:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.282+0000 7fb58c0b4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5840d3bc0 con 0x7fb5840d2910 2023-11-18T20:43:28.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.282+0000 7fb58c0b4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5840d1590 con 0x7fb5840d3730 2023-11-18T20:43:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.282+0000 7fb58964f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb5840d2910 0x7fb5840d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.282+0000 7fb589e50640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb58415d330 0x7fb5840b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45446/0 (socket says 172.21.15.60:45446) 2023-11-18T20:43:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.282+0000 7fb589e50640 1 -- 172.21.15.60:0/1641770399 learned_addr learned my addr 172.21.15.60:0/1641770399 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.283+0000 7fb58a651640 1 --2- 172.21.15.60:0/1641770399 >> v2:172.21.15.100:3300/0 conn(0x7fb5840d3730 0x7fb5840b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.283+0000 7fb58964f640 1 -- 172.21.15.60:0/1641770399 >> v2:172.21.15.100:3300/0 conn(0x7fb5840d3730 msgr2=0x7fb5840b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.283+0000 7fb58964f640 1 --2- 172.21.15.60:0/1641770399 >> v2:172.21.15.100:3300/0 conn(0x7fb5840d3730 0x7fb5840b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.283+0000 7fb58964f640 1 -- 172.21.15.60:0/1641770399 >> v1:172.21.15.60:6789/0 conn(0x7fb58415d330 legacy=0x7fb5840b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.283+0000 7fb58964f640 1 -- 172.21.15.60:0/1641770399 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5840d1870 con 0x7fb5840d2910 2023-11-18T20:43:28.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.283+0000 7fb588e4e640 1 -- 172.21.15.60:0/1641770399 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366833632 0 0) 0x7fb5840b2b10 con 0x7fb58415d330 2023-11-18T20:43:28.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb58964f640 1 --2- 172.21.15.60:0/1641770399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d2910 0x7fb5840d31f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb58406c500 tx=0x7fb57802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6414e4a19f44e1f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:28.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb588e4e640 1 -- 172.21.15.60:0/1641770399 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57802e070 con 0x7fb5840d2910 2023-11-18T20:43:28.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb588e4e640 1 -- 172.21.15.60:0/1641770399 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5780315a0 con 0x7fb5840d2910 2023-11-18T20:43:28.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb588e4e640 1 -- 172.21.15.60:0/1641770399 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578031930 con 0x7fb5840d2910 2023-11-18T20:43:28.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1641770399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d2910 msgr2=0x7fb5840d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb58c0b4640 1 --2- 172.21.15.60:0/1641770399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d2910 0x7fb5840d31f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb58406c500 tx=0x7fb57802b840 comp rx=0 tx=0).stop 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1641770399 shutdown_connections 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb58c0b4640 1 --2- 172.21.15.60:0/1641770399 >> v2:172.21.15.100:3300/0 conn(0x7fb5840d3730 0x7fb5840b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb58c0b4640 1 --2- 172.21.15.60:0/1641770399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d2910 0x7fb5840d31f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1641770399 >> 172.21.15.60:0/1641770399 conn(0x7fb584072020 msgr2=0x7fb584072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1641770399 shutdown_connections 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.284+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1641770399 wait complete. 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58c0b4640 1 Processor -- start 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58c0b4640 1 -- start start 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58c0b4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5840d2910 0x7fb58415e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58c0b4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d3730 0x7fb584160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58c0b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb58415d330 0x7fb5841605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58c0b4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5841f5f90 con 0x7fb5840d3730 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58c0b4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5841f6230 con 0x7fb58415d330 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58c0b4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5841f6510 con 0x7fb5840d2910 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58964f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d3730 0x7fb584160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58964f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d3730 0x7fb584160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50044/0 (socket says 172.21.15.60:50044) 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.286+0000 7fb58964f640 1 -- 172.21.15.60:0/1087089205 learned_addr learned my addr 172.21.15.60:0/1087089205 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb58a651640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb58415d330 0x7fb5841605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb589e50640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5840d2910 0x7fb58415e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb58964f640 1 -- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5840d2910 msgr2=0x7fb58415e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb58964f640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5840d2910 0x7fb58415e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb58964f640 1 -- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb58415d330 msgr2=0x7fb5841605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb58964f640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb58415d330 0x7fb5841605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb58964f640 1 -- 172.21.15.60:0/1087089205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578009400 con 0x7fb5840d3730 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb58964f640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d3730 0x7fb584160060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb578009890 tx=0x7fb5780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb5827fc640 1 -- 172.21.15.60:0/1087089205 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578032840 con 0x7fb5840d3730 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb5827fc640 1 -- 172.21.15.60:0/1087089205 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5780329a0 con 0x7fb5840d3730 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1087089205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5841f67f0 con 0x7fb5840d3730 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb5827fc640 1 -- 172.21.15.60:0/1087089205 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578032d50 con 0x7fb5840d3730 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.287+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1087089205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58415eb00 con 0x7fb5840d3730 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.288+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1087089205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5841f6d10 con 0x7fb5840d3730 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.289+0000 7fb5827fc640 1 -- 172.21.15.60:0/1087089205 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb57803b3f0 con 0x7fb5840d3730 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.289+0000 7fb5827fc640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb57006ae40 0x7fb57006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.289+0000 7fb5827fc640 1 -- 172.21.15.60:0/1087089205 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb5780ab270 con 0x7fb5840d3730 2023-11-18T20:43:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.289+0000 7fb589e50640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb57006ae40 0x7fb57006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.290+0000 7fb589e50640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb57006ae40 0x7fb57006d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb5740012e0 tx=0x7fb574005eb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:28.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.294+0000 7fb5827fc640 1 -- 172.21.15.60:0/1087089205 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb5841f6d10 con 0x7fb5840d3730 2023-11-18T20:43:28.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.535+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1087089205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fb5841f6d10 con 0x7fb5840d3730 2023-11-18T20:43:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.537+0000 7fb5827fc640 1 -- 172.21.15.60:0/1087089205 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb5841f6d10 con 0x7fb5840d3730 2023-11-18T20:43:28.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb57006ae40 msgr2=0x7fb57006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb57006ae40 0x7fb57006d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb5740012e0 tx=0x7fb574005eb0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1087089205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d3730 msgr2=0x7fb584160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d3730 0x7fb584160060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb578009890 tx=0x7fb5780046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1087089205 shutdown_connections 2023-11-18T20:43:28.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb57006ae40 0x7fb57006d300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb58415d330 0x7fb5841605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5840d3730 0x7fb584160060 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 --2- 172.21.15.60:0/1087089205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5840d2910 0x7fb58415e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1087089205 >> 172.21.15.60:0/1087089205 conn(0x7fb584072020 msgr2=0x7fb584072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:28.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1087089205 shutdown_connections 2023-11-18T20:43:28.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.539+0000 7fb58c0b4640 1 -- 172.21.15.60:0/1087089205 wait complete. 2023-11-18T20:43:28.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-18T20:43:28.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.657+0000 7f8861b47640 1 Processor -- start 2023-11-18T20:43:28.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.657+0000 7f8861b47640 1 -- start start 2023-11-18T20:43:28.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.658+0000 7f8861b47640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f885c15d330 0x7f885c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.658+0000 7f8861b47640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f885c0d2910 0x7f885c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.658+0000 7f8861b47640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885c0b2b10 con 0x7f885c0d3730 2023-11-18T20:43:28.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.658+0000 7f8861b47640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c0d3bc0 con 0x7f885c0d2910 2023-11-18T20:43:28.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.658+0000 7f8861b47640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c0d1590 con 0x7f885c15d330 2023-11-18T20:43:28.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.658+0000 7f885affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f885c0d2910 0x7f885c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.658+0000 7f885affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f885c0d2910 0x7f885c0d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50060/0 (socket says 172.21.15.60:50060) 2023-11-18T20:43:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.658+0000 7f885affd640 1 -- 172.21.15.60:0/736133148 learned_addr learned my addr 172.21.15.60:0/736133148 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.659+0000 7f885b7fe640 1 --2- 172.21.15.60:0/736133148 >> v2:172.21.15.100:3300/0 conn(0x7f885c15d330 0x7f885c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.659+0000 7f885affd640 1 -- 172.21.15.60:0/736133148 >> v2:172.21.15.100:3300/0 conn(0x7f885c15d330 msgr2=0x7f885c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.659+0000 7f885affd640 1 --2- 172.21.15.60:0/736133148 >> v2:172.21.15.100:3300/0 conn(0x7f885c15d330 0x7f885c0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.659+0000 7f885affd640 1 -- 172.21.15.60:0/736133148 >> v1:172.21.15.100:6789/0 conn(0x7f885c0d3730 legacy=0x7f885c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.661 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.659+0000 7f885affd640 1 -- 172.21.15.60:0/736133148 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0d1870 con 0x7f885c0d2910 2023-11-18T20:43:28.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.659+0000 7f885affd640 1 --2- 172.21.15.60:0/736133148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 0x7f885c0d31f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f885c0d0d10 tx=0x7f885002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180419910a4846b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:28.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f885a7fc640 1 -- 172.21.15.60:0/736133148 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850031870 con 0x7f885c0d2910 2023-11-18T20:43:28.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f885a7fc640 1 -- 172.21.15.60:0/736133148 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88500319d0 con 0x7f885c0d2910 2023-11-18T20:43:28.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f885a7fc640 1 -- 172.21.15.60:0/736133148 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850031d60 con 0x7f885c0d2910 2023-11-18T20:43:28.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f8861b47640 1 -- 172.21.15.60:0/736133148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 msgr2=0x7f885c0d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f8861b47640 1 --2- 172.21.15.60:0/736133148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 0x7f885c0d31f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f885c0d0d10 tx=0x7f885002b910 comp rx=0 tx=0).stop 2023-11-18T20:43:28.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f8861b47640 1 -- 172.21.15.60:0/736133148 shutdown_connections 2023-11-18T20:43:28.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f8861b47640 1 --2- 172.21.15.60:0/736133148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 0x7f885c0d31f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f8861b47640 1 --2- 172.21.15.60:0/736133148 >> v2:172.21.15.100:3300/0 conn(0x7f885c15d330 0x7f885c0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f8861b47640 1 -- 172.21.15.60:0/736133148 >> 172.21.15.60:0/736133148 conn(0x7f885c072020 msgr2=0x7f885c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:28.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f8861b47640 1 -- 172.21.15.60:0/736133148 shutdown_connections 2023-11-18T20:43:28.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.660+0000 7f8861b47640 1 -- 172.21.15.60:0/736133148 wait complete. 2023-11-18T20:43:28.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.661+0000 7f8861b47640 1 Processor -- start 2023-11-18T20:43:28.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.662+0000 7f8861b47640 1 -- start start 2023-11-18T20:43:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.662+0000 7f8861b47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 0x7f885c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.662+0000 7f8861b47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f885c0d3730 0x7f885c1f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.662+0000 7f8861b47640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f885c15d330 0x7f885c0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.662+0000 7f8861b47640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c152130 con 0x7f885c0d2910 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.662+0000 7f8861b47640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c1f6c80 con 0x7f885c0d3730 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.662+0000 7f8861b47640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c1f6f60 con 0x7f885c15d330 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.662+0000 7f885b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 0x7f885c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.662+0000 7f885b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 0x7f885c0ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50064/0 (socket says 172.21.15.60:50064) 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.662+0000 7f885b7fe640 1 -- 172.21.15.60:0/298294612 learned_addr learned my addr 172.21.15.60:0/298294612 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f885affd640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f885c0d3730 0x7f885c1f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f885bfff640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f885c15d330 0x7f885c0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f885b7fe640 1 -- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f885c15d330 msgr2=0x7f885c0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f885b7fe640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f885c15d330 0x7f885c0c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f885b7fe640 1 -- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f885c0d3730 msgr2=0x7f885c1f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f885b7fe640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f885c0d3730 0x7f885c1f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f885b7fe640 1 -- 172.21.15.60:0/298294612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88500094d0 con 0x7f885c0d2910 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f885affd640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f885c0d3730 0x7f885c1f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f885bfff640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f885c15d330 0x7f885c0c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f885b7fe640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 0x7f885c0ccc40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f884c00d5f0 tx=0x7f884c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f8843fff640 1 -- 172.21.15.60:0/298294612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0125f0 con 0x7f885c0d2910 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f8843fff640 1 -- 172.21.15.60:0/298294612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c012750 con 0x7f885c0d2910 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f8861b47640 1 -- 172.21.15.60:0/298294612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885c1523a0 con 0x7f885c0d2910 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f8843fff640 1 -- 172.21.15.60:0/298294612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c012b00 con 0x7f885c0d2910 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.663+0000 7f8861b47640 1 -- 172.21.15.60:0/298294612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c1f6900 con 0x7f885c0d2910 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.664+0000 7f8861b47640 1 -- 172.21.15.60:0/298294612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885c069400 con 0x7f885c0d2910 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.665+0000 7f8843fff640 1 -- 172.21.15.60:0/298294612 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f884c012c60 con 0x7f885c0d2910 2023-11-18T20:43:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.665+0000 7f8843fff640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f884806ae70 0x7f884806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.665+0000 7f8843fff640 1 -- 172.21.15.60:0/298294612 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f884c0896f0 con 0x7f885c0d2910 2023-11-18T20:43:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.665+0000 7f885affd640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f884806ae70 0x7f884806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:28.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.666+0000 7f885affd640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f884806ae70 0x7f884806d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8850036000 tx=0x7f885003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.669+0000 7f8843fff640 1 -- 172.21.15.60:0/298294612 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f884c054b80 con 0x7f885c0d2910 2023-11-18T20:43:28.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.917+0000 7f8861b47640 1 -- 172.21.15.60:0/298294612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f885c0b8d50 con 0x7f885c0d2910 2023-11-18T20:43:28.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.919+0000 7f8843fff640 1 -- 172.21.15.60:0/298294612 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f884c0581a0 con 0x7f885c0d2910 2023-11-18T20:43:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 -- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f884806ae70 msgr2=0x7f884806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f884806ae70 0x7f884806d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8850036000 tx=0x7f885003c040 comp rx=0 tx=0).stop 2023-11-18T20:43:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 -- 172.21.15.60:0/298294612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 msgr2=0x7f885c0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 0x7f885c0ccc40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f884c00d5f0 tx=0x7f884c00dab0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 -- 172.21.15.60:0/298294612 shutdown_connections 2023-11-18T20:43:28.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f884806ae70 0x7f884806d330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f885c15d330 0x7f885c0c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f885c0d3730 0x7f885c1f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 --2- 172.21.15.60:0/298294612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f885c0d2910 0x7f885c0ccc40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:28.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 -- 172.21.15.60:0/298294612 >> 172.21.15.60:0/298294612 conn(0x7f885c072020 msgr2=0x7f885c072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:28.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 -- 172.21.15.60:0/298294612 shutdown_connections 2023-11-18T20:43:28.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:28.921+0000 7f8861b47640 1 -- 172.21.15.60:0/298294612 wait complete. 2023-11-18T20:43:28.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-18T20:43:29.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.039+0000 7f4f952da640 1 Processor -- start 2023-11-18T20:43:29.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.039+0000 7f4f952da640 1 -- start start 2023-11-18T20:43:29.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.039+0000 7f4f952da640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4f9015d340 0x7f4f9006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.039+0000 7f4f952da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900aeb80 0x7f4f9006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f952da640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9015d720 con 0x7f4f900ad640 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f952da640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f900adae0 con 0x7f4f9015d340 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f952da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900addc0 con 0x7f4f900aeb80 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f8effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4f9015d340 0x7f4f9006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f8e7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4f900ad640 0x7f4f90160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45450/0 (socket says 172.21.15.60:45450) 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f8e7fc640 1 -- 172.21.15.60:0/3012651384 learned_addr learned my addr 172.21.15.60:0/3012651384 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f8f7fe640 1 --2- 172.21.15.60:0/3012651384 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900aeb80 0x7f4f9006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f8effd640 1 -- 172.21.15.60:0/3012651384 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900aeb80 msgr2=0x7f4f9006ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f8dffb640 1 -- 172.21.15.60:0/3012651384 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750401207 0 0) 0x7f4f9015d720 con 0x7f4f900ad640 2023-11-18T20:43:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f8effd640 1 --2- 172.21.15.60:0/3012651384 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900aeb80 0x7f4f9006ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f8effd640 1 -- 172.21.15.60:0/3012651384 >> v1:172.21.15.60:6789/0 conn(0x7f4f900ad640 legacy=0x7f4f90160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f8effd640 1 -- 172.21.15.60:0/3012651384 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f900ae0a0 con 0x7f4f9015d340 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.040+0000 7f4f8f7fe640 1 --2- 172.21.15.60:0/3012651384 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900aeb80 0x7f4f9006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.041+0000 7f4f8effd640 1 --2- 172.21.15.60:0/3012651384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 0x7f4f9006b4e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4f800097a0 tx=0x7f4f8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284b1fb096ef2726 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.041+0000 7f4f8dffb640 1 -- 172.21.15.60:0/3012651384 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80031830 con 0x7f4f9015d340 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.041+0000 7f4f8dffb640 1 -- 172.21.15.60:0/3012651384 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f80031990 con 0x7f4f9015d340 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.041+0000 7f4f8dffb640 1 -- 172.21.15.60:0/3012651384 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80031d20 con 0x7f4f9015d340 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.041+0000 7f4f952da640 1 -- 172.21.15.60:0/3012651384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 msgr2=0x7f4f9006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.041+0000 7f4f952da640 1 --2- 172.21.15.60:0/3012651384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 0x7f4f9006b4e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4f800097a0 tx=0x7f4f8002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.041+0000 7f4f952da640 1 -- 172.21.15.60:0/3012651384 shutdown_connections 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.042+0000 7f4f952da640 1 --2- 172.21.15.60:0/3012651384 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900aeb80 0x7f4f9006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.042+0000 7f4f952da640 1 --2- 172.21.15.60:0/3012651384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 0x7f4f9006b4e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.042+0000 7f4f952da640 1 -- 172.21.15.60:0/3012651384 >> 172.21.15.60:0/3012651384 conn(0x7f4f90070e70 msgr2=0x7f4f90071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.042+0000 7f4f952da640 1 -- 172.21.15.60:0/3012651384 shutdown_connections 2023-11-18T20:43:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.042+0000 7f4f952da640 1 -- 172.21.15.60:0/3012651384 wait complete. 2023-11-18T20:43:29.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.043+0000 7f4f952da640 1 Processor -- start 2023-11-18T20:43:29.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.043+0000 7f4f952da640 1 -- start start 2023-11-18T20:43:29.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f952da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900ad640 0x7f4f901513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f952da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f900aeb80 0x7f4f9006d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f952da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 0x7f4f9006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f952da640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900cf370 con 0x7f4f9015d340 2023-11-18T20:43:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f952da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900ce7a0 con 0x7f4f900aeb80 2023-11-18T20:43:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f952da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900cea40 con 0x7f4f900ad640 2023-11-18T20:43:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 0x7f4f9006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 0x7f4f9006d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50080/0 (socket says 172.21.15.60:50080) 2023-11-18T20:43:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8f7fe640 1 -- 172.21.15.60:0/814745676 learned_addr learned my addr 172.21.15.60:0/814745676 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8effd640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900ad640 0x7f4f901513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8e7fc640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f900aeb80 0x7f4f9006d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8f7fe640 1 -- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900ad640 msgr2=0x7f4f901513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8f7fe640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900ad640 0x7f4f901513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8f7fe640 1 -- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f900aeb80 msgr2=0x7f4f9006d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8f7fe640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f900aeb80 0x7f4f9006d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8f7fe640 1 -- 172.21.15.60:0/814745676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80009400 con 0x7f4f9015d340 2023-11-18T20:43:29.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f8f7fe640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 0x7f4f9006d660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800cef0 tx=0x7f4f8800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:29.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f777fe640 1 -- 172.21.15.60:0/814745676 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8800d840 con 0x7f4f9015d340 2023-11-18T20:43:29.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f777fe640 1 -- 172.21.15.60:0/814745676 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8800d9c0 con 0x7f4f9015d340 2023-11-18T20:43:29.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f952da640 1 -- 172.21.15.60:0/814745676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f900ced20 con 0x7f4f9015d340 2023-11-18T20:43:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f777fe640 1 -- 172.21.15.60:0/814745676 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8800dd70 con 0x7f4f9015d340 2023-11-18T20:43:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.044+0000 7f4f952da640 1 -- 172.21.15.60:0/814745676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f901518f0 con 0x7f4f9015d340 2023-11-18T20:43:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.045+0000 7f4f952da640 1 -- 172.21.15.60:0/814745676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f900cf520 con 0x7f4f9015d340 2023-11-18T20:43:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.046+0000 7f4f777fe640 1 -- 172.21.15.60:0/814745676 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f8801e3b0 con 0x7f4f9015d340 2023-11-18T20:43:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.046+0000 7f4f777fe640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4f7c06ae70 0x7f4f7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.046+0000 7f4f777fe640 1 -- 172.21.15.60:0/814745676 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f88089cd0 con 0x7f4f9015d340 2023-11-18T20:43:29.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.046+0000 7f4f8effd640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4f7c06ae70 0x7f4f7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.047+0000 7f4f8effd640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4f7c06ae70 0x7f4f7c06d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4f80036000 tx=0x7f4f8003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:29.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.050+0000 7f4f777fe640 1 -- 172.21.15.60:0/814745676 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f900cf520 con 0x7f4f9015d340 2023-11-18T20:43:29.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.291+0000 7f4f952da640 1 -- 172.21.15.60:0/814745676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f4f900677a0 con 0x7f4f9015d340 2023-11-18T20:43:29.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.293+0000 7f4f777fe640 1 -- 172.21.15.60:0/814745676 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4f8801a0a0 con 0x7f4f9015d340 2023-11-18T20:43:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 -- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4f7c06ae70 msgr2=0x7f4f7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4f7c06ae70 0x7f4f7c06d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4f80036000 tx=0x7f4f8003c040 comp rx=0 tx=0).stop 2023-11-18T20:43:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 -- 172.21.15.60:0/814745676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 msgr2=0x7f4f9006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 0x7f4f9006d660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800cef0 tx=0x7f4f8800a710 comp rx=0 tx=0).stop 2023-11-18T20:43:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 -- 172.21.15.60:0/814745676 shutdown_connections 2023-11-18T20:43:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4f7c06ae70 0x7f4f7c06d330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f9015d340 0x7f4f9006d660 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f900aeb80 0x7f4f9006d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 --2- 172.21.15.60:0/814745676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f900ad640 0x7f4f901513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 -- 172.21.15.60:0/814745676 >> 172.21.15.60:0/814745676 conn(0x7f4f90070e70 msgr2=0x7f4f9006eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 -- 172.21.15.60:0/814745676 shutdown_connections 2023-11-18T20:43:29.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.296+0000 7f4f952da640 1 -- 172.21.15.60:0/814745676 wait complete. 2023-11-18T20:43:29.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-18T20:43:29.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.419+0000 7fbda8141640 1 Processor -- start 2023-11-18T20:43:29.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.420+0000 7fbda8141640 1 -- start start 2023-11-18T20:43:29.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.420+0000 7fbda8141640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda015d330 0x7fbda00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.420+0000 7fbda8141640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbda00d2910 0x7fbda00d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.420+0000 7fbda8141640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda00b2b10 con 0x7fbda00d2d90 2023-11-18T20:43:29.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.420+0000 7fbda8141640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda00d1590 con 0x7fbda00d2910 2023-11-18T20:43:29.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.420+0000 7fbda8141640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00d1870 con 0x7fbda015d330 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda66de640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbda00d2d90 0x7fbda00b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45466/0 (socket says 172.21.15.60:45466) 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda66de640 1 -- 172.21.15.60:0/2781720800 learned_addr learned my addr 172.21.15.60:0/2781720800 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda56dc640 1 --2- 172.21.15.60:0/2781720800 >> v2:172.21.15.100:3300/0 conn(0x7fbda00d2910 0x7fbda00d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda5edd640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda015d330 0x7fbda00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351219204 0 0) 0x7fbda00b2b10 con 0x7fbda00d2d90 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd8c003520 con 0x7fbda00d2d90 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3127350699 0 0) 0x7fbd8c003520 con 0x7fbda00d2d90 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda015d330 msgr2=0x7fbda00b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda015d330 0x7fbda00b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 >> v2:172.21.15.100:3300/0 conn(0x7fbda00d2910 msgr2=0x7fbda00d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 --2- 172.21.15.60:0/2781720800 >> v2:172.21.15.100:3300/0 conn(0x7fbda00d2910 0x7fbda00d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda00d1b50 con 0x7fbda00d2d90 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fbd980019f0 con 0x7fbda00d2d90 2023-11-18T20:43:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 >> v1:172.21.15.60:6789/0 conn(0x7fbda00d2d90 legacy=0x7fbda00b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda015d330 0x7fbd8c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd8c003ef0 0x7fbd8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd8c004d00 0x7fbd8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00d1590 con 0x7fbd8c003ef0 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00d1870 con 0x7fbd8c004d00 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c00b9d0 con 0x7fbda015d330 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda56dc640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd8c003ef0 0x7fbd8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda5edd640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda015d330 0x7fbd8c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.421+0000 7fbda66de640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd8c004d00 0x7fbd8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.422+0000 7fbda56dc640 1 -- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda015d330 msgr2=0x7fbd8c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.422+0000 7fbda56dc640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda015d330 0x7fbd8c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.422+0000 7fbda56dc640 1 -- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd8c004d00 msgr2=0x7fbd8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.422+0000 7fbda56dc640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd8c004d00 0x7fbd8c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.422+0000 7fbda56dc640 1 -- 172.21.15.60:0/2781720800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c003520 con 0x7fbd8c003ef0 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.422+0000 7fbda56dc640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd8c003ef0 0x7fbd8c0047c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009af0 tx=0x7fbd8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8963c3b9c5e898 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.423+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9002c710 con 0x7fbd8c003ef0 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.423+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9002c870 con 0x7fbd8c003ef0 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.423+0000 7fbda4edb640 1 -- 172.21.15.60:0/2781720800 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9002cc00 con 0x7fbd8c003ef0 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.423+0000 7fbda8141640 1 -- 172.21.15.60:0/2781720800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd8c003ef0 msgr2=0x7fbd8c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.423+0000 7fbda8141640 1 --2- 172.21.15.60:0/2781720800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd8c003ef0 0x7fbd8c0047c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009af0 tx=0x7fbd8c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.423+0000 7fbda66de640 1 -- 172.21.15.60:0/2781720800 reap_dead start 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.423+0000 7fbda8141640 1 -- 172.21.15.60:0/2781720800 shutdown_connections 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.423+0000 7fbda8141640 1 -- 172.21.15.60:0/2781720800 >> 172.21.15.60:0/2781720800 conn(0x7fbda0072020 msgr2=0x7fbda0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.423+0000 7fbda8141640 1 -- 172.21.15.60:0/2781720800 shutdown_connections 2023-11-18T20:43:29.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.423+0000 7fbda8141640 1 -- 172.21.15.60:0/2781720800 wait complete. 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.424+0000 7fbda8141640 1 Processor -- start 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.424+0000 7fbda8141640 1 -- start start 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda8141640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd8c003ef0 0x7fbda014d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda8141640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbda00cb750 0x7fbda00cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda8141640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda00c8c70 0x7fbda00c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda8141640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda014db30 con 0x7fbda00cb750 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda8141640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda0157450 con 0x7fbd8c003ef0 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda8141640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda0157730 con 0x7fbda00c8c70 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda56dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbda00cb750 0x7fbda00cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda56dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbda00cb750 0x7fbda00cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50092/0 (socket says 172.21.15.60:50092) 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda56dc640 1 -- 172.21.15.60:0/297638298 learned_addr learned my addr 172.21.15.60:0/297638298 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:29.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda5edd640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd8c003ef0 0x7fbda014d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda66de640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda00c8c70 0x7fbda00c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda56dc640 1 -- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda00c8c70 msgr2=0x7fbda00c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda56dc640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda00c8c70 0x7fbda00c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda56dc640 1 -- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd8c003ef0 msgr2=0x7fbda014d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.425+0000 7fbda56dc640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd8c003ef0 0x7fbda014d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.426+0000 7fbda56dc640 1 -- 172.21.15.60:0/297638298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda0157a10 con 0x7fbda00cb750 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.426+0000 7fbda5edd640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd8c003ef0 0x7fbda014d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.426+0000 7fbda66de640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda00c8c70 0x7fbda00c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.426+0000 7fbda56dc640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbda00cb750 0x7fbda00cbbd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbd90037040 tx=0x7fbd90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.426+0000 7fbd967fc640 1 -- 172.21.15.60:0/297638298 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90003810 con 0x7fbda00cb750 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.426+0000 7fbd967fc640 1 -- 172.21.15.60:0/297638298 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9002de90 con 0x7fbda00cb750 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.426+0000 7fbd967fc640 1 -- 172.21.15.60:0/297638298 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90003b70 con 0x7fbda00cb750 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.426+0000 7fbda8141640 1 -- 172.21.15.60:0/297638298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda0157cf0 con 0x7fbda00cb750 2023-11-18T20:43:29.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.426+0000 7fbda8141640 1 -- 172.21.15.60:0/297638298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda01570e0 con 0x7fbda00cb750 2023-11-18T20:43:29.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.427+0000 7fbda8141640 1 -- 172.21.15.60:0/297638298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda0069400 con 0x7fbda00cb750 2023-11-18T20:43:29.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.427+0000 7fbd967fc640 1 -- 172.21.15.60:0/297638298 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd90003cd0 con 0x7fbda00cb750 2023-11-18T20:43:29.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.428+0000 7fbd967fc640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd8406ae40 0x7fbd8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.428+0000 7fbd967fc640 1 -- 172.21.15.60:0/297638298 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd900ab180 con 0x7fbda00cb750 2023-11-18T20:43:29.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.428+0000 7fbda5edd640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd8406ae40 0x7fbd8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.429+0000 7fbda5edd640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd8406ae40 0x7fbd8406d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbda00c6b50 tx=0x7fbd9c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:29.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.432+0000 7fbd967fc640 1 -- 172.21.15.60:0/297638298 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd90076610 con 0x7fbda00cb750 2023-11-18T20:43:29.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.674+0000 7fbda8141640 1 -- 172.21.15.60:0/297638298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fbda015ca50 con 0x7fbda00cb750 2023-11-18T20:43:29.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.676+0000 7fbd967fc640 1 -- 172.21.15.60:0/297638298 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbd90079c30 con 0x7fbda00cb750 2023-11-18T20:43:29.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 -- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd8406ae40 msgr2=0x7fbd8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd8406ae40 0x7fbd8406d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbda00c6b50 tx=0x7fbd9c008040 comp rx=0 tx=0).stop 2023-11-18T20:43:29.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 -- 172.21.15.60:0/297638298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbda00cb750 msgr2=0x7fbda00cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbda00cb750 0x7fbda00cbbd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbd90037040 tx=0x7fbd90004730 comp rx=0 tx=0).stop 2023-11-18T20:43:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 -- 172.21.15.60:0/297638298 shutdown_connections 2023-11-18T20:43:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbd8406ae40 0x7fbd8406d300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbda00c8c70 0x7fbda00c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbda00cb750 0x7fbda00cbbd0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 --2- 172.21.15.60:0/297638298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd8c003ef0 0x7fbda014d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 -- 172.21.15.60:0/297638298 >> 172.21.15.60:0/297638298 conn(0x7fbda0072020 msgr2=0x7fbda00b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 -- 172.21.15.60:0/297638298 shutdown_connections 2023-11-18T20:43:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.678+0000 7fbda8141640 1 -- 172.21.15.60:0/297638298 wait complete. 2023-11-18T20:43:29.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-18T20:43:29.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.796+0000 7f1639d5e640 1 Processor -- start 2023-11-18T20:43:29.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.796+0000 7f1639d5e640 1 -- start start 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.797+0000 7f1639d5e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f163415d330 0x7f16340b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.797+0000 7f1639d5e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f16340d3900 0x7f16340b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.797+0000 7f1639d5e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340b2b10 con 0x7f16340d2910 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.797+0000 7f1639d5e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16340d1590 con 0x7f16340d3900 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.797+0000 7f1639d5e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16340d1870 con 0x7f163415d330 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.797+0000 7f1633fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f16340d3900 0x7f16340b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.797+0000 7f1633fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f16340d3900 0x7f16340b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50108/0 (socket says 172.21.15.60:50108) 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.797+0000 7f1633fff640 1 -- 172.21.15.60:0/1061606251 learned_addr learned my addr 172.21.15.60:0/1061606251 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.798+0000 7f16337fe640 1 --2- 172.21.15.60:0/1061606251 >> v2:172.21.15.100:3300/0 conn(0x7f163415d330 0x7f16340b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.798+0000 7f1633fff640 1 -- 172.21.15.60:0/1061606251 >> v2:172.21.15.100:3300/0 conn(0x7f163415d330 msgr2=0x7f16340b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.798+0000 7f1633fff640 1 --2- 172.21.15.60:0/1061606251 >> v2:172.21.15.100:3300/0 conn(0x7f163415d330 0x7f16340b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.798+0000 7f1633fff640 1 -- 172.21.15.60:0/1061606251 >> v1:172.21.15.100:6789/0 conn(0x7f16340d2910 legacy=0x7f16340d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:29.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.798+0000 7f1633fff640 1 -- 172.21.15.60:0/1061606251 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340d1b50 con 0x7f16340d3900 2023-11-18T20:43:29.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.798+0000 7f16337fe640 1 --2- 172.21.15.60:0/1061606251 >> v2:172.21.15.100:3300/0 conn(0x7f163415d330 0x7f16340b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.798+0000 7f1633fff640 1 --2- 172.21.15.60:0/1061606251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 0x7f16340b6390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f16340d0d10 tx=0x7f162c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5150bb53d165bf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f16327fc640 1 -- 172.21.15.60:0/1061606251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c02c9f0 con 0x7f16340d3900 2023-11-18T20:43:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f16327fc640 1 -- 172.21.15.60:0/1061606251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c02cb50 con 0x7f16340d3900 2023-11-18T20:43:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f16327fc640 1 -- 172.21.15.60:0/1061606251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c0343f0 con 0x7f16340d3900 2023-11-18T20:43:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f1639d5e640 1 -- 172.21.15.60:0/1061606251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 msgr2=0x7f16340b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f1639d5e640 1 --2- 172.21.15.60:0/1061606251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 0x7f16340b6390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f16340d0d10 tx=0x7f162c02be50 comp rx=0 tx=0).stop 2023-11-18T20:43:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f1639d5e640 1 -- 172.21.15.60:0/1061606251 shutdown_connections 2023-11-18T20:43:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f1639d5e640 1 --2- 172.21.15.60:0/1061606251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 0x7f16340b6390 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f1639d5e640 1 --2- 172.21.15.60:0/1061606251 >> v2:172.21.15.100:3300/0 conn(0x7f163415d330 0x7f16340b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f1639d5e640 1 -- 172.21.15.60:0/1061606251 >> 172.21.15.60:0/1061606251 conn(0x7f1634072020 msgr2=0x7f1634072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f1639d5e640 1 -- 172.21.15.60:0/1061606251 shutdown_connections 2023-11-18T20:43:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.799+0000 7f1639d5e640 1 -- 172.21.15.60:0/1061606251 wait complete. 2023-11-18T20:43:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.800+0000 7f1639d5e640 1 Processor -- start 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.800+0000 7f1639d5e640 1 -- start start 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.801+0000 7f1639d5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340d2910 0x7f16340cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.801+0000 7f1639d5e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 0x7f1634152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.801+0000 7f1639d5e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f163415d330 0x7f16340cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.801+0000 7f1639d5e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1634152dc0 con 0x7f16340d3900 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.801+0000 7f1639d5e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340cbdd0 con 0x7f16340d2910 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.801+0000 7f1639d5e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340cc070 con 0x7f163415d330 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.801+0000 7f1632ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 0x7f1634152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.801+0000 7f1632ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 0x7f1634152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50124/0 (socket says 172.21.15.60:50124) 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.801+0000 7f1632ffd640 1 -- 172.21.15.60:0/89934610 learned_addr learned my addr 172.21.15.60:0/89934610 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f16337fe640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340d2910 0x7f16340cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f1633fff640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f163415d330 0x7f16340cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f1632ffd640 1 -- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f163415d330 msgr2=0x7f16340cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f1632ffd640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f163415d330 0x7f16340cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f1632ffd640 1 -- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340d2910 msgr2=0x7f16340cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f1632ffd640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340d2910 0x7f16340cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f1632ffd640 1 -- 172.21.15.60:0/89934610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c009b30 con 0x7f16340d3900 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f1632ffd640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 0x7f1634152880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f162800c430 tx=0x7f162800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f161bfff640 1 -- 172.21.15.60:0/89934610 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628011790 con 0x7f16340d3900 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f161bfff640 1 -- 172.21.15.60:0/89934610 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16280118f0 con 0x7f16340d3900 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f161bfff640 1 -- 172.21.15.60:0/89934610 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628011ce0 con 0x7f16340d3900 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f1639d5e640 1 -- 172.21.15.60:0/89934610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16340cc350 con 0x7f16340d3900 2023-11-18T20:43:29.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.802+0000 7f1639d5e640 1 -- 172.21.15.60:0/89934610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163415ee50 con 0x7f16340d3900 2023-11-18T20:43:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.803+0000 7f1639d5e640 1 -- 172.21.15.60:0/89934610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163415e9d0 con 0x7f16340d3900 2023-11-18T20:43:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.804+0000 7f161bfff640 1 -- 172.21.15.60:0/89934610 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f162801c460 con 0x7f16340d3900 2023-11-18T20:43:29.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.804+0000 7f161bfff640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f162006ae40 0x7f162006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:29.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.804+0000 7f161bfff640 1 -- 172.21.15.60:0/89934610 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1628089260 con 0x7f16340d3900 2023-11-18T20:43:29.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.804+0000 7f16337fe640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f162006ae40 0x7f162006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:29.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.805+0000 7f16337fe640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f162006ae40 0x7f162006d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f162400a950 tx=0x7f1624008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:29.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:29.808+0000 7f161bfff640 1 -- 172.21.15.60:0/89934610 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f163415e9d0 con 0x7f16340d3900 2023-11-18T20:43:30.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.053+0000 7f1639d5e640 1 -- 172.21.15.60:0/89934610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f163415e9d0 con 0x7f16340d3900 2023-11-18T20:43:30.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.055+0000 7f161bfff640 1 -- 172.21.15.60:0/89934610 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f163415e9d0 con 0x7f16340d3900 2023-11-18T20:43:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 -- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f162006ae40 msgr2=0x7f162006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f162006ae40 0x7f162006d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f162400a950 tx=0x7f1624008040 comp rx=0 tx=0).stop 2023-11-18T20:43:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 -- 172.21.15.60:0/89934610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 msgr2=0x7f1634152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 0x7f1634152880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f162800c430 tx=0x7f162800c8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 -- 172.21.15.60:0/89934610 shutdown_connections 2023-11-18T20:43:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f162006ae40 0x7f162006d300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f163415d330 0x7f16340cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16340d3900 0x7f1634152880 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 --2- 172.21.15.60:0/89934610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16340d2910 0x7f16340cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 -- 172.21.15.60:0/89934610 >> 172.21.15.60:0/89934610 conn(0x7f1634072020 msgr2=0x7f16340d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 -- 172.21.15.60:0/89934610 shutdown_connections 2023-11-18T20:43:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.057+0000 7f1639d5e640 1 -- 172.21.15.60:0/89934610 wait complete. 2023-11-18T20:43:30.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-18T20:43:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.175+0000 7fa8f8a83640 1 Processor -- start 2023-11-18T20:43:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.175+0000 7fa8f8a83640 1 -- start start 2023-11-18T20:43:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f8a83640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa8f4142270 0x7fa8f41426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f8a83640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f4161d20 con 0x7fa8f406ea50 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f8a83640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f415e760 con 0x7fa8f4160fb0 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f8a83640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f40d1ee0 con 0x7fa8f4142270 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa8f406ea50 0x7fa8f406ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45482/0 (socket says 172.21.15.60:45482) 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f259c640 1 -- 172.21.15.60:0/1887053319 learned_addr learned my addr 172.21.15.60:0/1887053319 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f2d9d640 1 --2- 172.21.15.60:0/1887053319 >> v2:172.21.15.100:3300/0 conn(0x7fa8f4142270 0x7fa8f41426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f1d9b640 1 -- 172.21.15.60:0/1887053319 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619414375 0 0) 0x7fa8f4161d20 con 0x7fa8f406ea50 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f1d9b640 1 -- 172.21.15.60:0/1887053319 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8e0003520 con 0x7fa8f406ea50 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f2d9d640 1 -- 172.21.15.60:0/1887053319 >> v1:172.21.15.100:6789/0 conn(0x7fa8f4160fb0 legacy=0x7fa8f4161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f1d9b640 1 -- 172.21.15.60:0/1887053319 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1165314155 0 0) 0x7fa8e0003520 con 0x7fa8f406ea50 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f2d9d640 1 -- 172.21.15.60:0/1887053319 >> v1:172.21.15.60:6789/0 conn(0x7fa8f406ea50 legacy=0x7fa8f406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.176+0000 7fa8f2d9d640 1 -- 172.21.15.60:0/1887053319 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f415d9a0 con 0x7fa8f4142270 2023-11-18T20:43:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.177+0000 7fa8f2d9d640 1 --2- 172.21.15.60:0/1887053319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f4142270 0x7fa8f41426c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009d90 tx=0x7fa8ec02bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6addf09078959f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.177+0000 7fa8f1d9b640 1 -- 172.21.15.60:0/1887053319 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4001670 con 0x7fa8f4142270 2023-11-18T20:43:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.177+0000 7fa8f1d9b640 1 -- 172.21.15.60:0/1887053319 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec02d720 con 0x7fa8f4142270 2023-11-18T20:43:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.177+0000 7fa8f1d9b640 1 -- 172.21.15.60:0/1887053319 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec02dab0 con 0x7fa8f4142270 2023-11-18T20:43:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.178+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1887053319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f4142270 msgr2=0x7fa8f41426c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.178+0000 7fa8f8a83640 1 --2- 172.21.15.60:0/1887053319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f4142270 0x7fa8f41426c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009d90 tx=0x7fa8ec02bbe0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.178+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1887053319 shutdown_connections 2023-11-18T20:43:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.178+0000 7fa8f8a83640 1 --2- 172.21.15.60:0/1887053319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f4142270 0x7fa8f41426c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.178+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1887053319 >> 172.21.15.60:0/1887053319 conn(0x7fa8f406a6b0 msgr2=0x7fa8f406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.178+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1887053319 shutdown_connections 2023-11-18T20:43:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.178+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1887053319 wait complete. 2023-11-18T20:43:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.179+0000 7fa8f8a83640 1 Processor -- start 2023-11-18T20:43:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.179+0000 7fa8f8a83640 1 -- start start 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8f8a83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f406ea50 0x7fa8f4157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8f8a83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f4142270 0x7fa8f413f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8f8a83640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f4160fb0 0x7fa8f4155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8f8a83640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f413e4d0 con 0x7fa8f4142270 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8f8a83640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f413e720 con 0x7fa8f406ea50 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8f8a83640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f413ea00 con 0x7fa8f4160fb0 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f4142270 0x7fa8f413f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f4142270 0x7fa8f413f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50130/0 (socket says 172.21.15.60:50130) 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8ebfff640 1 -- 172.21.15.60:0/1220994533 learned_addr learned my addr 172.21.15.60:0/1220994533 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8f2d9d640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f4160fb0 0x7fa8f4155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8f259c640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f406ea50 0x7fa8f4157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8ebfff640 1 -- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f4160fb0 msgr2=0x7fa8f4155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8ebfff640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f4160fb0 0x7fa8f4155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8ebfff640 1 -- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f406ea50 msgr2=0x7fa8f4157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.180+0000 7fa8ebfff640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f406ea50 0x7fa8f4157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.181+0000 7fa8ebfff640 1 -- 172.21.15.60:0/1220994533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ec009a70 con 0x7fa8f4142270 2023-11-18T20:43:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.181+0000 7fa8ebfff640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f4142270 0x7fa8f413f870 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c780 tx=0x7fa8dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.181+0000 7fa8eaffd640 1 -- 172.21.15.60:0/1220994533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc005270 con 0x7fa8f4142270 2023-11-18T20:43:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.181+0000 7fa8eaffd640 1 -- 172.21.15.60:0/1220994533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8dc0053f0 con 0x7fa8f4142270 2023-11-18T20:43:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.181+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1220994533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f40b2d70 con 0x7fa8f4142270 2023-11-18T20:43:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.181+0000 7fa8eaffd640 1 -- 172.21.15.60:0/1220994533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00a880 con 0x7fa8f4142270 2023-11-18T20:43:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.181+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1220994533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f40b3330 con 0x7fa8f4142270 2023-11-18T20:43:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.182+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1220994533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f40b3520 con 0x7fa8f4142270 2023-11-18T20:43:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.182+0000 7fa8eaffd640 1 -- 172.21.15.60:0/1220994533 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa8dc00a9e0 con 0x7fa8f4142270 2023-11-18T20:43:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.183+0000 7fa8eaffd640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8d406ae70 0x7fa8d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.183+0000 7fa8eaffd640 1 -- 172.21.15.60:0/1220994533 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa8dc088970 con 0x7fa8f4142270 2023-11-18T20:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.183+0000 7fa8f259c640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8d406ae70 0x7fa8d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.184+0000 7fa8f259c640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8d406ae70 0x7fa8d406d330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4001eb0 tx=0x7fa8e4001c60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:30.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.187+0000 7fa8eaffd640 1 -- 172.21.15.60:0/1220994533 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8dc053d70 con 0x7fa8f4142270 2023-11-18T20:43:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.429+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1220994533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa8f4069400 con 0x7fa8f4142270 2023-11-18T20:43:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.431+0000 7fa8eaffd640 1 -- 172.21.15.60:0/1220994533 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa8dc017070 con 0x7fa8f4142270 2023-11-18T20:43:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8d406ae70 msgr2=0x7fa8d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8d406ae70 0x7fa8d406d330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4001eb0 tx=0x7fa8e4001c60 comp rx=0 tx=0).stop 2023-11-18T20:43:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1220994533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f4142270 msgr2=0x7fa8f413f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f4142270 0x7fa8f413f870 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00c780 tx=0x7fa8dc00cc40 comp rx=0 tx=0).stop 2023-11-18T20:43:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1220994533 shutdown_connections 2023-11-18T20:43:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa8d406ae70 0x7fa8d406d330 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8f4160fb0 0x7fa8f4155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8f4142270 0x7fa8f413f870 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 --2- 172.21.15.60:0/1220994533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8f406ea50 0x7fa8f4157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1220994533 >> 172.21.15.60:0/1220994533 conn(0x7fa8f406a6b0 msgr2=0x7fa8f40733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1220994533 shutdown_connections 2023-11-18T20:43:30.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.433+0000 7fa8f8a83640 1 -- 172.21.15.60:0/1220994533 wait complete. 2023-11-18T20:43:30.445 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:30.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.551+0000 7f2d54a61640 1 Processor -- start 2023-11-18T20:43:30.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.551+0000 7f2d54a61640 1 -- start start 2023-11-18T20:43:30.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d54a61640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2d500b1400 0x7f2d5015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d54a61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d5015e730 0x7f2d5015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d54a61640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d5015e2a0 con 0x7f2d50143490 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d54a61640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5015b520 con 0x7f2d500b1400 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d54a61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50143870 con 0x7f2d5015e730 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4e59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2d50143490 0x7f2d500b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45488/0 (socket says 172.21.15.60:45488) 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4e59c640 1 -- 172.21.15.60:0/1075170209 learned_addr learned my addr 172.21.15.60:0/1075170209 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d47fff640 1 --2- 172.21.15.60:0/1075170209 >> v2:172.21.15.100:3300/0 conn(0x7f2d500b1400 0x7f2d5015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4ed9d640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d5015e730 0x7f2d5015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844408017 0 0) 0x7f2d5015e2a0 con 0x7f2d50143490 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d3c003520 con 0x7f2d50143490 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1268735048 0 0) 0x7f2d3c003520 con 0x7f2d50143490 2023-11-18T20:43:30.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d5015e730 msgr2=0x7f2d5015aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d5015e730 0x7f2d5015aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 >> v2:172.21.15.100:3300/0 conn(0x7f2d500b1400 msgr2=0x7f2d5015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 --2- 172.21.15.60:0/1075170209 >> v2:172.21.15.100:3300/0 conn(0x7f2d500b1400 0x7f2d5015a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5015b240 con 0x7f2d50143490 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f2d40001670 con 0x7f2d50143490 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 >> v1:172.21.15.60:6789/0 conn(0x7f2d50143490 legacy=0x7f2d500b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c003970 0x7f2d3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d5015e730 0x7f2d3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d3c004d00 0x7f2d3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5015b520 con 0x7f2d3c004d00 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50143870 con 0x7f2d3c003970 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.552+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c00b980 con 0x7f2d5015e730 2023-11-18T20:43:30.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.553+0000 7f2d4ed9d640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d3c004d00 0x7f2d3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.553+0000 7f2d4e59c640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c003970 0x7f2d3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.553+0000 7f2d47fff640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d5015e730 0x7f2d3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.553+0000 7f2d4ed9d640 1 -- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d5015e730 msgr2=0x7f2d3c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.553+0000 7f2d4ed9d640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d5015e730 0x7f2d3c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.553+0000 7f2d4ed9d640 1 -- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c003970 msgr2=0x7f2d3c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.553+0000 7f2d4ed9d640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d3c003970 0x7f2d3c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.553+0000 7f2d4ed9d640 1 -- 172.21.15.60:0/1075170209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c003520 con 0x7f2d3c004d00 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d4ed9d640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d3c004d00 0x7f2d3c005180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2d48010480 tx=0x7f2d3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed3bd13e2f6971d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48037660 con 0x7f2d3c004d00 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d480377c0 con 0x7f2d3c004d00 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d4dd9b640 1 -- 172.21.15.60:0/1075170209 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48037b70 con 0x7f2d3c004d00 2023-11-18T20:43:30.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d54a61640 1 -- 172.21.15.60:0/1075170209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d3c004d00 msgr2=0x7f2d3c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d54a61640 1 --2- 172.21.15.60:0/1075170209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d3c004d00 0x7f2d3c005180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2d48010480 tx=0x7f2d3c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d4ed9d640 1 -- 172.21.15.60:0/1075170209 reap_dead start 2023-11-18T20:43:30.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d54a61640 1 -- 172.21.15.60:0/1075170209 shutdown_connections 2023-11-18T20:43:30.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d54a61640 1 -- 172.21.15.60:0/1075170209 >> 172.21.15.60:0/1075170209 conn(0x7f2d5006a4d0 msgr2=0x7f2d5006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:30.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d54a61640 1 -- 172.21.15.60:0/1075170209 shutdown_connections 2023-11-18T20:43:30.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.554+0000 7f2d54a61640 1 -- 172.21.15.60:0/1075170209 wait complete. 2023-11-18T20:43:30.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d54a61640 1 Processor -- start 2023-11-18T20:43:30.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d54a61640 1 -- start start 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d54a61640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500ade20 0x7f2d50133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d54a61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500af310 0x7f2d501f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d54a61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d501f4280 0x7f2d500afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d54a61640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d501f3360 con 0x7f2d500af310 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d54a61640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50134ea0 con 0x7f2d500ade20 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d54a61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500ae200 con 0x7f2d501f4280 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d47fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500af310 0x7f2d501f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d47fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500af310 0x7f2d501f3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50136/0 (socket says 172.21.15.60:50136) 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.556+0000 7f2d47fff640 1 -- 172.21.15.60:0/2862506409 learned_addr learned my addr 172.21.15.60:0/2862506409 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d4e59c640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500ade20 0x7f2d50133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d4ed9d640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d501f4280 0x7f2d500afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d47fff640 1 -- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d501f4280 msgr2=0x7f2d500afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d47fff640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d501f4280 0x7f2d500afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d47fff640 1 -- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500ade20 msgr2=0x7f2d50133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d47fff640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500ade20 0x7f2d50133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d47fff640 1 -- 172.21.15.60:0/2862506409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d501348e0 con 0x7f2d500af310 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d47fff640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500af310 0x7f2d501f3d40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2d38011a70 tx=0x7f2d38011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d46ffd640 1 -- 172.21.15.60:0/2862506409 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38009680 con 0x7f2d500af310 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d46ffd640 1 -- 172.21.15.60:0/2862506409 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3800fb50 con 0x7f2d500af310 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d54a61640 1 -- 172.21.15.60:0/2862506409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d50134bc0 con 0x7f2d500af310 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d46ffd640 1 -- 172.21.15.60:0/2862506409 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800fee0 con 0x7f2d500af310 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.557+0000 7f2d54a61640 1 -- 172.21.15.60:0/2862506409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d501f3840 con 0x7f2d500af310 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.558+0000 7f2d54a61640 1 -- 172.21.15.60:0/2862506409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d50069400 con 0x7f2d500af310 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.559+0000 7f2d46ffd640 1 -- 172.21.15.60:0/2862506409 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d38014440 con 0x7f2d500af310 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.559+0000 7f2d46ffd640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d3006ae40 0x7f2d3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.559+0000 7f2d46ffd640 1 -- 172.21.15.60:0/2862506409 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2d38089bf0 con 0x7f2d500af310 2023-11-18T20:43:30.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.559+0000 7f2d4e59c640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d3006ae40 0x7f2d3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.560+0000 7f2d4e59c640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d3006ae40 0x7f2d3006d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d40004840 tx=0x7f2d40009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:30.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.564+0000 7f2d46ffd640 1 -- 172.21.15.60:0/2862506409 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d38054ff0 con 0x7f2d500af310 2023-11-18T20:43:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.741+0000 7f2d54a61640 1 -- 172.21.15.60:0/2862506409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d500b2820 con 0x7f2d500af310 2023-11-18T20:43:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.741+0000 7f2d46ffd640 1 -- 172.21.15.60:0/2862506409 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12845 (secure 0 0 0) 0x7f2d38058610 con 0x7f2d500af310 2023-11-18T20:43:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:30.743 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":30,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:27.773930+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 -- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d3006ae40 msgr2=0x7f2d3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d3006ae40 0x7f2d3006d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d40004840 tx=0x7f2d40009040 comp rx=0 tx=0).stop 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 -- 172.21.15.60:0/2862506409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500af310 msgr2=0x7f2d501f3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500af310 0x7f2d501f3d40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2d38011a70 tx=0x7f2d38011f30 comp rx=0 tx=0).stop 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 -- 172.21.15.60:0/2862506409 shutdown_connections 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d3006ae40 0x7f2d3006d300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d501f4280 0x7f2d500afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d500af310 0x7f2d501f3d40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 --2- 172.21.15.60:0/2862506409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d500ade20 0x7f2d50133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 -- 172.21.15.60:0/2862506409 >> 172.21.15.60:0/2862506409 conn(0x7f2d5006a4d0 msgr2=0x7f2d500ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 -- 172.21.15.60:0/2862506409 shutdown_connections 2023-11-18T20:43:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.743+0000 7f2d54a61640 1 -- 172.21.15.60:0/2862506409 wait complete. 2023-11-18T20:43:30.756 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:30.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.864+0000 7f5fef989640 1 Processor -- start 2023-11-18T20:43:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.864+0000 7f5fef989640 1 -- start start 2023-11-18T20:43:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.865+0000 7f5fef989640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5fe815b5e0 0x7f5fe815ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.865+0000 7f5fef989640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fe806a4d0 0x7f5fe806a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.865+0000 7f5fef989640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe8155b70 con 0x7f5fe8155790 2023-11-18T20:43:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.865+0000 7f5fef989640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe806ee30 con 0x7f5fe806a4d0 2023-11-18T20:43:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.865+0000 7f5fef989640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe806f3b0 con 0x7f5fe815b5e0 2023-11-18T20:43:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.865+0000 7f5fedf26640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fe806a4d0 0x7f5fe806a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.865+0000 7f5fed725640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5fe8155790 0x7f5fe806f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36968/0 (socket says 172.21.15.60:36968) 2023-11-18T20:43:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.865+0000 7f5fed725640 1 -- 172.21.15.60:0/2085198979 learned_addr learned my addr 172.21.15.60:0/2085198979 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.865+0000 7f5fecf24640 1 --2- 172.21.15.60:0/2085198979 >> v2:172.21.15.100:3300/0 conn(0x7f5fe815b5e0 0x7f5fe815ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.866+0000 7f5fedf26640 1 -- 172.21.15.60:0/2085198979 >> v2:172.21.15.100:3300/0 conn(0x7f5fe815b5e0 msgr2=0x7f5fe815ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.866+0000 7f5fedf26640 1 --2- 172.21.15.60:0/2085198979 >> v2:172.21.15.100:3300/0 conn(0x7f5fe815b5e0 0x7f5fe815ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.866+0000 7f5fedf26640 1 -- 172.21.15.60:0/2085198979 >> v1:172.21.15.60:6789/0 conn(0x7f5fe8155790 legacy=0x7f5fe806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.866+0000 7f5fedf26640 1 -- 172.21.15.60:0/2085198979 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe806adf0 con 0x7f5fe806a4d0 2023-11-18T20:43:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.866+0000 7f5fe7fff640 1 -- 172.21.15.60:0/2085198979 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034739569 0 0) 0x7f5fe8155b70 con 0x7f5fe8155790 2023-11-18T20:43:30.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.866+0000 7f5fedf26640 1 --2- 172.21.15.60:0/2085198979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe806a4d0 0x7f5fe806a8b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8071430 tx=0x7f5fd802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1b39b9646a523b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:30.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.866+0000 7f5fe7fff640 1 -- 172.21.15.60:0/2085198979 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd802cc80 con 0x7f5fe806a4d0 2023-11-18T20:43:30.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.867+0000 7f5fe7fff640 1 -- 172.21.15.60:0/2085198979 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd802cde0 con 0x7f5fe806a4d0 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.867+0000 7f5fe7fff640 1 -- 172.21.15.60:0/2085198979 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8034660 con 0x7f5fe806a4d0 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.867+0000 7f5fef989640 1 -- 172.21.15.60:0/2085198979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe806a4d0 msgr2=0x7f5fe806a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.867+0000 7f5fef989640 1 --2- 172.21.15.60:0/2085198979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe806a4d0 0x7f5fe806a8b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8071430 tx=0x7f5fd802bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.867+0000 7f5fef989640 1 -- 172.21.15.60:0/2085198979 shutdown_connections 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.867+0000 7f5fef989640 1 --2- 172.21.15.60:0/2085198979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe806a4d0 0x7f5fe806a8b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.867+0000 7f5fef989640 1 --2- 172.21.15.60:0/2085198979 >> v2:172.21.15.100:3300/0 conn(0x7f5fe815b5e0 0x7f5fe815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.867+0000 7f5fef989640 1 -- 172.21.15.60:0/2085198979 >> 172.21.15.60:0/2085198979 conn(0x7f5fe80b1e50 msgr2=0x7f5fe80b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.867+0000 7f5fef989640 1 -- 172.21.15.60:0/2085198979 shutdown_connections 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.867+0000 7f5fef989640 1 -- 172.21.15.60:0/2085198979 wait complete. 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.868+0000 7f5fef989640 1 Processor -- start 2023-11-18T20:43:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.868+0000 7f5fef989640 1 -- start start 2023-11-18T20:43:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fef989640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fe806a4d0 0x7f5fe813d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fef989640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8155790 0x7f5fe814ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fef989640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe815b5e0 0x7f5fe814f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fef989640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe814cc80 con 0x7f5fe815b5e0 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fef989640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe813d570 con 0x7f5fe8155790 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fef989640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8155f90 con 0x7f5fe806a4d0 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fedf26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe815b5e0 0x7f5fe814f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fedf26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe815b5e0 0x7f5fe814f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44524/0 (socket says 172.21.15.60:44524) 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fedf26640 1 -- 172.21.15.60:0/169247631 learned_addr learned my addr 172.21.15.60:0/169247631 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fecf24640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8155790 0x7f5fe814ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.869+0000 7f5fed725640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fe806a4d0 0x7f5fe813d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fedf26640 1 -- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fe806a4d0 msgr2=0x7f5fe813d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fedf26640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fe806a4d0 0x7f5fe813d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fedf26640 1 -- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8155790 msgr2=0x7f5fe814ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fedf26640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8155790 0x7f5fe814ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fedf26640 1 -- 172.21.15.60:0/169247631 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd8009a70 con 0x7f5fe815b5e0 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fedf26640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe815b5e0 0x7f5fe814f3a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009e80 tx=0x7f5fd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fe5ffb640 1 -- 172.21.15.60:0/169247631 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8040650 con 0x7f5fe815b5e0 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fe5ffb640 1 -- 172.21.15.60:0/169247631 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd80407b0 con 0x7f5fe815b5e0 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fef989640 1 -- 172.21.15.60:0/169247631 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe8150010 con 0x7f5fe815b5e0 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fe5ffb640 1 -- 172.21.15.60:0/169247631 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8040b60 con 0x7f5fe815b5e0 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.870+0000 7f5fef989640 1 -- 172.21.15.60:0/169247631 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe81505b0 con 0x7f5fe815b5e0 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.871+0000 7f5fef989640 1 -- 172.21.15.60:0/169247631 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe8069400 con 0x7f5fe815b5e0 2023-11-18T20:43:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.872+0000 7f5fe5ffb640 1 -- 172.21.15.60:0/169247631 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5fd8040cc0 con 0x7f5fe815b5e0 2023-11-18T20:43:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.872+0000 7f5fe5ffb640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5fd406ae40 0x7f5fd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.872+0000 7f5fe5ffb640 1 -- 172.21.15.60:0/169247631 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5fd80abf30 con 0x7f5fe815b5e0 2023-11-18T20:43:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.872+0000 7f5fed725640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5fd406ae40 0x7f5fd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.873+0000 7f5fed725640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5fd406ae40 0x7f5fd406d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0000f60 tx=0x7f5fe0007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:30.876+0000 7f5fe5ffb640 1 -- 172.21.15.60:0/169247631 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5fd80773e0 con 0x7f5fe815b5e0 2023-11-18T20:43:31.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.050+0000 7f5fef989640 1 -- 172.21.15.60:0/169247631 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fe80b2c50 con 0x7f5fe815b5e0 2023-11-18T20:43:31.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.050+0000 7f5fe5ffb640 1 -- 172.21.15.60:0/169247631 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fd8009450 con 0x7f5fe815b5e0 2023-11-18T20:43:31.052 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 -- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5fd406ae40 msgr2=0x7f5fd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5fd406ae40 0x7f5fd406d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0000f60 tx=0x7f5fe0007d50 comp rx=0 tx=0).stop 2023-11-18T20:43:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 -- 172.21.15.60:0/169247631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe815b5e0 msgr2=0x7f5fe814f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe815b5e0 0x7f5fe814f3a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009e80 tx=0x7f5fd8004a40 comp rx=0 tx=0).stop 2023-11-18T20:43:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 -- 172.21.15.60:0/169247631 shutdown_connections 2023-11-18T20:43:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5fd406ae40 0x7f5fd406d300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.054 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fe815b5e0 0x7f5fe814f3a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fe8155790 0x7f5fe814ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 --2- 172.21.15.60:0/169247631 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fe806a4d0 0x7f5fe813d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 -- 172.21.15.60:0/169247631 >> 172.21.15.60:0/169247631 conn(0x7f5fe80b1e50 msgr2=0x7f5fe806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 -- 172.21.15.60:0/169247631 shutdown_connections 2023-11-18T20:43:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.052+0000 7f5fef989640 1 -- 172.21.15.60:0/169247631 wait complete. 2023-11-18T20:43:31.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-18T20:43:31.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.170+0000 7f630ceaf640 1 Processor -- start 2023-11-18T20:43:31.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.170+0000 7f630ceaf640 1 -- start start 2023-11-18T20:43:31.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.170+0000 7f630ceaf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f630815d330 0x7f63080b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.170+0000 7f630ceaf640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f63080d2910 0x7f63080d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.170+0000 7f630ceaf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63080d2d90 0x7f63080d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.170+0000 7f630ceaf640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63080b2b10 con 0x7f63080d2d90 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.170+0000 7f630ceaf640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63080d1590 con 0x7f63080d2910 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.170+0000 7f630ceaf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63080d1870 con 0x7f630815d330 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f6306d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63080d2d90 0x7f63080d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f6306d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63080d2d90 0x7f63080d3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44536/0 (socket says 172.21.15.60:44536) 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f6306d9d640 1 -- 172.21.15.60:0/4083444075 learned_addr learned my addr 172.21.15.60:0/4083444075 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f62fffff640 1 --2- 172.21.15.60:0/4083444075 >> v2:172.21.15.100:3300/0 conn(0x7f63080d2910 0x7f63080d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f630659c640 1 --2- 172.21.15.60:0/4083444075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f630815d330 0x7f63080b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f6306d9d640 1 -- 172.21.15.60:0/4083444075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f630815d330 msgr2=0x7f63080b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f6306d9d640 1 --2- 172.21.15.60:0/4083444075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f630815d330 0x7f63080b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f6306d9d640 1 -- 172.21.15.60:0/4083444075 >> v2:172.21.15.100:3300/0 conn(0x7f63080d2910 msgr2=0x7f63080d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:31.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f6306d9d640 1 --2- 172.21.15.60:0/4083444075 >> v2:172.21.15.100:3300/0 conn(0x7f63080d2910 0x7f63080d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f6306d9d640 1 -- 172.21.15.60:0/4083444075 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63080d1b50 con 0x7f63080d2d90 2023-11-18T20:43:31.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.171+0000 7f62fffff640 1 --2- 172.21.15.60:0/4083444075 >> v2:172.21.15.100:3300/0 conn(0x7f63080d2910 0x7f63080d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:31.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f6306d9d640 1 --2- 172.21.15.60:0/4083444075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2d90 0x7f63080d3730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f63080d4360 tx=0x7f630002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bf3f7a9cc7d618 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:31.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f6305d9b640 1 -- 172.21.15.60:0/4083444075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630002c9e0 con 0x7f63080d2d90 2023-11-18T20:43:31.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f6305d9b640 1 -- 172.21.15.60:0/4083444075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630002cb40 con 0x7f63080d2d90 2023-11-18T20:43:31.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f6305d9b640 1 -- 172.21.15.60:0/4083444075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63000343f0 con 0x7f63080d2d90 2023-11-18T20:43:31.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f630ceaf640 1 -- 172.21.15.60:0/4083444075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2d90 msgr2=0x7f63080d3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f630ceaf640 1 --2- 172.21.15.60:0/4083444075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2d90 0x7f63080d3730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f63080d4360 tx=0x7f630002be40 comp rx=0 tx=0).stop 2023-11-18T20:43:31.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f630ceaf640 1 -- 172.21.15.60:0/4083444075 shutdown_connections 2023-11-18T20:43:31.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f630ceaf640 1 --2- 172.21.15.60:0/4083444075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2d90 0x7f63080d3730 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f630ceaf640 1 --2- 172.21.15.60:0/4083444075 >> v2:172.21.15.100:3300/0 conn(0x7f63080d2910 0x7f63080d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f630ceaf640 1 --2- 172.21.15.60:0/4083444075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f630815d330 0x7f63080b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f630ceaf640 1 -- 172.21.15.60:0/4083444075 >> 172.21.15.60:0/4083444075 conn(0x7f6308072020 msgr2=0x7f6308072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:31.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.172+0000 7f630ceaf640 1 -- 172.21.15.60:0/4083444075 shutdown_connections 2023-11-18T20:43:31.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.173+0000 7f630ceaf640 1 -- 172.21.15.60:0/4083444075 wait complete. 2023-11-18T20:43:31.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.174+0000 7f630ceaf640 1 Processor -- start 2023-11-18T20:43:31.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.174+0000 7f630ceaf640 1 -- start start 2023-11-18T20:43:31.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.174+0000 7f630ceaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2910 0x7f6308150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.174+0000 7f630ceaf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63080d2d90 0x7f63080cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630ceaf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f630815d330 0x7f63080b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.178 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630ceaf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63080cd240 con 0x7f63080d2910 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630ceaf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63081f2810 con 0x7f63080d2d90 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630ceaf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63080ccf60 con 0x7f630815d330 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2910 0x7f6308150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2910 0x7f6308150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44546/0 (socket says 172.21.15.60:44546) 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630659c640 1 -- 172.21.15.60:0/1956211254 learned_addr learned my addr 172.21.15.60:0/1956211254 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f6306d9d640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f630815d330 0x7f63080b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f62fffff640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63080d2d90 0x7f63080cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630659c640 1 -- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f630815d330 msgr2=0x7f63080b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630659c640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f630815d330 0x7f63080b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630659c640 1 -- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63080d2d90 msgr2=0x7f63080cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630659c640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63080d2d90 0x7f63080cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630659c640 1 -- 172.21.15.60:0/1956211254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6300009b30 con 0x7f63080d2910 2023-11-18T20:43:31.179 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630659c640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2910 0x7f6308150df0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f62f800d5f0 tx=0x7f62f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:31.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f62feffd640 1 -- 172.21.15.60:0/1956211254 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80125f0 con 0x7f63080d2910 2023-11-18T20:43:31.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f62feffd640 1 -- 172.21.15.60:0/1956211254 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f8012750 con 0x7f63080d2910 2023-11-18T20:43:31.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f62feffd640 1 -- 172.21.15.60:0/1956211254 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8012b00 con 0x7f63080d2910 2023-11-18T20:43:31.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630ceaf640 1 -- 172.21.15.60:0/1956211254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63080ca460 con 0x7f63080d2910 2023-11-18T20:43:31.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.175+0000 7f630ceaf640 1 -- 172.21.15.60:0/1956211254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63080caa20 con 0x7f63080d2910 2023-11-18T20:43:31.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.176+0000 7f630ceaf640 1 -- 172.21.15.60:0/1956211254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6308069400 con 0x7f63080d2910 2023-11-18T20:43:31.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.177+0000 7f62feffd640 1 -- 172.21.15.60:0/1956211254 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62f8012c60 con 0x7f63080d2910 2023-11-18T20:43:31.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.177+0000 7f62feffd640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f62e806ae40 0x7f62e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.177+0000 7f62feffd640 1 -- 172.21.15.60:0/1956211254 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f62f8089660 con 0x7f63080d2910 2023-11-18T20:43:31.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.177+0000 7f62fffff640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f62e806ae40 0x7f62e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.178+0000 7f62fffff640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f62e806ae40 0x7f62e806d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f63080b2750 tx=0x7f62f0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.181+0000 7f62feffd640 1 -- 172.21.15.60:0/1956211254 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62f8054ba0 con 0x7f63080d2910 2023-11-18T20:43:31.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.359+0000 7f630ceaf640 1 -- 172.21.15.60:0/1956211254 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f630815cc90 con 0x7f63080d2910 2023-11-18T20:43:31.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.360+0000 7f62feffd640 1 -- 172.21.15.60:0/1956211254 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f62f80581c0 con 0x7f63080d2910 2023-11-18T20:43:31.361 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:31.361 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-11-18T20:43:31.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:43:31.361 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-11-18T20:43:31.361 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-11-18T20:43:31.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:31.361 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-11-18T20:43:31.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 -- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f62e806ae40 msgr2=0x7f62e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f62e806ae40 0x7f62e806d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f63080b2750 tx=0x7f62f0008040 comp rx=0 tx=0).stop 2023-11-18T20:43:31.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 -- 172.21.15.60:0/1956211254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2910 msgr2=0x7f6308150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2910 0x7f6308150df0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f62f800d5f0 tx=0x7f62f800dab0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 -- 172.21.15.60:0/1956211254 shutdown_connections 2023-11-18T20:43:31.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f62e806ae40 0x7f62e806d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f630815d330 0x7f63080b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63080d2d90 0x7f63080cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 --2- 172.21.15.60:0/1956211254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63080d2910 0x7f6308150df0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 -- 172.21.15.60:0/1956211254 >> 172.21.15.60:0/1956211254 conn(0x7f6308072020 msgr2=0x7f63080cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:31.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 -- 172.21.15.60:0/1956211254 shutdown_connections 2023-11-18T20:43:31.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.361+0000 7f630ceaf640 1 -- 172.21.15.60:0/1956211254 wait complete. 2023-11-18T20:43:31.373 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-18T20:43:31.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-18T20:43:31.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.482+0000 7f39a6d3a640 1 Processor -- start 2023-11-18T20:43:31.484 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.483+0000 7f39a6d3a640 1 -- start start 2023-11-18T20:43:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.483+0000 7f39a6d3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015d330 0x7f39a00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.483+0000 7f39a6d3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39a00d2910 0x7f39a00d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.483+0000 7f39a6d3a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f39a00d2d90 0x7f39a00d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.483+0000 7f39a6d3a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a00b2b10 con 0x7f39a00d2910 2023-11-18T20:43:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.483+0000 7f39a6d3a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a00d1590 con 0x7f39a00d2d90 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.483+0000 7f39a6d3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00d1870 con 0x7f39a015d330 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.483+0000 7f399ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39a00d2910 0x7f39a00d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.483+0000 7f399ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39a00d2910 0x7f39a00d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44550/0 (socket says 172.21.15.60:44550) 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.484+0000 7f399ffff640 1 -- 172.21.15.60:0/2799841910 learned_addr learned my addr 172.21.15.60:0/2799841910 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.484+0000 7f39a52d7640 1 --2- 172.21.15.60:0/2799841910 >> v2:172.21.15.100:3300/0 conn(0x7f39a00d2d90 0x7f39a00d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.484+0000 7f39a4ad6640 1 --2- 172.21.15.60:0/2799841910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015d330 0x7f39a00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.484+0000 7f399ffff640 1 -- 172.21.15.60:0/2799841910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015d330 msgr2=0x7f39a00b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.484+0000 7f399ffff640 1 --2- 172.21.15.60:0/2799841910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015d330 0x7f39a00b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.484+0000 7f399ffff640 1 -- 172.21.15.60:0/2799841910 >> v2:172.21.15.100:3300/0 conn(0x7f39a00d2d90 msgr2=0x7f39a00d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.484+0000 7f399ffff640 1 --2- 172.21.15.60:0/2799841910 >> v2:172.21.15.100:3300/0 conn(0x7f39a00d2d90 0x7f39a00d3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.484+0000 7f399ffff640 1 -- 172.21.15.60:0/2799841910 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00d1b50 con 0x7f39a00d2910 2023-11-18T20:43:31.486 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.484+0000 7f39a4ad6640 1 --2- 172.21.15.60:0/2799841910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015d330 0x7f39a00b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:31.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.484+0000 7f39a52d7640 1 --2- 172.21.15.60:0/2799841910 >> v2:172.21.15.100:3300/0 conn(0x7f39a00d2d90 0x7f39a00d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f399ffff640 1 --2- 172.21.15.60:0/2799841910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d2910 0x7f39a00d31f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f39a00d1540 tx=0x7f398c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2736f86879cd29 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f399f7fe640 1 -- 172.21.15.60:0/2799841910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c0317d0 con 0x7f39a00d2910 2023-11-18T20:43:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f399f7fe640 1 -- 172.21.15.60:0/2799841910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c031930 con 0x7f39a00d2910 2023-11-18T20:43:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f399f7fe640 1 -- 172.21.15.60:0/2799841910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c031cc0 con 0x7f39a00d2910 2023-11-18T20:43:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f39a6d3a640 1 -- 172.21.15.60:0/2799841910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d2910 msgr2=0x7f39a00d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f39a6d3a640 1 --2- 172.21.15.60:0/2799841910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d2910 0x7f39a00d31f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f39a00d1540 tx=0x7f398c02b870 comp rx=0 tx=0).stop 2023-11-18T20:43:31.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f39a6d3a640 1 -- 172.21.15.60:0/2799841910 shutdown_connections 2023-11-18T20:43:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f39a6d3a640 1 --2- 172.21.15.60:0/2799841910 >> v2:172.21.15.100:3300/0 conn(0x7f39a00d2d90 0x7f39a00d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f39a6d3a640 1 --2- 172.21.15.60:0/2799841910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d2910 0x7f39a00d31f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f39a6d3a640 1 --2- 172.21.15.60:0/2799841910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015d330 0x7f39a00b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f39a6d3a640 1 -- 172.21.15.60:0/2799841910 >> 172.21.15.60:0/2799841910 conn(0x7f39a0072020 msgr2=0x7f39a0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f39a6d3a640 1 -- 172.21.15.60:0/2799841910 shutdown_connections 2023-11-18T20:43:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.485+0000 7f39a6d3a640 1 -- 172.21.15.60:0/2799841910 wait complete. 2023-11-18T20:43:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.486+0000 7f39a6d3a640 1 Processor -- start 2023-11-18T20:43:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.487+0000 7f39a6d3a640 1 -- start start 2023-11-18T20:43:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a6d3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00d2910 0x7f39a01f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a6d3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d2d90 0x7f39a00c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a6d3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a015d330 0x7f39a015f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a6d3a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00cad10 con 0x7f39a015d330 2023-11-18T20:43:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a6d3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00c87c0 con 0x7f39a00d2d90 2023-11-18T20:43:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a6d3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00c7f20 con 0x7f39a00d2910 2023-11-18T20:43:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a52d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a015d330 0x7f39a015f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a52d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a015d330 0x7f39a015f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44556/0 (socket says 172.21.15.60:44556) 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a52d7640 1 -- 172.21.15.60:0/4127963324 learned_addr learned my addr 172.21.15.60:0/4127963324 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a4ad6640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00d2910 0x7f39a01f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f399ffff640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d2d90 0x7f39a00c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a52d7640 1 -- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00d2910 msgr2=0x7f39a01f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a52d7640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00d2910 0x7f39a01f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a52d7640 1 -- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d2d90 msgr2=0x7f39a00c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a52d7640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d2d90 0x7f39a00c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a52d7640 1 -- 172.21.15.60:0/4127963324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0094d0 con 0x7f39a015d330 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.488+0000 7f39a52d7640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a015d330 0x7f39a015f4f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f399800ce40 tx=0x7f399800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.489+0000 7f399d7fa640 1 -- 172.21.15.60:0/4127963324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800d840 con 0x7f39a015d330 2023-11-18T20:43:31.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.489+0000 7f399d7fa640 1 -- 172.21.15.60:0/4127963324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399800d9a0 con 0x7f39a015d330 2023-11-18T20:43:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.489+0000 7f399d7fa640 1 -- 172.21.15.60:0/4127963324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800dd50 con 0x7f39a015d330 2023-11-18T20:43:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.489+0000 7f39a6d3a640 1 -- 172.21.15.60:0/4127963324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a00c8200 con 0x7f39a015d330 2023-11-18T20:43:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.489+0000 7f39a6d3a640 1 -- 172.21.15.60:0/4127963324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a00caf40 con 0x7f39a015d330 2023-11-18T20:43:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.490+0000 7f39a6d3a640 1 -- 172.21.15.60:0/4127963324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a00cb130 con 0x7f39a015d330 2023-11-18T20:43:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.490+0000 7f399d7fa640 1 -- 172.21.15.60:0/4127963324 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f399801e3b0 con 0x7f39a015d330 2023-11-18T20:43:31.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.491+0000 7f399d7fa640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f398406ae40 0x7f398406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.491+0000 7f399d7fa640 1 -- 172.21.15.60:0/4127963324 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3998089c10 con 0x7f39a015d330 2023-11-18T20:43:31.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.491+0000 7f39a4ad6640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f398406ae40 0x7f398406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.491+0000 7f39a4ad6640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f398406ae40 0x7f398406d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f399400ab60 tx=0x7f3994008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:31.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.495+0000 7f399d7fa640 1 -- 172.21.15.60:0/4127963324 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f39980550c0 con 0x7f39a015d330 2023-11-18T20:43:31.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.672+0000 7f39a6d3a640 1 -- 172.21.15.60:0/4127963324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f39a0069400 con 0x7f39a015d330 2023-11-18T20:43:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.766+0000 7f399d7fa640 1 -- 172.21.15.60:0/4127963324 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f399801a0b0 con 0x7f39a015d330 2023-11-18T20:43:31.768 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-11-18T20:43:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.768+0000 7f39a6d3a640 1 -- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f398406ae40 msgr2=0x7f398406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.768+0000 7f39a6d3a640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f398406ae40 0x7f398406d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f399400ab60 tx=0x7f3994008040 comp rx=0 tx=0).stop 2023-11-18T20:43:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.768+0000 7f39a6d3a640 1 -- 172.21.15.60:0/4127963324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a015d330 msgr2=0x7f39a015f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.768+0000 7f39a6d3a640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a015d330 0x7f39a015f4f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f399800ce40 tx=0x7f399800a710 comp rx=0 tx=0).stop 2023-11-18T20:43:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.769+0000 7f39a6d3a640 1 -- 172.21.15.60:0/4127963324 shutdown_connections 2023-11-18T20:43:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.769+0000 7f39a6d3a640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f398406ae40 0x7f398406d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.769+0000 7f39a6d3a640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a015d330 0x7f39a015f4f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.769+0000 7f39a6d3a640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d2d90 0x7f39a00c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.769+0000 7f39a6d3a640 1 --2- 172.21.15.60:0/4127963324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a00d2910 0x7f39a01f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.769+0000 7f39a6d3a640 1 -- 172.21.15.60:0/4127963324 >> 172.21.15.60:0/4127963324 conn(0x7f39a0072020 msgr2=0x7f39a00b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.769+0000 7f39a6d3a640 1 -- 172.21.15.60:0/4127963324 shutdown_connections 2023-11-18T20:43:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.769+0000 7f39a6d3a640 1 -- 172.21.15.60:0/4127963324 wait complete. 2023-11-18T20:43:31.783 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-18T20:43:31.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.896+0000 7f42ff42d640 1 Processor -- start 2023-11-18T20:43:31.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.896+0000 7f42ff42d640 1 -- start start 2023-11-18T20:43:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.897+0000 7f42ff42d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f42f80b8ec0 0x7f42f80d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.897+0000 7f42ff42d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f815d8b0 con 0x7f42f80b4cd0 2023-11-18T20:43:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.897+0000 7f42ff42d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f80d1ae0 con 0x7f42f815d4d0 2023-11-18T20:43:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.897+0000 7f42ff42d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f80d1dc0 con 0x7f42f80b8ec0 2023-11-18T20:43:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.897+0000 7f42fd9ca640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f42f80b4cd0 0x7f42f80b6380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36976/0 (socket says 172.21.15.60:36976) 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.897+0000 7f42fd9ca640 1 -- 172.21.15.60:0/3561705453 learned_addr learned my addr 172.21.15.60:0/3561705453 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.897+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969823092 0 0) 0x7f42f815d8b0 con 0x7f42f80b4cd0 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.897+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42e0003520 con 0x7f42f80b4cd0 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.897+0000 7f42fc9c8640 1 --2- 172.21.15.60:0/3561705453 >> v2:172.21.15.100:3300/0 conn(0x7f42f80b8ec0 0x7f42f80d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1741693985 0 0) 0x7f42e0003520 con 0x7f42f80b4cd0 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 >> v2:172.21.15.100:3300/0 conn(0x7f42f80b8ec0 msgr2=0x7f42f80d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 --2- 172.21.15.60:0/3561705453 >> v2:172.21.15.100:3300/0 conn(0x7f42f80b8ec0 0x7f42f80d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 >> v1:172.21.15.100:6789/0 conn(0x7f42f815d4d0 legacy=0x7f42f80d1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80d20a0 con 0x7f42f80b4cd0 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f42f0001670 con 0x7f42f80b4cd0 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 >> v1:172.21.15.60:6789/0 conn(0x7f42f80b4cd0 legacy=0x7f42f80b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e0003970 0x7f42e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e00047c0 0x7f42e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e0005180 0x7f42e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80d1dc0 con 0x7f42e0005180 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e000c020 con 0x7f42e00047c0 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e000c1e0 con 0x7f42e0003970 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42fd9ca640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e0005180 0x7f42e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42fc9c8640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e00047c0 0x7f42e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42fd1c9640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e0003970 0x7f42e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42fd9ca640 1 -- 172.21.15.60:0/3561705453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e0003970 msgr2=0x7f42e0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42fd9ca640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e0003970 0x7f42e0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42fd9ca640 1 -- 172.21.15.60:0/3561705453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e00047c0 msgr2=0x7f42e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:31.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42fd9ca640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e00047c0 0x7f42e0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.898+0000 7f42fd9ca640 1 -- 172.21.15.60:0/3561705453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e0003520 con 0x7f42e0005180 2023-11-18T20:43:31.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.899+0000 7f42fd9ca640 1 -- 172.21.15.60:0/3561705453 reap_dead start 2023-11-18T20:43:31.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.899+0000 7f42fd9ca640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e0005180 0x7f42e000b980 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f42f000c240 tx=0x7f42e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8bcc884a156514 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:31.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.899+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0005ad0 con 0x7f42e0005180 2023-11-18T20:43:31.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.899+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f0005c70 con 0x7f42e0005180 2023-11-18T20:43:31.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.899+0000 7f42f7fff640 1 -- 172.21.15.60:0/3561705453 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f002c8b0 con 0x7f42e0005180 2023-11-18T20:43:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.900+0000 7f42ff42d640 1 -- 172.21.15.60:0/3561705453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e0005180 msgr2=0x7f42e000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.900+0000 7f42ff42d640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e0005180 0x7f42e000b980 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f42f000c240 tx=0x7f42e00057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.900+0000 7f42ff42d640 1 -- 172.21.15.60:0/3561705453 shutdown_connections 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.900+0000 7f42ff42d640 1 --2- 172.21.15.60:0/3561705453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e0005180 0x7f42e000b980 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.900+0000 7f42ff42d640 1 -- 172.21.15.60:0/3561705453 >> 172.21.15.60:0/3561705453 conn(0x7f42f8072230 msgr2=0x7f42f80cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.900+0000 7f42ff42d640 1 -- 172.21.15.60:0/3561705453 shutdown_connections 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.900+0000 7f42ff42d640 1 -- 172.21.15.60:0/3561705453 wait complete. 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.901+0000 7f42ff42d640 1 Processor -- start 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.901+0000 7f42ff42d640 1 -- start start 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42ff42d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e0005180 0x7f42f814a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42ff42d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f81f3770 0x7f42f80cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42ff42d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f80b7390 0x7f42f80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42ff42d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f814af30 con 0x7f42f81f3770 2023-11-18T20:43:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42ff42d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80b7d60 con 0x7f42f80b7390 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42ff42d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80cb970 con 0x7f42e0005180 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42fc9c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f81f3770 0x7f42f80cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42fc9c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f81f3770 0x7f42f80cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44564/0 (socket says 172.21.15.60:44564) 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42fc9c8640 1 -- 172.21.15.60:0/3887437414 learned_addr learned my addr 172.21.15.60:0/3887437414 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42fd9ca640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f80b7390 0x7f42f80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42fd1c9640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e0005180 0x7f42f814a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42fc9c8640 1 -- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e0005180 msgr2=0x7f42f814a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42fc9c8640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e0005180 0x7f42f814a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42fc9c8640 1 -- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f80b7390 msgr2=0x7f42f80b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42fc9c8640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f80b7390 0x7f42f80b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.902+0000 7f42fc9c8640 1 -- 172.21.15.60:0/3887437414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80cb690 con 0x7f42f81f3770 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.903+0000 7f42fd9ca640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f80b7390 0x7f42f80b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:31.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.903+0000 7f42fc9c8640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f81f3770 0x7f42f80cad40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0118e0 tx=0x7f42ec011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.903+0000 7f42f5ffb640 1 -- 172.21.15.60:0/3887437414 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec00faf0 con 0x7f42f81f3770 2023-11-18T20:43:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.903+0000 7f42f5ffb640 1 -- 172.21.15.60:0/3887437414 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec00fc50 con 0x7f42f81f3770 2023-11-18T20:43:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.903+0000 7f42ff42d640 1 -- 172.21.15.60:0/3887437414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f815ed80 con 0x7f42f81f3770 2023-11-18T20:43:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.903+0000 7f42f5ffb640 1 -- 172.21.15.60:0/3887437414 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec014440 con 0x7f42f81f3770 2023-11-18T20:43:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.903+0000 7f42ff42d640 1 -- 172.21.15.60:0/3887437414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f815f340 con 0x7f42f81f3770 2023-11-18T20:43:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.904+0000 7f42ff42d640 1 -- 172.21.15.60:0/3887437414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f8069400 con 0x7f42f81f3770 2023-11-18T20:43:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.905+0000 7f42f5ffb640 1 -- 172.21.15.60:0/3887437414 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f42ec01d400 con 0x7f42f81f3770 2023-11-18T20:43:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.905+0000 7f42f5ffb640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42e406ae70 0x7f42e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.905+0000 7f42f5ffb640 1 -- 172.21.15.60:0/3887437414 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f42ec089ce0 con 0x7f42f81f3770 2023-11-18T20:43:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.905+0000 7f42fd1c9640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42e406ae70 0x7f42e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.906+0000 7f42fd1c9640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42e406ae70 0x7f42e406d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f42e8000f80 tx=0x7f42e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:31.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:31.910+0000 7f42f5ffb640 1 -- 172.21.15.60:0/3887437414 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42ec054f70 con 0x7f42f81f3770 2023-11-18T20:43:32.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.091+0000 7f42ff42d640 1 -- 172.21.15.60:0/3887437414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f42f80d13a0 con 0x7f42f81f3770 2023-11-18T20:43:32.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.769+0000 7f42f5ffb640 1 -- 172.21.15.60:0/3887437414 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f42ec058590 con 0x7f42f81f3770 2023-11-18T20:43:32.771 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-11-18T20:43:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.775+0000 7f42ff42d640 1 -- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42e406ae70 msgr2=0x7f42e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.775+0000 7f42ff42d640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42e406ae70 0x7f42e406d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f42e8000f80 tx=0x7f42e8009040 comp rx=0 tx=0).stop 2023-11-18T20:43:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.776+0000 7f42ff42d640 1 -- 172.21.15.60:0/3887437414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f81f3770 msgr2=0x7f42f80cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:32.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.776+0000 7f42ff42d640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f81f3770 0x7f42f80cad40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0118e0 tx=0x7f42ec011da0 comp rx=0 tx=0).stop 2023-11-18T20:43:32.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.777+0000 7f42ff42d640 1 -- 172.21.15.60:0/3887437414 shutdown_connections 2023-11-18T20:43:32.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.777+0000 7f42ff42d640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42e406ae70 0x7f42e406d330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:32.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.777+0000 7f42ff42d640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f80b7390 0x7f42f80b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.777+0000 7f42ff42d640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42f81f3770 0x7f42f80cad40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.777+0000 7f42ff42d640 1 --2- 172.21.15.60:0/3887437414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e0005180 0x7f42f814a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.777+0000 7f42ff42d640 1 -- 172.21.15.60:0/3887437414 >> 172.21.15.60:0/3887437414 conn(0x7f42f8072230 msgr2=0x7f42f815a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:32.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.778+0000 7f42ff42d640 1 -- 172.21.15.60:0/3887437414 shutdown_connections 2023-11-18T20:43:32.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.779+0000 7f42ff42d640 1 -- 172.21.15.60:0/3887437414 wait complete. 2023-11-18T20:43:32.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-18T20:43:32.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.918+0000 7f3f53690640 1 Processor -- start 2023-11-18T20:43:32.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.918+0000 7f3f53690640 1 -- start start 2023-11-18T20:43:32.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f53690640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f4c13ce30 0x7f3f4c0cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:32.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f53690640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f4c0ccf70 0x7f3f4c0714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:32.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f53690640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4c1405c0 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f53690640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4c13f6d0 con 0x7f3f4c13ce30 2023-11-18T20:43:32.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f53690640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c13f9b0 con 0x7f3f4c0ccf70 2023-11-18T20:43:32.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f5142c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f4c13ce30 0x7f3f4c0cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:32.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f5142c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3f4c13ce30 0x7f3f4c0cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44572/0 (socket says 172.21.15.60:44572) 2023-11-18T20:43:32.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f5142c640 1 -- 172.21.15.60:0/1576992541 learned_addr learned my addr 172.21.15.60:0/1576992541 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:32.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f50c2b640 1 --2- 172.21.15.60:0/1576992541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f4c0ccf70 0x7f3f4c0714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:32.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f5142c640 1 -- 172.21.15.60:0/1576992541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f4c0ccf70 msgr2=0x7f3f4c0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:32.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f5142c640 1 --2- 172.21.15.60:0/1576992541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f4c0ccf70 0x7f3f4c0714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:32.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f5142c640 1 -- 172.21.15.60:0/1576992541 >> v1:172.21.15.100:6789/0 conn(0x7f3f4c0cd3b0 legacy=0x7f3f4c06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:32.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.919+0000 7f3f5142c640 1 -- 172.21.15.60:0/1576992541 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c13fc90 con 0x7f3f4c13ce30 2023-11-18T20:43:32.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.920+0000 7f3f5142c640 1 --2- 172.21.15.60:0/1576992541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c13ce30 0x7f3f4c0cea10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009870 tx=0x7f3f4802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bcf52eb0174a1b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.920+0000 7f3f43fff640 1 -- 172.21.15.60:0/1576992541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48030840 con 0x7f3f4c13ce30 2023-11-18T20:43:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.920+0000 7f3f43fff640 1 -- 172.21.15.60:0/1576992541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f480309a0 con 0x7f3f4c13ce30 2023-11-18T20:43:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.920+0000 7f3f43fff640 1 -- 172.21.15.60:0/1576992541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48030d30 con 0x7f3f4c13ce30 2023-11-18T20:43:32.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.920+0000 7f3f53690640 1 -- 172.21.15.60:0/1576992541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c13ce30 msgr2=0x7f3f4c0cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:32.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.920+0000 7f3f53690640 1 --2- 172.21.15.60:0/1576992541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c13ce30 0x7f3f4c0cea10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009870 tx=0x7f3f4802b980 comp rx=0 tx=0).stop 2023-11-18T20:43:32.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.921+0000 7f3f53690640 1 -- 172.21.15.60:0/1576992541 shutdown_connections 2023-11-18T20:43:32.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.921+0000 7f3f53690640 1 --2- 172.21.15.60:0/1576992541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f4c0ccf70 0x7f3f4c0714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:32.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.921+0000 7f3f53690640 1 --2- 172.21.15.60:0/1576992541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c13ce30 0x7f3f4c0cea10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:32.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.921+0000 7f3f53690640 1 -- 172.21.15.60:0/1576992541 >> 172.21.15.60:0/1576992541 conn(0x7f3f4c0b1e50 msgr2=0x7f3f4c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:32.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.921+0000 7f3f53690640 1 -- 172.21.15.60:0/1576992541 shutdown_connections 2023-11-18T20:43:32.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.921+0000 7f3f53690640 1 -- 172.21.15.60:0/1576992541 wait complete. 2023-11-18T20:43:32.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.922+0000 7f3f53690640 1 Processor -- start 2023-11-18T20:43:32.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.922+0000 7f3f53690640 1 -- start start 2023-11-18T20:43:32.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.922+0000 7f3f53690640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f4c0ccf70 0x7f3f4c0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:32.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f53690640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c0cd3b0 0x7f3f4c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:32.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f53690640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f4c13ce30 0x7f3f4c150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:32.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f53690640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c152470 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f53690640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c13b710 con 0x7f3f4c13ce30 2023-11-18T20:43:32.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f53690640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c157a00 con 0x7f3f4c0ccf70 2023-11-18T20:43:32.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f50c2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c0cd3b0 0x7f3f4c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:32.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f50c2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c0cd3b0 0x7f3f4c1574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44582/0 (socket says 172.21.15.60:44582) 2023-11-18T20:43:32.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f50c2b640 1 -- 172.21.15.60:0/1090572125 learned_addr learned my addr 172.21.15.60:0/1090572125 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:32.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f5142c640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f4c0ccf70 0x7f3f4c0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:32.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f51c2d640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f4c13ce30 0x7f3f4c150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:32.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f50c2b640 1 -- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f4c0ccf70 msgr2=0x7f3f4c0b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:32.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f50c2b640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f4c0ccf70 0x7f3f4c0b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:32.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f50c2b640 1 -- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f4c13ce30 msgr2=0x7f3f4c150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:32.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f50c2b640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f4c13ce30 0x7f3f4c150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:32.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f50c2b640 1 -- 172.21.15.60:0/1090572125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f480094d0 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f50c2b640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c0cd3b0 0x7f3f4c1574c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00d830 tx=0x7f3f3c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:32.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f41ffb640 1 -- 172.21.15.60:0/1090572125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00b660 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f41ffb640 1 -- 172.21.15.60:0/1090572125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c00b7c0 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f53690640 1 -- 172.21.15.60:0/1090572125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c151b40 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.923+0000 7f3f41ffb640 1 -- 172.21.15.60:0/1090572125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00bb50 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.924+0000 7f3f53690640 1 -- 172.21.15.60:0/1090572125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c152100 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.925+0000 7f3f53690640 1 -- 172.21.15.60:0/1090572125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c069400 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.925+0000 7f3f41ffb640 1 -- 172.21.15.60:0/1090572125 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f3c01e460 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.926+0000 7f3f41ffb640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3006ae40 0x7f3f3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.926+0000 7f3f41ffb640 1 -- 172.21.15.60:0/1090572125 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3f3c089c40 con 0x7f3f4c0cd3b0 2023-11-18T20:43:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.926+0000 7f3f5142c640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3006ae40 0x7f3f3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.926+0000 7f3f5142c640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3006ae40 0x7f3f3006d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f48036000 tx=0x7f3f48008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:32.930+0000 7f3f41ffb640 1 -- 172.21.15.60:0/1090572125 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f3c054d60 con 0x7f3f4c0cd3b0 2023-11-18T20:43:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:33.144+0000 7f3f53690640 1 -- 172.21.15.60:0/1090572125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3f4c154b90 con 0x7f3f4c0cd3b0 2023-11-18T20:43:34.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.099+0000 7f3f41ffb640 1 -- 172.21.15.60:0/1090572125 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3f3c058380 con 0x7f3f4c0cd3b0 2023-11-18T20:43:34.100 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 4 and data pool 5 2023-11-18T20:43:34.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 -- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3006ae40 msgr2=0x7f3f3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3006ae40 0x7f3f3006d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3f48036000 tx=0x7f3f48008030 comp rx=0 tx=0).stop 2023-11-18T20:43:34.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 -- 172.21.15.60:0/1090572125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c0cd3b0 msgr2=0x7f3f4c1574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c0cd3b0 0x7f3f4c1574c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00d830 tx=0x7f3f3c00dcf0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 -- 172.21.15.60:0/1090572125 shutdown_connections 2023-11-18T20:43:34.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3f3006ae40 0x7f3f3006d300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f4c13ce30 0x7f3f4c150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f4c0cd3b0 0x7f3f4c1574c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 --2- 172.21.15.60:0/1090572125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f4c0ccf70 0x7f3f4c0b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 -- 172.21.15.60:0/1090572125 >> 172.21.15.60:0/1090572125 conn(0x7f3f4c0b1e50 msgr2=0x7f3f4c15c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:34.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.103+0000 7f3f53690640 1 -- 172.21.15.60:0/1090572125 shutdown_connections 2023-11-18T20:43:34.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.104+0000 7f3f53690640 1 -- 172.21.15.60:0/1090572125 wait complete. 2023-11-18T20:43:34.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:34.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.233+0000 7f42dcd7c640 1 Processor -- start 2023-11-18T20:43:34.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.233+0000 7f42dcd7c640 1 -- start start 2023-11-18T20:43:34.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.234+0000 7f42dcd7c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42d81415f0 0x7f42d8141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.234+0000 7f42dcd7c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f42d8144a00 0x7f42d806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.234+0000 7f42dcd7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d8144440 0x7f42d806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.234+0000 7f42dcd7c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d80b3b70 con 0x7f42d81415f0 2023-11-18T20:43:34.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.234+0000 7f42dcd7c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d80b2ef0 con 0x7f42d8144a00 2023-11-18T20:43:34.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.234+0000 7f42dcd7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80b31d0 con 0x7f42d8144440 2023-11-18T20:43:34.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.234+0000 7f42d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42d81415f0 0x7f42d8141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.234+0000 7f42d659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42d81415f0 0x7f42d8141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44584/0 (socket says 172.21.15.60:44584) 2023-11-18T20:43:34.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.234+0000 7f42d659c640 1 -- 172.21.15.60:0/4140882395 learned_addr learned my addr 172.21.15.60:0/4140882395 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:34.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.235+0000 7f42d5d9b640 1 --2- 172.21.15.60:0/4140882395 >> v2:172.21.15.100:3300/0 conn(0x7f42d8144a00 0x7f42d806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.235+0000 7f42d659c640 1 -- 172.21.15.60:0/4140882395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d8144440 msgr2=0x7f42d806ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:34.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.235+0000 7f42d6d9d640 1 --2- 172.21.15.60:0/4140882395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d8144440 0x7f42d806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.235+0000 7f42d659c640 1 --2- 172.21.15.60:0/4140882395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d8144440 0x7f42d806ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.235+0000 7f42d659c640 1 -- 172.21.15.60:0/4140882395 >> v2:172.21.15.100:3300/0 conn(0x7f42d8144a00 msgr2=0x7f42d806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:34.239 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.235+0000 7f42d659c640 1 --2- 172.21.15.60:0/4140882395 >> v2:172.21.15.100:3300/0 conn(0x7f42d8144a00 0x7f42d806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.235+0000 7f42d659c640 1 -- 172.21.15.60:0/4140882395 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80b34b0 con 0x7f42d81415f0 2023-11-18T20:43:34.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.235+0000 7f42d5d9b640 1 --2- 172.21.15.60:0/4140882395 >> v2:172.21.15.100:3300/0 conn(0x7f42d8144a00 0x7f42d806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:34.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.235+0000 7f42d6d9d640 1 --2- 172.21.15.60:0/4140882395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d8144440 0x7f42d806ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:34.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.235+0000 7f42d659c640 1 --2- 172.21.15.60:0/4140882395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d81415f0 0x7f42d8141e00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f42d8144f90 tx=0x7f42d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7baf429c6fc11dda server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:34.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42d559a640 1 -- 172.21.15.60:0/4140882395 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0031870 con 0x7f42d81415f0 2023-11-18T20:43:34.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42d559a640 1 -- 172.21.15.60:0/4140882395 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d00319d0 con 0x7f42d81415f0 2023-11-18T20:43:34.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42d559a640 1 -- 172.21.15.60:0/4140882395 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0031d60 con 0x7f42d81415f0 2023-11-18T20:43:34.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42dcd7c640 1 -- 172.21.15.60:0/4140882395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d81415f0 msgr2=0x7f42d8141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42dcd7c640 1 --2- 172.21.15.60:0/4140882395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d81415f0 0x7f42d8141e00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f42d8144f90 tx=0x7f42d002b910 comp rx=0 tx=0).stop 2023-11-18T20:43:34.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42dcd7c640 1 -- 172.21.15.60:0/4140882395 shutdown_connections 2023-11-18T20:43:34.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42dcd7c640 1 --2- 172.21.15.60:0/4140882395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d8144440 0x7f42d806ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42dcd7c640 1 --2- 172.21.15.60:0/4140882395 >> v2:172.21.15.100:3300/0 conn(0x7f42d8144a00 0x7f42d806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42dcd7c640 1 --2- 172.21.15.60:0/4140882395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d81415f0 0x7f42d8141e00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42dcd7c640 1 -- 172.21.15.60:0/4140882395 >> 172.21.15.60:0/4140882395 conn(0x7f42d8070e70 msgr2=0x7f42d8071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:34.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42dcd7c640 1 -- 172.21.15.60:0/4140882395 shutdown_connections 2023-11-18T20:43:34.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.236+0000 7f42dcd7c640 1 -- 172.21.15.60:0/4140882395 wait complete. 2023-11-18T20:43:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42dcd7c640 1 Processor -- start 2023-11-18T20:43:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42dcd7c640 1 -- start start 2023-11-18T20:43:34.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42dcd7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d81415f0 0x7f42d80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42dcd7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8144440 0x7f42d81557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42dcd7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d8144a00 0x7f42d8155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42dcd7c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8130730 con 0x7f42d8144440 2023-11-18T20:43:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42dcd7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d81404f0 con 0x7f42d8144a00 2023-11-18T20:43:34.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42dcd7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8130100 con 0x7f42d81415f0 2023-11-18T20:43:34.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42d5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8144440 0x7f42d81557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42d5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8144440 0x7f42d81557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44600/0 (socket says 172.21.15.60:44600) 2023-11-18T20:43:34.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.238+0000 7f42d5d9b640 1 -- 172.21.15.60:0/479521422 learned_addr learned my addr 172.21.15.60:0/479521422 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:34.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42d659c640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d81415f0 0x7f42d80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42d6d9d640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d8144a00 0x7f42d8155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42d5d9b640 1 -- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d81415f0 msgr2=0x7f42d80cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42d5d9b640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d81415f0 0x7f42d80cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42d5d9b640 1 -- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d8144a00 msgr2=0x7f42d8155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42d5d9b640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d8144a00 0x7f42d8155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42d5d9b640 1 -- 172.21.15.60:0/479521422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d00094d0 con 0x7f42d8144440 2023-11-18T20:43:34.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42d5d9b640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8144440 0x7f42d81557f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f42c000c7f0 tx=0x7f42c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:34.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42ceffd640 1 -- 172.21.15.60:0/479521422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c000cea0 con 0x7f42d8144440 2023-11-18T20:43:34.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42ceffd640 1 -- 172.21.15.60:0/479521422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c0007a30 con 0x7f42d8144440 2023-11-18T20:43:34.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42dcd7c640 1 -- 172.21.15.60:0/479521422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d81563b0 con 0x7f42d8144440 2023-11-18T20:43:34.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42ceffd640 1 -- 172.21.15.60:0/479521422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0007dc0 con 0x7f42d8144440 2023-11-18T20:43:34.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.239+0000 7f42dcd7c640 1 -- 172.21.15.60:0/479521422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d81303c0 con 0x7f42d8144440 2023-11-18T20:43:34.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.240+0000 7f42dcd7c640 1 -- 172.21.15.60:0/479521422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d81308e0 con 0x7f42d8144440 2023-11-18T20:43:34.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.241+0000 7f42ceffd640 1 -- 172.21.15.60:0/479521422 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f42c00113f0 con 0x7f42d8144440 2023-11-18T20:43:34.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.241+0000 7f42ceffd640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b806ae70 0x7f42b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.241+0000 7f42ceffd640 1 -- 172.21.15.60:0/479521422 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42c00896d0 con 0x7f42d8144440 2023-11-18T20:43:34.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.242+0000 7f42d659c640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b806ae70 0x7f42b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.242+0000 7f42d659c640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b806ae70 0x7f42b806d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42d0036000 tx=0x7f42d003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:34.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.246+0000 7f42ceffd640 1 -- 172.21.15.60:0/479521422 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42d81308e0 con 0x7f42d8144440 2023-11-18T20:43:34.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.422+0000 7f42dcd7c640 1 -- 172.21.15.60:0/479521422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42d8069400 con 0x7f42d8144440 2023-11-18T20:43:34.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.424+0000 7f42ceffd640 1 -- 172.21.15.60:0/479521422 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16570 (secure 0 0 0) 0x7f42c001f090 con 0x7f42d8144440 2023-11-18T20:43:34.425 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:34.426 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":33,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:33.145948+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:34.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 -- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b806ae70 msgr2=0x7f42b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b806ae70 0x7f42b806d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42d0036000 tx=0x7f42d003c040 comp rx=0 tx=0).stop 2023-11-18T20:43:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 -- 172.21.15.60:0/479521422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8144440 msgr2=0x7f42d81557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8144440 0x7f42d81557f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f42c000c7f0 tx=0x7f42c000ccb0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 -- 172.21.15.60:0/479521422 shutdown_connections 2023-11-18T20:43:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b806ae70 0x7f42b806d330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d8144a00 0x7f42d8155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d8144440 0x7f42d81557f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 --2- 172.21.15.60:0/479521422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d81415f0 0x7f42d80cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 -- 172.21.15.60:0/479521422 >> 172.21.15.60:0/479521422 conn(0x7f42d8070e70 msgr2=0x7f42d8142fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 -- 172.21.15.60:0/479521422 shutdown_connections 2023-11-18T20:43:34.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.426+0000 7f42dcd7c640 1 -- 172.21.15.60:0/479521422 wait complete. 2023-11-18T20:43:34.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:34.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.544+0000 7f4351b73640 1 Processor -- start 2023-11-18T20:43:34.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.544+0000 7f4351b73640 1 -- start start 2023-11-18T20:43:34.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f4351b73640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f434c143490 0x7f434c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f4351b73640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434c15e2a0 con 0x7f434c15abf0 2023-11-18T20:43:34.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f4351b73640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434c15b240 con 0x7f434c15e730 2023-11-18T20:43:34.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f4351b73640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f434c15b520 con 0x7f434c143490 2023-11-18T20:43:34.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f434c15abf0 0x7f434c0b2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36986/0 (socket says 172.21.15.60:36986) 2023-11-18T20:43:34.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434bfff640 1 -- 172.21.15.60:0/2131149628 learned_addr learned my addr 172.21.15.60:0/2131149628 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:34.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434b7fe640 1 --2- 172.21.15.60:0/2131149628 >> v2:172.21.15.100:3300/0 conn(0x7f434c143490 0x7f434c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112392878 0 0) 0x7f434c15e2a0 con 0x7f434c15abf0 2023-11-18T20:43:34.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4334003520 con 0x7f434c15abf0 2023-11-18T20:43:34.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3708722070 0 0) 0x7f4334003520 con 0x7f434c15abf0 2023-11-18T20:43:34.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 >> v2:172.21.15.100:3300/0 conn(0x7f434c143490 msgr2=0x7f434c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:34.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 --2- 172.21.15.60:0/2131149628 >> v2:172.21.15.100:3300/0 conn(0x7f434c143490 0x7f434c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 >> v1:172.21.15.100:6789/0 conn(0x7f434c15e730 legacy=0x7f434c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c15dc50 con 0x7f434c15abf0 2023-11-18T20:43:34.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f43440019f0 con 0x7f434c15abf0 2023-11-18T20:43:34.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 >> v1:172.21.15.60:6789/0 conn(0x7f434c15abf0 legacy=0x7f434c0b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4334003970 0x7f4334003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43340047c0 0x7f4334004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4334005180 0x7f433400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c15b520 con 0x7f4334003970 2023-11-18T20:43:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433400c020 con 0x7f4334005180 2023-11-18T20:43:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433400c1e0 con 0x7f43340047c0 2023-11-18T20:43:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.545+0000 7f434b7fe640 1 --2- 172.21.15.60:0/2131149628 >> v2:172.21.15.100:3300/0 conn(0x7f434c143490 0x7f434c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.546+0000 7f434b7fe640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4334003970 0x7f4334003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.546+0000 7f434bfff640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4334005180 0x7f433400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.546+0000 7f434affd640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43340047c0 0x7f4334004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.546+0000 7f434b7fe640 1 -- 172.21.15.60:0/2131149628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43340047c0 msgr2=0x7f4334004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:34.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.546+0000 7f434b7fe640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43340047c0 0x7f4334004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.546+0000 7f434b7fe640 1 -- 172.21.15.60:0/2131149628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4334005180 msgr2=0x7f433400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:34.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.546+0000 7f434b7fe640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4334005180 0x7f433400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.546+0000 7f434b7fe640 1 -- 172.21.15.60:0/2131149628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4334003520 con 0x7f4334003970 2023-11-18T20:43:34.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.546+0000 7f434bfff640 1 -- 172.21.15.60:0/2131149628 reap_dead start 2023-11-18T20:43:34.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f434b7fe640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4334003970 0x7f4334003d90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4340009c70 tx=0x7f43340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef6c125d82c010d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:34.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340004020 con 0x7f4334003970 2023-11-18T20:43:34.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4340004180 con 0x7f4334003970 2023-11-18T20:43:34.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f434a7fc640 1 -- 172.21.15.60:0/2131149628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434002c8f0 con 0x7f4334003970 2023-11-18T20:43:34.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f4351b73640 1 -- 172.21.15.60:0/2131149628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4334003970 msgr2=0x7f4334003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f4351b73640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4334003970 0x7f4334003d90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4340009c70 tx=0x7f43340057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f4351b73640 1 -- 172.21.15.60:0/2131149628 shutdown_connections 2023-11-18T20:43:34.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f4351b73640 1 --2- 172.21.15.60:0/2131149628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4334003970 0x7f4334003d90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f4351b73640 1 -- 172.21.15.60:0/2131149628 >> 172.21.15.60:0/2131149628 conn(0x7f434c06a4d0 msgr2=0x7f434c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:34.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f4351b73640 1 -- 172.21.15.60:0/2131149628 shutdown_connections 2023-11-18T20:43:34.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.547+0000 7f4351b73640 1 -- 172.21.15.60:0/2131149628 wait complete. 2023-11-18T20:43:34.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.548+0000 7f4351b73640 1 Processor -- start 2023-11-18T20:43:34.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.549+0000 7f4351b73640 1 -- start start 2023-11-18T20:43:34.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.549+0000 7f4351b73640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4334003970 0x7f434c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.549+0000 7f4351b73640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f434c12f260 0x7f434c0ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.549+0000 7f4351b73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f434c13f1a0 0x7f434c13f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.549+0000 7f4351b73640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c13fb60 con 0x7f434c13f1a0 2023-11-18T20:43:34.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.549+0000 7f4351b73640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0cb860 con 0x7f4334003970 2023-11-18T20:43:34.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.549+0000 7f4351b73640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0cac50 con 0x7f434c12f260 2023-11-18T20:43:34.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.549+0000 7f434bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f434c13f1a0 0x7f434c13f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f434bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f434c13f1a0 0x7f434c13f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44620/0 (socket says 172.21.15.60:44620) 2023-11-18T20:43:34.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f434bfff640 1 -- 172.21.15.60:0/1717030351 learned_addr learned my addr 172.21.15.60:0/1717030351 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:34.559 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f434affd640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f434c12f260 0x7f434c0ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f434b7fe640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4334003970 0x7f434c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f434bfff640 1 -- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f434c12f260 msgr2=0x7f434c0ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f434bfff640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f434c12f260 0x7f434c0ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f434bfff640 1 -- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4334003970 msgr2=0x7f434c159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f434bfff640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4334003970 0x7f434c159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f434bfff640 1 -- 172.21.15.60:0/1717030351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c0caf30 con 0x7f434c13f1a0 2023-11-18T20:43:34.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f434bfff640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f434c13f1a0 0x7f434c13f620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f434c0b1df0 tx=0x7f4344007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f4333fff640 1 -- 172.21.15.60:0/1717030351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434400ab70 con 0x7f434c13f1a0 2023-11-18T20:43:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f4333fff640 1 -- 172.21.15.60:0/1717030351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434400acd0 con 0x7f434c13f1a0 2023-11-18T20:43:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f4351b73640 1 -- 172.21.15.60:0/1717030351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434c0cb210 con 0x7f434c13f1a0 2023-11-18T20:43:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f4333fff640 1 -- 172.21.15.60:0/1717030351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440135f0 con 0x7f434c13f1a0 2023-11-18T20:43:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.550+0000 7f4351b73640 1 -- 172.21.15.60:0/1717030351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434c159ec0 con 0x7f434c13f1a0 2023-11-18T20:43:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.551+0000 7f4351b73640 1 -- 172.21.15.60:0/1717030351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434c069400 con 0x7f434c13f1a0 2023-11-18T20:43:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.552+0000 7f4333fff640 1 -- 172.21.15.60:0/1717030351 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4344013750 con 0x7f434c13f1a0 2023-11-18T20:43:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.552+0000 7f4333fff640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f433806ae70 0x7f433806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.552+0000 7f4333fff640 1 -- 172.21.15.60:0/1717030351 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4344089c20 con 0x7f434c13f1a0 2023-11-18T20:43:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.553+0000 7f434b7fe640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f433806ae70 0x7f433806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.553+0000 7f434b7fe640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f433806ae70 0x7f433806d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4340035000 tx=0x7f4340030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.556+0000 7f4333fff640 1 -- 172.21.15.60:0/1717030351 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4344054ce0 con 0x7f434c13f1a0 2023-11-18T20:43:34.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.733+0000 7f4351b73640 1 -- 172.21.15.60:0/1717030351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f434c0b20e0 con 0x7f434c13f1a0 2023-11-18T20:43:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.734+0000 7f4333fff640 1 -- 172.21.15.60:0/1717030351 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16570 (secure 0 0 0) 0x7f4344058300 con 0x7f434c13f1a0 2023-11-18T20:43:34.735 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:34.735 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":33,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:33.145948+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 -- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f433806ae70 msgr2=0x7f433806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f433806ae70 0x7f433806d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4340035000 tx=0x7f4340030000 comp rx=0 tx=0).stop 2023-11-18T20:43:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 -- 172.21.15.60:0/1717030351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f434c13f1a0 msgr2=0x7f434c13f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f434c13f1a0 0x7f434c13f620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f434c0b1df0 tx=0x7f4344007b00 comp rx=0 tx=0).stop 2023-11-18T20:43:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 -- 172.21.15.60:0/1717030351 shutdown_connections 2023-11-18T20:43:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f433806ae70 0x7f433806d330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f434c13f1a0 0x7f434c13f620 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f434c12f260 0x7f434c0ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 --2- 172.21.15.60:0/1717030351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4334003970 0x7f434c159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 -- 172.21.15.60:0/1717030351 >> 172.21.15.60:0/1717030351 conn(0x7f434c06a4d0 msgr2=0x7f434c140e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 -- 172.21.15.60:0/1717030351 shutdown_connections 2023-11-18T20:43:34.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.736+0000 7f4351b73640 1 -- 172.21.15.60:0/1717030351 wait complete. 2023-11-18T20:43:34.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-18T20:43:34.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.858+0000 7f92f8644640 1 Processor -- start 2023-11-18T20:43:34.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.858+0000 7f92f8644640 1 -- start start 2023-11-18T20:43:34.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.859+0000 7f92f8644640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f92f015d330 0x7f92f00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.859+0000 7f92f8644640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f00b2b10 con 0x7f92f00d2910 2023-11-18T20:43:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.859+0000 7f92f8644640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f00d1590 con 0x7f92f00d3900 2023-11-18T20:43:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.859+0000 7f92f8644640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f00d1870 con 0x7f92f015d330 2023-11-18T20:43:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.859+0000 7f92f5bdf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f92f00d2910 0x7f92f00d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36992/0 (socket says 172.21.15.60:36992) 2023-11-18T20:43:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.859+0000 7f92f5bdf640 1 -- 172.21.15.60:0/2682399524 learned_addr learned my addr 172.21.15.60:0/2682399524 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830963500 0 0) 0x7f92f00b2b10 con 0x7f92f00d2910 2023-11-18T20:43:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92d8003520 con 0x7f92f00d2910 2023-11-18T20:43:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f63e0640 1 --2- 172.21.15.60:0/2682399524 >> v2:172.21.15.100:3300/0 conn(0x7f92f015d330 0x7f92f00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3646577806 0 0) 0x7f92d8003520 con 0x7f92f00d2910 2023-11-18T20:43:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 >> v2:172.21.15.100:3300/0 conn(0x7f92f015d330 msgr2=0x7f92f00b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 --2- 172.21.15.60:0/2682399524 >> v2:172.21.15.100:3300/0 conn(0x7f92f015d330 0x7f92f00b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 >> v1:172.21.15.100:6789/0 conn(0x7f92f00d3900 legacy=0x7f92f00b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00d1b50 con 0x7f92f00d2910 2023-11-18T20:43:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f92e4001670 con 0x7f92f00d2910 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 >> v1:172.21.15.60:6789/0 conn(0x7f92f00d2910 legacy=0x7f92f00d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d80047c0 0x7f92d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92d8005180 0x7f92d8014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00d1870 con 0x7f92d8003970 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d8014930 con 0x7f92d80047c0 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d8014af0 con 0x7f92d8005180 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f63e0640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f5bdf640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d80047c0 0x7f92d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f6be1640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92d8005180 0x7f92d8014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f63e0640 1 -- 172.21.15.60:0/2682399524 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92d8005180 msgr2=0x7f92d8014190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f63e0640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92d8005180 0x7f92d8014190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f63e0640 1 -- 172.21.15.60:0/2682399524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d80047c0 msgr2=0x7f92d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f63e0640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d80047c0 0x7f92d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f63e0640 1 -- 172.21.15.60:0/2682399524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d8003520 con 0x7f92d8003970 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f5bdf640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d80047c0 0x7f92d8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.860+0000 7f92f6be1640 1 -- 172.21.15.60:0/2682399524 reap_dead start 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.861+0000 7f92f63e0640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92d8003d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009ac0 tx=0x7f92d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1921db7f971c0cc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.861+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e802c6b0 con 0x7f92d8003970 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.861+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e802c810 con 0x7f92d8003970 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.861+0000 7f92f53de640 1 -- 172.21.15.60:0/2682399524 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e802cba0 con 0x7f92d8003970 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.862+0000 7f92f8644640 1 -- 172.21.15.60:0/2682399524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 msgr2=0x7f92d8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.862+0000 7f92f8644640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92d8003d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009ac0 tx=0x7f92d80057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.862+0000 7f92f8644640 1 -- 172.21.15.60:0/2682399524 shutdown_connections 2023-11-18T20:43:34.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.862+0000 7f92f8644640 1 --2- 172.21.15.60:0/2682399524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92d8003d90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.862+0000 7f92f8644640 1 -- 172.21.15.60:0/2682399524 >> 172.21.15.60:0/2682399524 conn(0x7f92f0072020 msgr2=0x7f92f0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:34.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.862+0000 7f92f8644640 1 -- 172.21.15.60:0/2682399524 shutdown_connections 2023-11-18T20:43:34.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.862+0000 7f92f8644640 1 -- 172.21.15.60:0/2682399524 wait complete. 2023-11-18T20:43:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.863+0000 7f92f8644640 1 Processor -- start 2023-11-18T20:43:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.863+0000 7f92f8644640 1 -- start start 2023-11-18T20:43:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f8644640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92f0157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f8644640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f015f480 0x7f92f014e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f8644640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92f015a360 0x7f92f015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f8644640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f015ad20 con 0x7f92d8003970 2023-11-18T20:43:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f8644640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00ca330 con 0x7f92f015f480 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f8644640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f015fc40 con 0x7f92f015a360 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f63e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92f0157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f63e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92f0157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44638/0 (socket says 172.21.15.60:44638) 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f63e0640 1 -- 172.21.15.60:0/2312009070 learned_addr learned my addr 172.21.15.60:0/2312009070 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f5bdf640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f015f480 0x7f92f014e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f6be1640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92f015a360 0x7f92f015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f63e0640 1 -- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92f015a360 msgr2=0x7f92f015a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f63e0640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92f015a360 0x7f92f015a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f63e0640 1 -- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f015f480 msgr2=0x7f92f014e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f63e0640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f015f480 0x7f92f014e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f63e0640 1 -- 172.21.15.60:0/2312009070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00ca580 con 0x7f92d8003970 2023-11-18T20:43:34.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.864+0000 7f92f63e0640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92f0157b10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f92e802c250 tx=0x7f92e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.865+0000 7f92eeffd640 1 -- 172.21.15.60:0/2312009070 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8003810 con 0x7f92d8003970 2023-11-18T20:43:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.865+0000 7f92eeffd640 1 -- 172.21.15.60:0/2312009070 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e8003970 con 0x7f92d8003970 2023-11-18T20:43:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.865+0000 7f92f8644640 1 -- 172.21.15.60:0/2312009070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f00ca860 con 0x7f92d8003970 2023-11-18T20:43:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.865+0000 7f92eeffd640 1 -- 172.21.15.60:0/2312009070 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e8003d00 con 0x7f92d8003970 2023-11-18T20:43:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.865+0000 7f92f8644640 1 -- 172.21.15.60:0/2312009070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f015f900 con 0x7f92d8003970 2023-11-18T20:43:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.866+0000 7f92f8644640 1 -- 172.21.15.60:0/2312009070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f0069400 con 0x7f92d8003970 2023-11-18T20:43:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.866+0000 7f92eeffd640 1 -- 172.21.15.60:0/2312009070 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f92e8047450 con 0x7f92d8003970 2023-11-18T20:43:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.867+0000 7f92eeffd640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f92dc06ae70 0x7f92dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.867+0000 7f92eeffd640 1 -- 172.21.15.60:0/2312009070 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92e80ab800 con 0x7f92d8003970 2023-11-18T20:43:34.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.867+0000 7f92f5bdf640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f92dc06ae70 0x7f92dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:34.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.868+0000 7f92f5bdf640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f92dc06ae70 0x7f92dc06d330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92f014f240 tx=0x7f92e4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:34.871+0000 7f92eeffd640 1 -- 172.21.15.60:0/2312009070 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92e8076940 con 0x7f92d8003970 2023-11-18T20:43:35.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.086+0000 7f92f8644640 1 -- 172.21.15.60:0/2312009070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f92f015ad20 con 0x7f92d8003970 2023-11-18T20:43:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.100+0000 7f92eeffd640 1 -- 172.21.15.60:0/2312009070 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f92e8003e60 con 0x7f92d8003970 2023-11-18T20:43:35.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 -- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f92dc06ae70 msgr2=0x7f92dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:35.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f92dc06ae70 0x7f92dc06d330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92f014f240 tx=0x7f92e4009040 comp rx=0 tx=0).stop 2023-11-18T20:43:35.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 -- 172.21.15.60:0/2312009070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 msgr2=0x7f92f0157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:35.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92f0157b10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f92e802c250 tx=0x7f92e8004730 comp rx=0 tx=0).stop 2023-11-18T20:43:35.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 -- 172.21.15.60:0/2312009070 shutdown_connections 2023-11-18T20:43:35.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f92dc06ae70 0x7f92dc06d330 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:35.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92f015a360 0x7f92f015a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:35.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f015f480 0x7f92f014e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:35.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 --2- 172.21.15.60:0/2312009070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8003970 0x7f92f0157b10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:35.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 -- 172.21.15.60:0/2312009070 >> 172.21.15.60:0/2312009070 conn(0x7f92f0072020 msgr2=0x7f92f0158af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:35.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.102+0000 7f92f8644640 1 -- 172.21.15.60:0/2312009070 shutdown_connections 2023-11-18T20:43:35.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.103+0000 7f92f8644640 1 -- 172.21.15.60:0/2312009070 wait complete. 2023-11-18T20:43:35.114 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-18T20:43:35.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-18T20:43:35.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.222+0000 7f0b775fc640 1 Processor -- start 2023-11-18T20:43:35.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.222+0000 7f0b775fc640 1 -- start start 2023-11-18T20:43:35.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.223+0000 7f0b775fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0b70143490 0x7f0b700b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:35.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.223+0000 7f0b775fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0b7015abf0 0x7f0b700b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:35.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.223+0000 7f0b775fc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7015e2a0 con 0x7f0b7015e730 2023-11-18T20:43:35.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.223+0000 7f0b775fc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7015b520 con 0x7f0b70143490 2023-11-18T20:43:35.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.223+0000 7f0b775fc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b70143870 con 0x7f0b7015abf0 2023-11-18T20:43:35.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.223+0000 7f0b75398640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0b70143490 0x7f0b700b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:35.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.223+0000 7f0b75398640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0b70143490 0x7f0b700b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44654/0 (socket says 172.21.15.60:44654) 2023-11-18T20:43:35.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.223+0000 7f0b75398640 1 -- 172.21.15.60:0/3449245811 learned_addr learned my addr 172.21.15.60:0/3449245811 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:35.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.223+0000 7f0b75b99640 1 --2- 172.21.15.60:0/3449245811 >> v2:172.21.15.100:3300/0 conn(0x7f0b7015abf0 0x7f0b700b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:35.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.224+0000 7f0b75398640 1 -- 172.21.15.60:0/3449245811 >> v2:172.21.15.100:3300/0 conn(0x7f0b7015abf0 msgr2=0x7f0b700b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:35.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.224+0000 7f0b75398640 1 --2- 172.21.15.60:0/3449245811 >> v2:172.21.15.100:3300/0 conn(0x7f0b7015abf0 0x7f0b700b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:35.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.224+0000 7f0b75398640 1 -- 172.21.15.60:0/3449245811 >> v1:172.21.15.100:6789/0 conn(0x7f0b7015e730 legacy=0x7f0b7015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:35.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.224+0000 7f0b75398640 1 -- 172.21.15.60:0/3449245811 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7015b240 con 0x7f0b70143490 2023-11-18T20:43:35.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.224+0000 7f0b75398640 1 --2- 172.21.15.60:0/3449245811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 0x7f0b700b0cf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0b700b4ca0 tx=0x7f0b6802e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5333552e516ec6e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:35.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b6ffff640 1 -- 172.21.15.60:0/3449245811 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68036440 con 0x7f0b70143490 2023-11-18T20:43:35.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b6ffff640 1 -- 172.21.15.60:0/3449245811 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b680365a0 con 0x7f0b70143490 2023-11-18T20:43:35.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b6ffff640 1 -- 172.21.15.60:0/3449245811 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68036930 con 0x7f0b70143490 2023-11-18T20:43:35.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b775fc640 1 -- 172.21.15.60:0/3449245811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 msgr2=0x7f0b700b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:35.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b775fc640 1 --2- 172.21.15.60:0/3449245811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 0x7f0b700b0cf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0b700b4ca0 tx=0x7f0b6802e170 comp rx=0 tx=0).stop 2023-11-18T20:43:35.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b775fc640 1 -- 172.21.15.60:0/3449245811 shutdown_connections 2023-11-18T20:43:35.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b775fc640 1 --2- 172.21.15.60:0/3449245811 >> v2:172.21.15.100:3300/0 conn(0x7f0b7015abf0 0x7f0b700b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:35.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b775fc640 1 --2- 172.21.15.60:0/3449245811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 0x7f0b700b0cf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:35.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b775fc640 1 -- 172.21.15.60:0/3449245811 >> 172.21.15.60:0/3449245811 conn(0x7f0b7006a4d0 msgr2=0x7f0b7006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:35.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b775fc640 1 -- 172.21.15.60:0/3449245811 shutdown_connections 2023-11-18T20:43:35.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.225+0000 7f0b775fc640 1 -- 172.21.15.60:0/3449245811 wait complete. 2023-11-18T20:43:35.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.226+0000 7f0b775fc640 1 Processor -- start 2023-11-18T20:43:35.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.227+0000 7f0b775fc640 1 -- start start 2023-11-18T20:43:35.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.227+0000 7f0b775fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 0x7f0b700ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:35.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.227+0000 7f0b775fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7015abf0 0x7f0b700b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:35.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.227+0000 7f0b775fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b7015e730 0x7f0b700b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:35.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.227+0000 7f0b775fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700ad1e0 con 0x7f0b70143490 2023-11-18T20:43:35.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.227+0000 7f0b775fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700ad4c0 con 0x7f0b7015abf0 2023-11-18T20:43:35.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.227+0000 7f0b775fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700ad7a0 con 0x7f0b7015e730 2023-11-18T20:43:35.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.227+0000 7f0b75398640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 0x7f0b700ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.227+0000 7f0b75398640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 0x7f0b700ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44670/0 (socket says 172.21.15.60:44670) 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.227+0000 7f0b75398640 1 -- 172.21.15.60:0/2327244616 learned_addr learned my addr 172.21.15.60:0/2327244616 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b74b97640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7015abf0 0x7f0b700b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b75b99640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b7015e730 0x7f0b700b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b75398640 1 -- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b7015e730 msgr2=0x7f0b700b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b75398640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b7015e730 0x7f0b700b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b75398640 1 -- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7015abf0 msgr2=0x7f0b700b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b75398640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7015abf0 0x7f0b700b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b75398640 1 -- 172.21.15.60:0/2327244616 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6800b770 con 0x7f0b70143490 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b75398640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 0x7f0b700ac8e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0b6802e1b0 tx=0x7f0b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b6dffb640 1 -- 172.21.15.60:0/2327244616 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68036440 con 0x7f0b70143490 2023-11-18T20:43:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b6dffb640 1 -- 172.21.15.60:0/2327244616 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68034b00 con 0x7f0b70143490 2023-11-18T20:43:35.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b775fc640 1 -- 172.21.15.60:0/2327244616 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b70133b50 con 0x7f0b70143490 2023-11-18T20:43:35.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b6dffb640 1 -- 172.21.15.60:0/2327244616 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68041460 con 0x7f0b70143490 2023-11-18T20:43:35.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.228+0000 7f0b775fc640 1 -- 172.21.15.60:0/2327244616 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b700ace20 con 0x7f0b70143490 2023-11-18T20:43:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.229+0000 7f0b775fc640 1 -- 172.21.15.60:0/2327244616 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b70134070 con 0x7f0b70143490 2023-11-18T20:43:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.230+0000 7f0b6dffb640 1 -- 172.21.15.60:0/2327244616 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b6804a3b0 con 0x7f0b70143490 2023-11-18T20:43:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.230+0000 7f0b6dffb640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b5c06ae40 0x7f0b5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.230+0000 7f0b6dffb640 1 -- 172.21.15.60:0/2327244616 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b680abd40 con 0x7f0b70143490 2023-11-18T20:43:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.230+0000 7f0b74b97640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b5c06ae40 0x7f0b5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.231+0000 7f0b74b97640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b5c06ae40 0x7f0b5c06d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b701f6770 tx=0x7f0b64009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:35.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.235+0000 7f0b6dffb640 1 -- 172.21.15.60:0/2327244616 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b70134070 con 0x7f0b70143490 2023-11-18T20:43:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:35.453+0000 7f0b775fc640 1 -- 172.21.15.60:0/2327244616 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0b70134070 con 0x7f0b70143490 2023-11-18T20:43:36.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.105+0000 7f0b6dffb640 1 -- 172.21.15.60:0/2327244616 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0b70134070 con 0x7f0b70143490 2023-11-18T20:43:36.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 -- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b5c06ae40 msgr2=0x7f0b5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b5c06ae40 0x7f0b5c06d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b701f6770 tx=0x7f0b64009040 comp rx=0 tx=0).stop 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 -- 172.21.15.60:0/2327244616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 msgr2=0x7f0b700ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 0x7f0b700ac8e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0b6802e1b0 tx=0x7f0b680046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 -- 172.21.15.60:0/2327244616 shutdown_connections 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b5c06ae40 0x7f0b5c06d300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b7015e730 0x7f0b700b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b7015abf0 0x7f0b700b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 --2- 172.21.15.60:0/2327244616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b70143490 0x7f0b700ac8e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 -- 172.21.15.60:0/2327244616 >> 172.21.15.60:0/2327244616 conn(0x7f0b7006a4d0 msgr2=0x7f0b70071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 -- 172.21.15.60:0/2327244616 shutdown_connections 2023-11-18T20:43:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.107+0000 7f0b775fc640 1 -- 172.21.15.60:0/2327244616 wait complete. 2023-11-18T20:43:36.121 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-18T20:43:36.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:43:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.233+0000 7fed35da9640 1 Processor -- start 2023-11-18T20:43:36.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.233+0000 7fed35da9640 1 -- start start 2023-11-18T20:43:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed35da9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed3015d1d0 0x7fed3006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed35da9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fed300aeb80 0x7fed3006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed35da9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed300adae0 con 0x7fed300ad640 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed35da9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed300addc0 con 0x7fed3015d1d0 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed35da9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed300ae0a0 con 0x7fed300aeb80 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed2f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed3015d1d0 0x7fed3006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed2f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed3015d1d0 0x7fed3006b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44674/0 (socket says 172.21.15.60:44674) 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed2f7fe640 1 -- 172.21.15.60:0/49417367 learned_addr learned my addr 172.21.15.60:0/49417367 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed2ffff640 1 --2- 172.21.15.60:0/49417367 >> v2:172.21.15.100:3300/0 conn(0x7fed300aeb80 0x7fed3006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed2f7fe640 1 -- 172.21.15.60:0/49417367 >> v2:172.21.15.100:3300/0 conn(0x7fed300aeb80 msgr2=0x7fed3006ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed2f7fe640 1 --2- 172.21.15.60:0/49417367 >> v2:172.21.15.100:3300/0 conn(0x7fed300aeb80 0x7fed3006ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed2f7fe640 1 -- 172.21.15.60:0/49417367 >> v1:172.21.15.100:6789/0 conn(0x7fed300ad640 legacy=0x7fed30160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.234+0000 7fed2f7fe640 1 -- 172.21.15.60:0/49417367 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300ae380 con 0x7fed3015d1d0 2023-11-18T20:43:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.235+0000 7fed2f7fe640 1 --2- 172.21.15.60:0/49417367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 0x7fed3006b4e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fed30066820 tx=0x7fed2002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9069781fb6f9c7d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:36.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.235+0000 7fed2e7fc640 1 -- 172.21.15.60:0/49417367 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20030860 con 0x7fed3015d1d0 2023-11-18T20:43:36.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.235+0000 7fed2e7fc640 1 -- 172.21.15.60:0/49417367 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed200309c0 con 0x7fed3015d1d0 2023-11-18T20:43:36.238 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.235+0000 7fed2e7fc640 1 -- 172.21.15.60:0/49417367 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20030d50 con 0x7fed3015d1d0 2023-11-18T20:43:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.236+0000 7fed35da9640 1 -- 172.21.15.60:0/49417367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 msgr2=0x7fed3006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.236+0000 7fed35da9640 1 --2- 172.21.15.60:0/49417367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 0x7fed3006b4e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fed30066820 tx=0x7fed2002b9a0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.236+0000 7fed35da9640 1 -- 172.21.15.60:0/49417367 shutdown_connections 2023-11-18T20:43:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.236+0000 7fed35da9640 1 --2- 172.21.15.60:0/49417367 >> v2:172.21.15.100:3300/0 conn(0x7fed300aeb80 0x7fed3006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.236+0000 7fed35da9640 1 --2- 172.21.15.60:0/49417367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 0x7fed3006b4e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.236+0000 7fed35da9640 1 -- 172.21.15.60:0/49417367 >> 172.21.15.60:0/49417367 conn(0x7fed30070e70 msgr2=0x7fed30071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.236+0000 7fed35da9640 1 -- 172.21.15.60:0/49417367 shutdown_connections 2023-11-18T20:43:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.236+0000 7fed35da9640 1 -- 172.21.15.60:0/49417367 wait complete. 2023-11-18T20:43:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.237+0000 7fed35da9640 1 Processor -- start 2023-11-18T20:43:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.237+0000 7fed35da9640 1 -- start start 2023-11-18T20:43:36.241 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed35da9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed300ad640 0x7fed300cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed35da9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed300aeb80 0x7fed3015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed35da9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 0x7fed30157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed35da9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed30150f30 con 0x7fed3015d1d0 2023-11-18T20:43:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed35da9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3015add0 con 0x7fed300aeb80 2023-11-18T20:43:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed35da9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3015b070 con 0x7fed300ad640 2023-11-18T20:43:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed2ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 0x7fed30157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed2ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 0x7fed30157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44682/0 (socket says 172.21.15.60:44682) 2023-11-18T20:43:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed2ffff640 1 -- 172.21.15.60:0/6130127 learned_addr learned my addr 172.21.15.60:0/6130127 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:36.242 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed2f7fe640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed300ad640 0x7fed300cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed2effd640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed300aeb80 0x7fed3015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed2ffff640 1 -- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed300ad640 msgr2=0x7fed300cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed2ffff640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed300ad640 0x7fed300cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed2ffff640 1 -- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed300aeb80 msgr2=0x7fed3015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed2ffff640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed300aeb80 0x7fed3015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.238+0000 7fed2ffff640 1 -- 172.21.15.60:0/6130127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200094d0 con 0x7fed3015d1d0 2023-11-18T20:43:36.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.239+0000 7fed2ffff640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 0x7fed30157e60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fed28007e40 tx=0x7fed2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:36.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.239+0000 7fed17fff640 1 -- 172.21.15.60:0/6130127 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28014620 con 0x7fed3015d1d0 2023-11-18T20:43:36.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.239+0000 7fed17fff640 1 -- 172.21.15.60:0/6130127 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2800ee90 con 0x7fed3015d1d0 2023-11-18T20:43:36.243 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.239+0000 7fed35da9640 1 -- 172.21.15.60:0/6130127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3015b350 con 0x7fed3015d1d0 2023-11-18T20:43:36.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.239+0000 7fed17fff640 1 -- 172.21.15.60:0/6130127 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280149e0 con 0x7fed3015d1d0 2023-11-18T20:43:36.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.239+0000 7fed35da9640 1 -- 172.21.15.60:0/6130127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3015b910 con 0x7fed3015d1d0 2023-11-18T20:43:36.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.240+0000 7fed35da9640 1 -- 172.21.15.60:0/6130127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed30069400 con 0x7fed3015d1d0 2023-11-18T20:43:36.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.240+0000 7fed17fff640 1 -- 172.21.15.60:0/6130127 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed28014b40 con 0x7fed3015d1d0 2023-11-18T20:43:36.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.241+0000 7fed17fff640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed1c06ae40 0x7fed1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.241+0000 7fed17fff640 1 -- 172.21.15.60:0/6130127 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed2808a040 con 0x7fed3015d1d0 2023-11-18T20:43:36.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.241+0000 7fed2f7fe640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed1c06ae40 0x7fed1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.242+0000 7fed2f7fe640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed1c06ae40 0x7fed1c06d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fed20036000 tx=0x7fed2003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:36.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.245+0000 7fed17fff640 1 -- 172.21.15.60:0/6130127 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed280bce90 con 0x7fed3015d1d0 2023-11-18T20:43:36.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.458+0000 7fed35da9640 1 -- 172.21.15.60:0/6130127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed3015bad0 con 0x7fed3015d1d0 2023-11-18T20:43:36.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.459+0000 7fed17fff640 1 -- 172.21.15.60:0/6130127 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fed280550e0 con 0x7fed3015d1d0 2023-11-18T20:43:36.460 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:36.461 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4814,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3885363174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3885363174},{"type":"v1","addr":"172.21.15.100:6837","nonce":3885363174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1102236552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1102236552},{"type":"v1","addr":"172.21.15.60:6835","nonce":1102236552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2680685354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2680685354},{"type":"v1","addr":"172.21.15.100:6836","nonce":2680685354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:43:33.146258+0000","modified":"2023-11-18T20:43:35.454446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5065":{"gid":5065,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3762312847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3762312847},{"type":"v1","addr":"172.21.15.60:6834","nonce":3762312847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-18T20:43:36.461 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 13 2023-11-18T20:43:36.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 -- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed1c06ae40 msgr2=0x7fed1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed1c06ae40 0x7fed1c06d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fed20036000 tx=0x7fed2003b040 comp rx=0 tx=0).stop 2023-11-18T20:43:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 -- 172.21.15.60:0/6130127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 msgr2=0x7fed30157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 0x7fed30157e60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fed28007e40 tx=0x7fed2800e4b0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 -- 172.21.15.60:0/6130127 shutdown_connections 2023-11-18T20:43:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed1c06ae40 0x7fed1c06d300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed3015d1d0 0x7fed30157e60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed300aeb80 0x7fed3015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 --2- 172.21.15.60:0/6130127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed300ad640 0x7fed300cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 -- 172.21.15.60:0/6130127 >> 172.21.15.60:0/6130127 conn(0x7fed30070e70 msgr2=0x7fed3006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 -- 172.21.15.60:0/6130127 shutdown_connections 2023-11-18T20:43:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.461+0000 7fed35da9640 1 -- 172.21.15.60:0/6130127 wait complete. 2023-11-18T20:43:36.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:36.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.578+0000 7fed4cff5640 1 Processor -- start 2023-11-18T20:43:36.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.578+0000 7fed4cff5640 1 -- start start 2023-11-18T20:43:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed4cff5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fed48143490 0x7fed480b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed4cff5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed4815e730 0x7fed480b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed4cff5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4815e2a0 con 0x7fed4815a4e0 2023-11-18T20:43:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed4cff5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4815b520 con 0x7fed4815e730 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed4cff5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed48143870 con 0x7fed48143490 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed3dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed4815e730 0x7fed480b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed3dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed4815e730 0x7fed480b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44688/0 (socket says 172.21.15.60:44688) 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed3dd9b640 1 -- 172.21.15.60:0/3690719158 learned_addr learned my addr 172.21.15.60:0/3690719158 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed4659c640 1 --2- 172.21.15.60:0/3690719158 >> v2:172.21.15.100:3300/0 conn(0x7fed48143490 0x7fed480b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed3dd9b640 1 -- 172.21.15.60:0/3690719158 >> v2:172.21.15.100:3300/0 conn(0x7fed48143490 msgr2=0x7fed480b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed3dd9b640 1 --2- 172.21.15.60:0/3690719158 >> v2:172.21.15.100:3300/0 conn(0x7fed48143490 0x7fed480b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed3dd9b640 1 -- 172.21.15.60:0/3690719158 >> v1:172.21.15.100:6789/0 conn(0x7fed4815a4e0 legacy=0x7fed4815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed3dd9b640 1 -- 172.21.15.60:0/3690719158 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4815b240 con 0x7fed4815e730 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.579+0000 7fed4659c640 1 --2- 172.21.15.60:0/3690719158 >> v2:172.21.15.100:3300/0 conn(0x7fed48143490 0x7fed480b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.580+0000 7fed3dd9b640 1 --2- 172.21.15.60:0/3690719158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed4815e730 0x7fed480b1230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed4806e620 tx=0x7fed3002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec22c6f88de271af server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.580+0000 7fed45d9b640 1 -- 172.21.15.60:0/3690719158 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30030860 con 0x7fed4815e730 2023-11-18T20:43:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.580+0000 7fed45d9b640 1 -- 172.21.15.60:0/3690719158 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed300309c0 con 0x7fed4815e730 2023-11-18T20:43:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.580+0000 7fed45d9b640 1 -- 172.21.15.60:0/3690719158 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30030d50 con 0x7fed4815e730 2023-11-18T20:43:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.581+0000 7fed4cff5640 1 -- 172.21.15.60:0/3690719158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed4815e730 msgr2=0x7fed480b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.581+0000 7fed4cff5640 1 --2- 172.21.15.60:0/3690719158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed4815e730 0x7fed480b1230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed4806e620 tx=0x7fed3002b9a0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.581+0000 7fed4cff5640 1 -- 172.21.15.60:0/3690719158 shutdown_connections 2023-11-18T20:43:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.581+0000 7fed4cff5640 1 --2- 172.21.15.60:0/3690719158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed4815e730 0x7fed480b1230 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.581+0000 7fed4cff5640 1 --2- 172.21.15.60:0/3690719158 >> v2:172.21.15.100:3300/0 conn(0x7fed48143490 0x7fed480b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.581+0000 7fed4cff5640 1 -- 172.21.15.60:0/3690719158 >> 172.21.15.60:0/3690719158 conn(0x7fed4806a4d0 msgr2=0x7fed4806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.581+0000 7fed4cff5640 1 -- 172.21.15.60:0/3690719158 shutdown_connections 2023-11-18T20:43:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.581+0000 7fed4cff5640 1 -- 172.21.15.60:0/3690719158 wait complete. 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.582+0000 7fed4cff5640 1 Processor -- start 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.582+0000 7fed4cff5640 1 -- start start 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4cff5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed48143490 0x7fed4814e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4cff5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed4815a4e0 0x7fed4806c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4cff5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed4815e730 0x7fed4806c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4cff5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4815d010 con 0x7fed48143490 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4cff5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4814ed10 con 0x7fed4815a4e0 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4cff5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4815c390 con 0x7fed4815e730 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed48143490 0x7fed4814e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed48143490 0x7fed4814e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44690/0 (socket says 172.21.15.60:44690) 2023-11-18T20:43:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4659c640 1 -- 172.21.15.60:0/430169323 learned_addr learned my addr 172.21.15.60:0/430169323 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed3dd9b640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed4815a4e0 0x7fed4806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed46d9d640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed4815e730 0x7fed4806c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4659c640 1 -- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed4815e730 msgr2=0x7fed4806c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4659c640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed4815e730 0x7fed4806c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4659c640 1 -- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed4815a4e0 msgr2=0x7fed4806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.583+0000 7fed4659c640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed4815a4e0 0x7fed4806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.584+0000 7fed4659c640 1 -- 172.21.15.60:0/430169323 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300094d0 con 0x7fed48143490 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.584+0000 7fed4659c640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed48143490 0x7fed4814e7d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fed3800e730 tx=0x7fed3800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.584+0000 7fed3f7fe640 1 -- 172.21.15.60:0/430169323 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38004230 con 0x7fed48143490 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.584+0000 7fed3f7fe640 1 -- 172.21.15.60:0/430169323 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed38004390 con 0x7fed48143490 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.584+0000 7fed4cff5640 1 -- 172.21.15.60:0/430169323 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4815c670 con 0x7fed48143490 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.584+0000 7fed3f7fe640 1 -- 172.21.15.60:0/430169323 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3800c8f0 con 0x7fed48143490 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.584+0000 7fed4cff5640 1 -- 172.21.15.60:0/430169323 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4815cc30 con 0x7fed48143490 2023-11-18T20:43:36.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.585+0000 7fed4cff5640 1 -- 172.21.15.60:0/430169323 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed48069400 con 0x7fed48143490 2023-11-18T20:43:36.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.585+0000 7fed3f7fe640 1 -- 172.21.15.60:0/430169323 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed3800ca50 con 0x7fed48143490 2023-11-18T20:43:36.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.586+0000 7fed3f7fe640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed2806ae40 0x7fed2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.586+0000 7fed3f7fe640 1 -- 172.21.15.60:0/430169323 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed38089230 con 0x7fed48143490 2023-11-18T20:43:36.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.586+0000 7fed3dd9b640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed2806ae40 0x7fed2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.587+0000 7fed3dd9b640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed2806ae40 0x7fed2806d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed30036000 tx=0x7fed3003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:36.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.590+0000 7fed3f7fe640 1 -- 172.21.15.60:0/430169323 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed380542e0 con 0x7fed48143490 2023-11-18T20:43:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.766+0000 7fed4cff5640 1 -- 172.21.15.60:0/430169323 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed4815d280 con 0x7fed48143490 2023-11-18T20:43:36.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.768+0000 7fed3f7fe640 1 -- 172.21.15.60:0/430169323 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fed38057900 con 0x7fed48143490 2023-11-18T20:43:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:36.769 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":34,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:34.779504+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.769+0000 7fed4cff5640 1 -- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed2806ae40 msgr2=0x7fed2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.769+0000 7fed4cff5640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed2806ae40 0x7fed2806d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed30036000 tx=0x7fed3003b040 comp rx=0 tx=0).stop 2023-11-18T20:43:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.769+0000 7fed4cff5640 1 -- 172.21.15.60:0/430169323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed48143490 msgr2=0x7fed4814e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.769+0000 7fed4cff5640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed48143490 0x7fed4814e7d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fed3800e730 tx=0x7fed3800ebf0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.769+0000 7fed4cff5640 1 -- 172.21.15.60:0/430169323 shutdown_connections 2023-11-18T20:43:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.769+0000 7fed4cff5640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fed2806ae40 0x7fed2806d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.769+0000 7fed4cff5640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed4815e730 0x7fed4806c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.769+0000 7fed4cff5640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed4815a4e0 0x7fed4806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.769+0000 7fed4cff5640 1 --2- 172.21.15.60:0/430169323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed48143490 0x7fed4814e7d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.769+0000 7fed4cff5640 1 -- 172.21.15.60:0/430169323 >> 172.21.15.60:0/430169323 conn(0x7fed4806a4d0 msgr2=0x7fed4806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.770+0000 7fed4cff5640 1 -- 172.21.15.60:0/430169323 shutdown_connections 2023-11-18T20:43:36.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.770+0000 7fed4cff5640 1 -- 172.21.15.60:0/430169323 wait complete. 2023-11-18T20:43:36.782 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-18T20:43:36.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:43:36.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.887+0000 7fc4d501f640 1 Processor -- start 2023-11-18T20:43:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.887+0000 7fc4d501f640 1 -- start start 2023-11-18T20:43:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.888+0000 7fc4d501f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 0x7fc4d00cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.888+0000 7fc4d501f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc4d00b4370 0x7fc4d00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.888+0000 7fc4d501f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d0071770 con 0x7fc4d0144980 2023-11-18T20:43:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.888+0000 7fc4d501f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d015ced0 con 0x7fc4d00b4370 2023-11-18T20:43:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.888+0000 7fc4d501f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d00b4d20 con 0x7fc4d00ceb70 2023-11-18T20:43:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.888+0000 7fc4ced9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc4d0144980 0x7fc4d015c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37000/0 (socket says 172.21.15.60:37000) 2023-11-18T20:43:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.888+0000 7fc4ced9d640 1 -- 172.21.15.60:0/2006307828 learned_addr learned my addr 172.21.15.60:0/2006307828 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.888+0000 7fc4c7fff640 1 --2- 172.21.15.60:0/2006307828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 0x7fc4d00cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.888+0000 7fc4cf59e640 1 --2- 172.21.15.60:0/2006307828 >> v2:172.21.15.100:3300/0 conn(0x7fc4d00b4370 0x7fc4d00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.889+0000 7fc4ce59c640 1 -- 172.21.15.60:0/2006307828 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932608744 0 0) 0x7fc4d0071770 con 0x7fc4d0144980 2023-11-18T20:43:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.889+0000 7fc4ce59c640 1 -- 172.21.15.60:0/2006307828 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4bc003520 con 0x7fc4d0144980 2023-11-18T20:43:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.889+0000 7fc4c7fff640 1 -- 172.21.15.60:0/2006307828 >> v2:172.21.15.100:3300/0 conn(0x7fc4d00b4370 msgr2=0x7fc4d00b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.889+0000 7fc4c7fff640 1 --2- 172.21.15.60:0/2006307828 >> v2:172.21.15.100:3300/0 conn(0x7fc4d00b4370 0x7fc4d00b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.889+0000 7fc4c7fff640 1 -- 172.21.15.60:0/2006307828 >> v1:172.21.15.60:6789/0 conn(0x7fc4d0144980 legacy=0x7fc4d015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.889+0000 7fc4c7fff640 1 -- 172.21.15.60:0/2006307828 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d00ccdc0 con 0x7fc4d00ceb70 2023-11-18T20:43:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4c7fff640 1 --2- 172.21.15.60:0/2006307828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 0x7fc4d00cc450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80097a0 tx=0x7fc4b802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb369e3110bf9a63 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4ce59c640 1 -- 172.21.15.60:0/2006307828 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8031830 con 0x7fc4d00ceb70 2023-11-18T20:43:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4ce59c640 1 -- 172.21.15.60:0/2006307828 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8031990 con 0x7fc4d00ceb70 2023-11-18T20:43:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4ce59c640 1 -- 172.21.15.60:0/2006307828 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8031d00 con 0x7fc4d00ceb70 2023-11-18T20:43:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4d501f640 1 -- 172.21.15.60:0/2006307828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 msgr2=0x7fc4d00cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4d501f640 1 --2- 172.21.15.60:0/2006307828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 0x7fc4d00cc450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80097a0 tx=0x7fc4b802b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4d501f640 1 -- 172.21.15.60:0/2006307828 shutdown_connections 2023-11-18T20:43:36.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4d501f640 1 --2- 172.21.15.60:0/2006307828 >> v2:172.21.15.100:3300/0 conn(0x7fc4d00b4370 0x7fc4d00b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4d501f640 1 --2- 172.21.15.60:0/2006307828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 0x7fc4d00cc450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4d501f640 1 -- 172.21.15.60:0/2006307828 >> 172.21.15.60:0/2006307828 conn(0x7fc4d006a4d0 msgr2=0x7fc4d006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:36.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4d501f640 1 -- 172.21.15.60:0/2006307828 shutdown_connections 2023-11-18T20:43:36.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.890+0000 7fc4d501f640 1 -- 172.21.15.60:0/2006307828 wait complete. 2023-11-18T20:43:36.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.892+0000 7fc4d501f640 1 Processor -- start 2023-11-18T20:43:36.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.892+0000 7fc4d501f640 1 -- start start 2023-11-18T20:43:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.892+0000 7fc4d501f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d00b4370 0x7fc4d0140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.892+0000 7fc4d501f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 0x7fc4d012f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.892+0000 7fc4d501f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4d0144980 0x7fc4d012fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.892+0000 7fc4d501f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0155290 con 0x7fc4d0144980 2023-11-18T20:43:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.892+0000 7fc4d501f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d01578a0 con 0x7fc4d00b4370 2023-11-18T20:43:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.892+0000 7fc4d501f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0157b80 con 0x7fc4d00ceb70 2023-11-18T20:43:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4cf59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4d0144980 0x7fc4d012fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4cf59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4d0144980 0x7fc4d012fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44694/0 (socket says 172.21.15.60:44694) 2023-11-18T20:43:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4cf59e640 1 -- 172.21.15.60:0/2667827720 learned_addr learned my addr 172.21.15.60:0/2667827720 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4ced9d640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d00b4370 0x7fc4d0140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4c7fff640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 0x7fc4d012f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4cf59e640 1 -- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 msgr2=0x7fc4d012f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4cf59e640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 0x7fc4d012f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4cf59e640 1 -- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d00b4370 msgr2=0x7fc4d0140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4cf59e640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d00b4370 0x7fc4d0140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4cf59e640 1 -- 172.21.15.60:0/2667827720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b8009400 con 0x7fc4d0144980 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4cf59e640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4d0144980 0x7fc4d012fcf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800efc0 tx=0x7fc4c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4c77fe640 1 -- 172.21.15.60:0/2667827720 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8009ba0 con 0x7fc4d0144980 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4c77fe640 1 -- 172.21.15.60:0/2667827720 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c800f040 con 0x7fc4d0144980 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4d501f640 1 -- 172.21.15.60:0/2667827720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d0157e60 con 0x7fc4d0144980 2023-11-18T20:43:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4c77fe640 1 -- 172.21.15.60:0/2667827720 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c80146c0 con 0x7fc4d0144980 2023-11-18T20:43:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.893+0000 7fc4d501f640 1 -- 172.21.15.60:0/2667827720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d0158420 con 0x7fc4d0144980 2023-11-18T20:43:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.894+0000 7fc4d501f640 1 -- 172.21.15.60:0/2667827720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d0158610 con 0x7fc4d0144980 2023-11-18T20:43:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.895+0000 7fc4c77fe640 1 -- 172.21.15.60:0/2667827720 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc4c8014820 con 0x7fc4d0144980 2023-11-18T20:43:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.895+0000 7fc4c77fe640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4b006ae70 0x7fc4b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.895+0000 7fc4c77fe640 1 -- 172.21.15.60:0/2667827720 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4c808a150 con 0x7fc4d0144980 2023-11-18T20:43:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.895+0000 7fc4ced9d640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4b006ae70 0x7fc4b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.896+0000 7fc4ced9d640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4b006ae70 0x7fc4b006d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0007fd0 tx=0x7fc4c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:36.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:36.899+0000 7fc4c77fe640 1 -- 172.21.15.60:0/2667827720 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc4c8055180 con 0x7fc4d0144980 2023-11-18T20:43:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.072+0000 7fc4d501f640 1 -- 172.21.15.60:0/2667827720 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4d0071400 con 0x7fc4b006ae70 2023-11-18T20:43:37.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.081+0000 7fc4c77fe640 1 -- 172.21.15.60:0/2667827720 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289515 (secure 0 0 0) 0x7fc4c00013f0 con 0x7fc4b006ae70 2023-11-18T20:43:37.085 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:37.086 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:43:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 -- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4b006ae70 msgr2=0x7fc4b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4b006ae70 0x7fc4b006d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0007fd0 tx=0x7fc4c0009040 comp rx=0 tx=0).stop 2023-11-18T20:43:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 -- 172.21.15.60:0/2667827720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4d0144980 msgr2=0x7fc4d012fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4d0144980 0x7fc4d012fcf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800efc0 tx=0x7fc4c800c490 comp rx=0 tx=0).stop 2023-11-18T20:43:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 -- 172.21.15.60:0/2667827720 shutdown_connections 2023-11-18T20:43:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc4b006ae70 0x7fc4b006d330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4d0144980 0x7fc4d012fcf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4d00ceb70 0x7fc4d012f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 --2- 172.21.15.60:0/2667827720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4d00b4370 0x7fc4d0140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 -- 172.21.15.60:0/2667827720 >> 172.21.15.60:0/2667827720 conn(0x7fc4d006a4d0 msgr2=0x7fc4d0141220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 -- 172.21.15.60:0/2667827720 shutdown_connections 2023-11-18T20:43:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.084+0000 7fc4d501f640 1 -- 172.21.15.60:0/2667827720 wait complete. 2023-11-18T20:43:37.096 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-18T20:43:35.449986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236792,"kb_used_data":3984,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737928,"statfs":{"total":773094113280,"available":772851638272,"internally_reserved":0,"allocated":4079616,"data_stored":2292396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000798"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.804790+0000","last_change":"2023-11-18T20:43:33.800432+0000","last_active":"2023-11-18T20:43:33.804790+0000","last_peered":"2023-11-18T20:43:33.804790+0000","last_clean":"2023-11-18T20:43:33.804790+0000","last_became_active":"2023-11-18T20:43:33.800158+0000","last_became_peered":"2023-11-18T20:43:33.800158+0000","last_unstale":"2023-11-18T20:43:33.804790+0000","last_undegraded":"2023-11-18T20:43:33.804790+0000","last_fullsized":"2023-11-18T20:43:33.804790+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:12:53.159267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.803544+0000","last_change":"2023-11-18T20:43:33.802355+0000","last_active":"2023-11-18T20:43:33.803544+0000","last_peered":"2023-11-18T20:43:33.803544+0000","last_clean":"2023-11-18T20:43:33.803544+0000","last_became_active":"2023-11-18T20:43:33.801934+0000","last_became_peered":"2023-11-18T20:43:33.801934+0000","last_unstale":"2023-11-18T20:43:33.803544+0000","last_undegraded":"2023-11-18T20:43:33.803544+0000","last_fullsized":"2023-11-18T20:43:33.803544+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:40:23.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807355+0000","last_change":"2023-11-18T20:43:33.800584+0000","last_active":"2023-11-18T20:43:33.807355+0000","last_peered":"2023-11-18T20:43:33.807355+0000","last_clean":"2023-11-18T20:43:33.807355+0000","last_became_active":"2023-11-18T20:43:33.800326+0000","last_became_peered":"2023-11-18T20:43:33.800326+0000","last_unstale":"2023-11-18T20:43:33.807355+0000","last_undegraded":"2023-11-18T20:43:33.807355+0000","last_fullsized":"2023-11-18T20:43:33.807355+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:34:56.639700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.801083+0000","last_change":"2023-11-18T20:43:33.798327+0000","last_active":"2023-11-18T20:43:33.801083+0000","last_peered":"2023-11-18T20:43:33.801083+0000","last_clean":"2023-11-18T20:43:33.801083+0000","last_became_active":"2023-11-18T20:43:33.790560+0000","last_became_peered":"2023-11-18T20:43:33.790560+0000","last_unstale":"2023-11-18T20:43:33.801083+0000","last_undegraded":"2023-11-18T20:43:33.801083+0000","last_fullsized":"2023-11-18T20:43:33.801083+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:34.983437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.795043+0000","last_change":"2023-11-18T20:43:33.791163+0000","last_active":"2023-11-18T20:43:33.795043+0000","last_peered":"2023-11-18T20:43:33.795043+0000","last_clean":"2023-11-18T20:43:33.795043+0000","last_became_active":"2023-11-18T20:43:33.790799+0000","last_became_peered":"2023-11-18T20:43:33.790799+0000","last_unstale":"2023-11-18T20:43:33.795043+0000","last_undegraded":"2023-11-18T20:43:33.795043+0000","last_fullsized":"2023-11-18T20:43:33.795043+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:10.088474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.801008+0000","last_change":"2023-11-18T20:43:33.798089+0000","last_active":"2023-11-18T20:43:33.801008+0000","last_peered":"2023-11-18T20:43:33.801008+0000","last_clean":"2023-11-18T20:43:33.801008+0000","last_became_active":"2023-11-18T20:43:33.797769+0000","last_became_peered":"2023-11-18T20:43:33.797769+0000","last_unstale":"2023-11-18T20:43:33.801008+0000","last_undegraded":"2023-11-18T20:43:33.801008+0000","last_fullsized":"2023-11-18T20:43:33.801008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:13.981406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.799975+0000","last_change":"2023-11-18T20:43:33.795891+0000","last_active":"2023-11-18T20:43:33.799975+0000","last_peered":"2023-11-18T20:43:33.799975+0000","last_clean":"2023-11-18T20:43:33.799975+0000","last_became_active":"2023-11-18T20:43:33.795604+0000","last_became_peered":"2023-11-18T20:43:33.795604+0000","last_unstale":"2023-11-18T20:43:33.799975+0000","last_undegraded":"2023-11-18T20:43:33.799975+0000","last_fullsized":"2023-11-18T20:43:33.799975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:58:12.405990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.799935+0000","last_change":"2023-11-18T20:43:33.796971+0000","last_active":"2023-11-18T20:43:33.799935+0000","last_peered":"2023-11-18T20:43:33.799935+0000","last_clean":"2023-11-18T20:43:33.799935+0000","last_became_active":"2023-11-18T20:43:33.796667+0000","last_became_peered":"2023-11-18T20:43:33.796667+0000","last_unstale":"2023-11-18T20:43:33.799935+0000","last_undegraded":"2023-11-18T20:43:33.799935+0000","last_fullsized":"2023-11-18T20:43:33.799935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:34:56.639700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798604+0000","last_change":"2023-11-18T20:43:33.792060+0000","last_active":"2023-11-18T20:43:33.798604+0000","last_peered":"2023-11-18T20:43:33.798604+0000","last_clean":"2023-11-18T20:43:33.798604+0000","last_became_active":"2023-11-18T20:43:33.791790+0000","last_became_peered":"2023-11-18T20:43:33.791790+0000","last_unstale":"2023-11-18T20:43:33.798604+0000","last_undegraded":"2023-11-18T20:43:33.798604+0000","last_fullsized":"2023-11-18T20:43:33.798604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:47:54.978135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.801198+0000","last_change":"2023-11-18T20:43:33.800118+0000","last_active":"2023-11-18T20:43:33.801198+0000","last_peered":"2023-11-18T20:43:33.801198+0000","last_clean":"2023-11-18T20:43:33.801198+0000","last_became_active":"2023-11-18T20:43:33.799816+0000","last_became_peered":"2023-11-18T20:43:33.799816+0000","last_unstale":"2023-11-18T20:43:33.801198+0000","last_undegraded":"2023-11-18T20:43:33.801198+0000","last_fullsized":"2023-11-18T20:43:33.801198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:18.711038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807413+0000","last_change":"2023-11-18T20:43:33.801605+0000","last_active":"2023-11-18T20:43:33.807413+0000","last_peered":"2023-11-18T20:43:33.807413+0000","last_clean":"2023-11-18T20:43:33.807413+0000","last_became_active":"2023-11-18T20:43:33.801337+0000","last_became_peered":"2023-11-18T20:43:33.801337+0000","last_unstale":"2023-11-18T20:43:33.807413+0000","last_undegraded":"2023-11-18T20:43:33.807413+0000","last_fullsized":"2023-11-18T20:43:33.807413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:13.981406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798521+0000","last_change":"2023-11-18T20:43:33.795597+0000","last_active":"2023-11-18T20:43:33.798521+0000","last_peered":"2023-11-18T20:43:33.798521+0000","last_clean":"2023-11-18T20:43:33.798521+0000","last_became_active":"2023-11-18T20:43:33.795334+0000","last_became_peered":"2023-11-18T20:43:33.795334+0000","last_unstale":"2023-11-18T20:43:33.798521+0000","last_undegraded":"2023-11-18T20:43:33.798521+0000","last_fullsized":"2023-11-18T20:43:33.798521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:56:23.474195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.786675+0000","last_change":"2023-11-18T20:43:33.786019+0000","last_active":"2023-11-18T20:43:33.786675+0000","last_peered":"2023-11-18T20:43:33.786675+0000","last_clean":"2023-11-18T20:43:33.786675+0000","last_became_active":"2023-11-18T20:43:33.785684+0000","last_became_peered":"2023-11-18T20:43:33.785684+0000","last_unstale":"2023-11-18T20:43:33.786675+0000","last_undegraded":"2023-11-18T20:43:33.786675+0000","last_fullsized":"2023-11-18T20:43:33.786675+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:05:56.809560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798812+0000","last_change":"2023-11-18T20:43:33.795983+0000","last_active":"2023-11-18T20:43:33.798812+0000","last_peered":"2023-11-18T20:43:33.798812+0000","last_clean":"2023-11-18T20:43:33.798812+0000","last_became_active":"2023-11-18T20:43:33.795689+0000","last_became_peered":"2023-11-18T20:43:33.795689+0000","last_unstale":"2023-11-18T20:43:33.798812+0000","last_undegraded":"2023-11-18T20:43:33.798812+0000","last_fullsized":"2023-11-18T20:43:33.798812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:04:51.358886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.804774+0000","last_change":"2023-11-18T20:43:33.798893+0000","last_active":"2023-11-18T20:43:33.804774+0000","last_peered":"2023-11-18T20:43:33.804774+0000","last_clean":"2023-11-18T20:43:33.804774+0000","last_became_active":"2023-11-18T20:43:33.798516+0000","last_became_peered":"2023-11-18T20:43:33.798516+0000","last_unstale":"2023-11-18T20:43:33.804774+0000","last_undegraded":"2023-11-18T20:43:33.804774+0000","last_fullsized":"2023-11-18T20:43:33.804774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:32:05.273998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.804621+0000","last_change":"2023-11-18T20:43:33.799828+0000","last_active":"2023-11-18T20:43:33.804621+0000","last_peered":"2023-11-18T20:43:33.804621+0000","last_clean":"2023-11-18T20:43:33.804621+0000","last_became_active":"2023-11-18T20:43:33.799564+0000","last_became_peered":"2023-11-18T20:43:33.799564+0000","last_unstale":"2023-11-18T20:43:33.804621+0000","last_undegraded":"2023-11-18T20:43:33.804621+0000","last_fullsized":"2023-11-18T20:43:33.804621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:40:23.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800028+0000","last_change":"2023-11-18T20:43:33.795938+0000","last_active":"2023-11-18T20:43:33.800028+0000","last_peered":"2023-11-18T20:43:33.800028+0000","last_clean":"2023-11-18T20:43:33.800028+0000","last_became_active":"2023-11-18T20:43:33.795569+0000","last_became_peered":"2023-11-18T20:43:33.795569+0000","last_unstale":"2023-11-18T20:43:33.800028+0000","last_undegraded":"2023-11-18T20:43:33.800028+0000","last_fullsized":"2023-11-18T20:43:33.800028+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:56:23.474195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.804905+0000","last_change":"2023-11-18T20:43:33.804285+0000","last_active":"2023-11-18T20:43:33.804905+0000","last_peered":"2023-11-18T20:43:33.804905+0000","last_clean":"2023-11-18T20:43:33.804905+0000","last_became_active":"2023-11-18T20:43:33.803960+0000","last_became_peered":"2023-11-18T20:43:33.803960+0000","last_unstale":"2023-11-18T20:43:33.804905+0000","last_undegraded":"2023-11-18T20:43:33.804905+0000","last_fullsized":"2023-11-18T20:43:33.804905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:42.882485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800963+0000","last_change":"2023-11-18T20:43:33.798064+0000","last_active":"2023-11-18T20:43:33.800963+0000","last_peered":"2023-11-18T20:43:33.800963+0000","last_clean":"2023-11-18T20:43:33.800963+0000","last_became_active":"2023-11-18T20:43:33.797789+0000","last_became_peered":"2023-11-18T20:43:33.797789+0000","last_unstale":"2023-11-18T20:43:33.800963+0000","last_undegraded":"2023-11-18T20:43:33.800963+0000","last_fullsized":"2023-11-18T20:43:33.800963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:32.362994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.796757+0000","last_change":"2023-11-18T20:43:33.796121+0000","last_active":"2023-11-18T20:43:33.796757+0000","last_peered":"2023-11-18T20:43:33.796757+0000","last_clean":"2023-11-18T20:43:33.796757+0000","last_became_active":"2023-11-18T20:43:33.795860+0000","last_became_peered":"2023-11-18T20:43:33.795860+0000","last_unstale":"2023-11-18T20:43:33.796757+0000","last_undegraded":"2023-11-18T20:43:33.796757+0000","last_fullsized":"2023-11-18T20:43:33.796757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:49:45.680897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.792586+0000","last_change":"2023-11-18T20:43:33.791329+0000","last_active":"2023-11-18T20:43:33.792586+0000","last_peered":"2023-11-18T20:43:33.792586+0000","last_clean":"2023-11-18T20:43:33.792586+0000","last_became_active":"2023-11-18T20:43:33.790929+0000","last_became_peered":"2023-11-18T20:43:33.790929+0000","last_unstale":"2023-11-18T20:43:33.792586+0000","last_undegraded":"2023-11-18T20:43:33.792586+0000","last_fullsized":"2023-11-18T20:43:33.792586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:31.306866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807575+0000","last_change":"2023-11-18T20:43:33.800433+0000","last_active":"2023-11-18T20:43:33.807575+0000","last_peered":"2023-11-18T20:43:33.807575+0000","last_clean":"2023-11-18T20:43:33.807575+0000","last_became_active":"2023-11-18T20:43:33.800148+0000","last_became_peered":"2023-11-18T20:43:33.800148+0000","last_unstale":"2023-11-18T20:43:33.807575+0000","last_undegraded":"2023-11-18T20:43:33.807575+0000","last_fullsized":"2023-11-18T20:43:33.807575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:26:40.774256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.797702+0000","last_change":"2023-11-18T20:43:33.794332+0000","last_active":"2023-11-18T20:43:33.797702+0000","last_peered":"2023-11-18T20:43:33.797702+0000","last_clean":"2023-11-18T20:43:33.797702+0000","last_became_active":"2023-11-18T20:43:33.794022+0000","last_became_peered":"2023-11-18T20:43:33.794022+0000","last_unstale":"2023-11-18T20:43:33.797702+0000","last_undegraded":"2023-11-18T20:43:33.797702+0000","last_fullsized":"2023-11-18T20:43:33.797702+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:32:55.637084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.794282+0000","last_change":"2023-11-18T20:43:33.791594+0000","last_active":"2023-11-18T20:43:33.794282+0000","last_peered":"2023-11-18T20:43:33.794282+0000","last_clean":"2023-11-18T20:43:33.794282+0000","last_became_active":"2023-11-18T20:43:33.791279+0000","last_became_peered":"2023-11-18T20:43:33.791279+0000","last_unstale":"2023-11-18T20:43:33.794282+0000","last_undegraded":"2023-11-18T20:43:33.794282+0000","last_fullsized":"2023-11-18T20:43:33.794282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:18.711038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800342+0000","last_change":"2023-11-18T20:43:33.796577+0000","last_active":"2023-11-18T20:43:33.800342+0000","last_peered":"2023-11-18T20:43:33.800342+0000","last_clean":"2023-11-18T20:43:33.800342+0000","last_became_active":"2023-11-18T20:43:33.796312+0000","last_became_peered":"2023-11-18T20:43:33.796312+0000","last_unstale":"2023-11-18T20:43:33.800342+0000","last_undegraded":"2023-11-18T20:43:33.800342+0000","last_fullsized":"2023-11-18T20:43:33.800342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:32.362994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.797713+0000","last_change":"2023-11-18T20:43:33.794492+0000","last_active":"2023-11-18T20:43:33.797713+0000","last_peered":"2023-11-18T20:43:33.797713+0000","last_clean":"2023-11-18T20:43:33.797713+0000","last_became_active":"2023-11-18T20:43:33.794213+0000","last_became_peered":"2023-11-18T20:43:33.794213+0000","last_unstale":"2023-11-18T20:43:33.797713+0000","last_undegraded":"2023-11-18T20:43:33.797713+0000","last_fullsized":"2023-11-18T20:43:33.797713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:56:23.474195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.792632+0000","last_change":"2023-11-18T20:43:33.790591+0000","last_active":"2023-11-18T20:43:33.792632+0000","last_peered":"2023-11-18T20:43:33.792632+0000","last_clean":"2023-11-18T20:43:33.792632+0000","last_became_active":"2023-11-18T20:43:33.790246+0000","last_became_peered":"2023-11-18T20:43:33.790246+0000","last_unstale":"2023-11-18T20:43:33.792632+0000","last_undegraded":"2023-11-18T20:43:33.792632+0000","last_fullsized":"2023-11-18T20:43:33.792632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:32:05.273998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798501+0000","last_change":"2023-11-18T20:43:33.794844+0000","last_active":"2023-11-18T20:43:33.798501+0000","last_peered":"2023-11-18T20:43:33.798501+0000","last_clean":"2023-11-18T20:43:33.798501+0000","last_became_active":"2023-11-18T20:43:33.794564+0000","last_became_peered":"2023-11-18T20:43:33.794564+0000","last_unstale":"2023-11-18T20:43:33.798501+0000","last_undegraded":"2023-11-18T20:43:33.798501+0000","last_fullsized":"2023-11-18T20:43:33.798501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:37.759359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807616+0000","last_change":"2023-11-18T20:43:33.800946+0000","last_active":"2023-11-18T20:43:33.807616+0000","last_peered":"2023-11-18T20:43:33.807616+0000","last_clean":"2023-11-18T20:43:33.807616+0000","last_became_active":"2023-11-18T20:43:33.800397+0000","last_became_peered":"2023-11-18T20:43:33.800397+0000","last_unstale":"2023-11-18T20:43:33.807616+0000","last_undegraded":"2023-11-18T20:43:33.807616+0000","last_fullsized":"2023-11-18T20:43:33.807616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:56:18.827236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.799239+0000","last_change":"2023-11-18T20:43:33.795175+0000","last_active":"2023-11-18T20:43:33.799239+0000","last_peered":"2023-11-18T20:43:33.799239+0000","last_clean":"2023-11-18T20:43:33.799239+0000","last_became_active":"2023-11-18T20:43:33.794845+0000","last_became_peered":"2023-11-18T20:43:33.794845+0000","last_unstale":"2023-11-18T20:43:33.799239+0000","last_undegraded":"2023-11-18T20:43:33.799239+0000","last_fullsized":"2023-11-18T20:43:33.799239+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:32.362994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800676+0000","last_change":"2023-11-18T20:43:33.797636+0000","last_active":"2023-11-18T20:43:33.800676+0000","last_peered":"2023-11-18T20:43:33.800676+0000","last_clean":"2023-11-18T20:43:33.800676+0000","last_became_active":"2023-11-18T20:43:33.797311+0000","last_became_peered":"2023-11-18T20:43:33.797311+0000","last_unstale":"2023-11-18T20:43:33.800676+0000","last_undegraded":"2023-11-18T20:43:33.800676+0000","last_fullsized":"2023-11-18T20:43:33.800676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:32.362994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.813018+0000","last_change":"2023-11-18T20:43:33.806648+0000","last_active":"2023-11-18T20:43:33.813018+0000","last_peered":"2023-11-18T20:43:33.813018+0000","last_clean":"2023-11-18T20:43:33.813018+0000","last_became_active":"2023-11-18T20:43:33.806113+0000","last_became_peered":"2023-11-18T20:43:33.806113+0000","last_unstale":"2023-11-18T20:43:33.813018+0000","last_undegraded":"2023-11-18T20:43:33.813018+0000","last_fullsized":"2023-11-18T20:43:33.813018+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:04:51.358886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.801663+0000","last_change":"2023-11-18T20:43:33.800284+0000","last_active":"2023-11-18T20:43:33.801663+0000","last_peered":"2023-11-18T20:43:33.801663+0000","last_clean":"2023-11-18T20:43:33.801663+0000","last_became_active":"2023-11-18T20:43:33.800020+0000","last_became_peered":"2023-11-18T20:43:33.800020+0000","last_unstale":"2023-11-18T20:43:33.801663+0000","last_undegraded":"2023-11-18T20:43:33.801663+0000","last_fullsized":"2023-11-18T20:43:33.801663+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:34.983437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.115774+0000","last_change":"2023-11-18T20:43:32.789148+0000","last_active":"2023-11-18T20:43:34.115774+0000","last_peered":"2023-11-18T20:43:34.115774+0000","last_clean":"2023-11-18T20:43:34.115774+0000","last_became_active":"2023-11-18T20:43:32.788721+0000","last_became_peered":"2023-11-18T20:43:32.788721+0000","last_unstale":"2023-11-18T20:43:34.115774+0000","last_undegraded":"2023-11-18T20:43:34.115774+0000","last_fullsized":"2023-11-18T20:43:34.115774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:33.979328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.812956+0000","last_change":"2023-11-18T20:43:33.805898+0000","last_active":"2023-11-18T20:43:33.812956+0000","last_peered":"2023-11-18T20:43:33.812956+0000","last_clean":"2023-11-18T20:43:33.812956+0000","last_became_active":"2023-11-18T20:43:33.805576+0000","last_became_peered":"2023-11-18T20:43:33.805576+0000","last_unstale":"2023-11-18T20:43:33.812956+0000","last_undegraded":"2023-11-18T20:43:33.812956+0000","last_fullsized":"2023-11-18T20:43:33.812956+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:40:08.752775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.113073+0000","last_change":"2023-11-18T20:43:32.786552+0000","last_active":"2023-11-18T20:43:34.113073+0000","last_peered":"2023-11-18T20:43:34.113073+0000","last_clean":"2023-11-18T20:43:34.113073+0000","last_became_active":"2023-11-18T20:43:32.786271+0000","last_became_peered":"2023-11-18T20:43:32.786271+0000","last_unstale":"2023-11-18T20:43:34.113073+0000","last_undegraded":"2023-11-18T20:43:34.113073+0000","last_fullsized":"2023-11-18T20:43:34.113073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:17.706929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807686+0000","last_change":"2023-11-18T20:43:33.799693+0000","last_active":"2023-11-18T20:43:33.807686+0000","last_peered":"2023-11-18T20:43:33.807686+0000","last_clean":"2023-11-18T20:43:33.807686+0000","last_became_active":"2023-11-18T20:43:33.799375+0000","last_became_peered":"2023-11-18T20:43:33.799375+0000","last_unstale":"2023-11-18T20:43:33.807686+0000","last_undegraded":"2023-11-18T20:43:33.807686+0000","last_fullsized":"2023-11-18T20:43:33.807686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:35:36.747539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.112923+0000","last_change":"2023-11-18T20:43:32.786101+0000","last_active":"2023-11-18T20:43:34.112923+0000","last_peered":"2023-11-18T20:43:34.112923+0000","last_clean":"2023-11-18T20:43:34.112923+0000","last_became_active":"2023-11-18T20:43:32.785754+0000","last_became_peered":"2023-11-18T20:43:32.785754+0000","last_unstale":"2023-11-18T20:43:34.112923+0000","last_undegraded":"2023-11-18T20:43:34.112923+0000","last_fullsized":"2023-11-18T20:43:34.112923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:36:53.103501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.790778+0000","last_change":"2023-11-18T20:43:33.788363+0000","last_active":"2023-11-18T20:43:33.790778+0000","last_peered":"2023-11-18T20:43:33.790778+0000","last_clean":"2023-11-18T20:43:33.790778+0000","last_became_active":"2023-11-18T20:43:33.788104+0000","last_became_peered":"2023-11-18T20:43:33.788104+0000","last_unstale":"2023-11-18T20:43:33.790778+0000","last_undegraded":"2023-11-18T20:43:33.790778+0000","last_fullsized":"2023-11-18T20:43:33.790778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:12:53.159267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.111039+0000","last_change":"2023-11-18T20:43:32.787841+0000","last_active":"2023-11-18T20:43:34.111039+0000","last_peered":"2023-11-18T20:43:34.111039+0000","last_clean":"2023-11-18T20:43:34.111039+0000","last_became_active":"2023-11-18T20:43:32.787545+0000","last_became_peered":"2023-11-18T20:43:32.787545+0000","last_unstale":"2023-11-18T20:43:34.111039+0000","last_undegraded":"2023-11-18T20:43:34.111039+0000","last_fullsized":"2023-11-18T20:43:34.111039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:18:42.906284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.813002+0000","last_change":"2023-11-18T20:43:33.805980+0000","last_active":"2023-11-18T20:43:33.813002+0000","last_peered":"2023-11-18T20:43:33.813002+0000","last_clean":"2023-11-18T20:43:33.813002+0000","last_became_active":"2023-11-18T20:43:33.805671+0000","last_became_peered":"2023-11-18T20:43:33.805671+0000","last_unstale":"2023-11-18T20:43:33.813002+0000","last_undegraded":"2023-11-18T20:43:33.813002+0000","last_fullsized":"2023-11-18T20:43:33.813002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:37.759359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.115729+0000","last_change":"2023-11-18T20:43:32.790506+0000","last_active":"2023-11-18T20:43:34.115729+0000","last_peered":"2023-11-18T20:43:34.115729+0000","last_clean":"2023-11-18T20:43:34.115729+0000","last_became_active":"2023-11-18T20:43:32.790228+0000","last_became_peered":"2023-11-18T20:43:32.790228+0000","last_unstale":"2023-11-18T20:43:34.115729+0000","last_undegraded":"2023-11-18T20:43:34.115729+0000","last_fullsized":"2023-11-18T20:43:34.115729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:12.977297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807486+0000","last_change":"2023-11-18T20:43:33.801617+0000","last_active":"2023-11-18T20:43:33.807486+0000","last_peered":"2023-11-18T20:43:33.807486+0000","last_clean":"2023-11-18T20:43:33.807486+0000","last_became_active":"2023-11-18T20:43:33.801286+0000","last_became_peered":"2023-11-18T20:43:33.801286+0000","last_unstale":"2023-11-18T20:43:33.807486+0000","last_undegraded":"2023-11-18T20:43:33.807486+0000","last_fullsized":"2023-11-18T20:43:33.807486+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:05:54.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.112861+0000","last_change":"2023-11-18T20:43:32.789438+0000","last_active":"2023-11-18T20:43:34.112861+0000","last_peered":"2023-11-18T20:43:34.112861+0000","last_clean":"2023-11-18T20:43:34.112861+0000","last_became_active":"2023-11-18T20:43:32.788839+0000","last_became_peered":"2023-11-18T20:43:32.788839+0000","last_unstale":"2023-11-18T20:43:34.112861+0000","last_undegraded":"2023-11-18T20:43:34.112861+0000","last_fullsized":"2023-11-18T20:43:34.112861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:33.979328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807550+0000","last_change":"2023-11-18T20:43:33.800718+0000","last_active":"2023-11-18T20:43:33.807550+0000","last_peered":"2023-11-18T20:43:33.807550+0000","last_clean":"2023-11-18T20:43:33.807550+0000","last_became_active":"2023-11-18T20:43:33.798535+0000","last_became_peered":"2023-11-18T20:43:33.798535+0000","last_unstale":"2023-11-18T20:43:33.807550+0000","last_undegraded":"2023-11-18T20:43:33.807550+0000","last_fullsized":"2023-11-18T20:43:33.807550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:19:33.538932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.791205+0000","last_change":"2023-11-18T20:43:32.784541+0000","last_active":"2023-11-18T20:43:33.791205+0000","last_peered":"2023-11-18T20:43:33.791205+0000","last_clean":"2023-11-18T20:43:33.791205+0000","last_became_active":"2023-11-18T20:43:32.784259+0000","last_became_peered":"2023-11-18T20:43:32.784259+0000","last_unstale":"2023-11-18T20:43:33.791205+0000","last_undegraded":"2023-11-18T20:43:33.791205+0000","last_fullsized":"2023-11-18T20:43:33.791205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:58:11.401881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800364+0000","last_change":"2023-11-18T20:43:33.795628+0000","last_active":"2023-11-18T20:43:33.800364+0000","last_peered":"2023-11-18T20:43:33.800364+0000","last_clean":"2023-11-18T20:43:33.800364+0000","last_became_active":"2023-11-18T20:43:33.795259+0000","last_became_peered":"2023-11-18T20:43:33.795259+0000","last_unstale":"2023-11-18T20:43:33.800364+0000","last_undegraded":"2023-11-18T20:43:33.800364+0000","last_fullsized":"2023-11-18T20:43:33.800364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:32:55.637084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.783620+0000","last_change":"2023-11-18T20:43:32.791435+0000","last_active":"2023-11-18T20:43:33.783620+0000","last_peered":"2023-11-18T20:43:33.783620+0000","last_clean":"2023-11-18T20:43:33.783620+0000","last_became_active":"2023-11-18T20:43:32.791075+0000","last_became_peered":"2023-11-18T20:43:32.791075+0000","last_unstale":"2023-11-18T20:43:33.783620+0000","last_undegraded":"2023-11-18T20:43:33.783620+0000","last_fullsized":"2023-11-18T20:43:33.783620+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:40:07.748665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800693+0000","last_change":"2023-11-18T20:43:33.796504+0000","last_active":"2023-11-18T20:43:33.800693+0000","last_peered":"2023-11-18T20:43:33.800693+0000","last_clean":"2023-11-18T20:43:33.800693+0000","last_became_active":"2023-11-18T20:43:33.796239+0000","last_became_peered":"2023-11-18T20:43:33.796239+0000","last_unstale":"2023-11-18T20:43:33.800693+0000","last_undegraded":"2023-11-18T20:43:33.800693+0000","last_fullsized":"2023-11-18T20:43:33.800693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:32.362994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.791735+0000","last_change":"2023-11-18T20:43:32.787724+0000","last_active":"2023-11-18T20:43:33.791735+0000","last_peered":"2023-11-18T20:43:33.791735+0000","last_clean":"2023-11-18T20:43:33.791735+0000","last_became_active":"2023-11-18T20:43:32.787451+0000","last_became_peered":"2023-11-18T20:43:32.787451+0000","last_unstale":"2023-11-18T20:43:33.791735+0000","last_undegraded":"2023-11-18T20:43:33.791735+0000","last_fullsized":"2023-11-18T20:43:33.791735+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:04:35.722671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.794868+0000","last_change":"2023-11-18T20:43:33.793661+0000","last_active":"2023-11-18T20:43:33.794868+0000","last_peered":"2023-11-18T20:43:33.794868+0000","last_clean":"2023-11-18T20:43:33.794868+0000","last_became_active":"2023-11-18T20:43:33.793410+0000","last_became_peered":"2023-11-18T20:43:33.793410+0000","last_unstale":"2023-11-18T20:43:33.794868+0000","last_undegraded":"2023-11-18T20:43:33.794868+0000","last_fullsized":"2023-11-18T20:43:33.794868+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:37.759359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.786256+0000","last_change":"2023-11-18T20:43:32.789378+0000","last_active":"2023-11-18T20:43:33.786256+0000","last_peered":"2023-11-18T20:43:33.786256+0000","last_clean":"2023-11-18T20:43:33.786256+0000","last_became_active":"2023-11-18T20:43:32.788879+0000","last_became_peered":"2023-11-18T20:43:32.788879+0000","last_unstale":"2023-11-18T20:43:33.786256+0000","last_undegraded":"2023-11-18T20:43:33.786256+0000","last_fullsized":"2023-11-18T20:43:33.786256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:31.358885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798794+0000","last_change":"2023-11-18T20:43:33.795955+0000","last_active":"2023-11-18T20:43:33.798794+0000","last_peered":"2023-11-18T20:43:33.798794+0000","last_clean":"2023-11-18T20:43:33.798794+0000","last_became_active":"2023-11-18T20:43:33.795679+0000","last_became_peered":"2023-11-18T20:43:33.795679+0000","last_unstale":"2023-11-18T20:43:33.798794+0000","last_undegraded":"2023-11-18T20:43:33.798794+0000","last_fullsized":"2023-11-18T20:43:33.798794+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:32:05.273998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.113092+0000","last_change":"2023-11-18T20:43:32.792377+0000","last_active":"2023-11-18T20:43:34.113092+0000","last_peered":"2023-11-18T20:43:34.113092+0000","last_clean":"2023-11-18T20:43:34.113092+0000","last_became_active":"2023-11-18T20:43:32.792087+0000","last_became_peered":"2023-11-18T20:43:32.792087+0000","last_unstale":"2023-11-18T20:43:34.113092+0000","last_undegraded":"2023-11-18T20:43:34.113092+0000","last_fullsized":"2023-11-18T20:43:34.113092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:40:22.845618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807589+0000","last_change":"2023-11-18T20:43:33.802336+0000","last_active":"2023-11-18T20:43:33.807589+0000","last_peered":"2023-11-18T20:43:33.807589+0000","last_clean":"2023-11-18T20:43:33.807589+0000","last_became_active":"2023-11-18T20:43:33.802058+0000","last_became_peered":"2023-11-18T20:43:33.802058+0000","last_unstale":"2023-11-18T20:43:33.807589+0000","last_undegraded":"2023-11-18T20:43:33.807589+0000","last_fullsized":"2023-11-18T20:43:33.807589+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:04:51.358886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.117642+0000","last_change":"2023-11-18T20:43:32.788567+0000","last_active":"2023-11-18T20:43:34.117642+0000","last_peered":"2023-11-18T20:43:34.117642+0000","last_clean":"2023-11-18T20:43:34.117642+0000","last_became_active":"2023-11-18T20:43:32.788211+0000","last_became_peered":"2023-11-18T20:43:32.788211+0000","last_unstale":"2023-11-18T20:43:34.117642+0000","last_undegraded":"2023-11-18T20:43:34.117642+0000","last_fullsized":"2023-11-18T20:43:34.117642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:15:30.342072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800333+0000","last_change":"2023-11-18T20:43:33.795595+0000","last_active":"2023-11-18T20:43:33.800333+0000","last_peered":"2023-11-18T20:43:33.800333+0000","last_clean":"2023-11-18T20:43:33.800333+0000","last_became_active":"2023-11-18T20:43:33.795177+0000","last_became_peered":"2023-11-18T20:43:33.795177+0000","last_unstale":"2023-11-18T20:43:33.800333+0000","last_undegraded":"2023-11-18T20:43:33.800333+0000","last_fullsized":"2023-11-18T20:43:33.800333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:37.759359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.116057+0000","last_change":"2023-11-18T20:43:32.791294+0000","last_active":"2023-11-18T20:43:34.116057+0000","last_peered":"2023-11-18T20:43:34.116057+0000","last_clean":"2023-11-18T20:43:34.116057+0000","last_became_active":"2023-11-18T20:43:32.790980+0000","last_became_peered":"2023-11-18T20:43:32.790980+0000","last_unstale":"2023-11-18T20:43:34.116057+0000","last_undegraded":"2023-11-18T20:43:34.116057+0000","last_fullsized":"2023-11-18T20:43:34.116057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:36.755249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.801010+0000","last_change":"2023-11-18T20:43:33.798954+0000","last_active":"2023-11-18T20:43:33.801010+0000","last_peered":"2023-11-18T20:43:33.801010+0000","last_clean":"2023-11-18T20:43:33.801010+0000","last_became_active":"2023-11-18T20:43:33.798420+0000","last_became_peered":"2023-11-18T20:43:33.798420+0000","last_unstale":"2023-11-18T20:43:33.801010+0000","last_undegraded":"2023-11-18T20:43:33.801010+0000","last_fullsized":"2023-11-18T20:43:33.801010+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:34:56.639700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.784526+0000","last_change":"2023-11-18T20:43:32.789906+0000","last_active":"2023-11-18T20:43:33.784526+0000","last_peered":"2023-11-18T20:43:33.784526+0000","last_clean":"2023-11-18T20:43:33.784526+0000","last_became_active":"2023-11-18T20:43:32.789582+0000","last_became_peered":"2023-11-18T20:43:32.789582+0000","last_unstale":"2023-11-18T20:43:33.784526+0000","last_undegraded":"2023-11-18T20:43:33.784526+0000","last_fullsized":"2023-11-18T20:43:33.784526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:56:22.470085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.812996+0000","last_change":"2023-11-18T20:43:33.805882+0000","last_active":"2023-11-18T20:43:33.812996+0000","last_peered":"2023-11-18T20:43:33.812996+0000","last_clean":"2023-11-18T20:43:33.812996+0000","last_became_active":"2023-11-18T20:43:33.805599+0000","last_became_peered":"2023-11-18T20:43:33.805599+0000","last_unstale":"2023-11-18T20:43:33.812996+0000","last_undegraded":"2023-11-18T20:43:33.812996+0000","last_fullsized":"2023-11-18T20:43:33.812996+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:42.882485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.791358+0000","last_change":"2023-11-18T20:43:32.783841+0000","last_active":"2023-11-18T20:43:33.791358+0000","last_peered":"2023-11-18T20:43:33.791358+0000","last_clean":"2023-11-18T20:43:33.791358+0000","last_became_active":"2023-11-18T20:43:32.783556+0000","last_became_peered":"2023-11-18T20:43:32.783556+0000","last_unstale":"2023-11-18T20:43:33.791358+0000","last_undegraded":"2023-11-18T20:43:33.791358+0000","last_fullsized":"2023-11-18T20:43:33.791358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:42:11.680696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807555+0000","last_change":"2023-11-18T20:43:33.805482+0000","last_active":"2023-11-18T20:43:33.807555+0000","last_peered":"2023-11-18T20:43:33.807555+0000","last_clean":"2023-11-18T20:43:33.807555+0000","last_became_active":"2023-11-18T20:43:33.805179+0000","last_became_peered":"2023-11-18T20:43:33.805179+0000","last_unstale":"2023-11-18T20:43:33.807555+0000","last_undegraded":"2023-11-18T20:43:33.807555+0000","last_fullsized":"2023-11-18T20:43:33.807555+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:32:55.637084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.113384+0000","last_change":"2023-11-18T20:43:32.788989+0000","last_active":"2023-11-18T20:43:34.113384+0000","last_peered":"2023-11-18T20:43:34.113384+0000","last_clean":"2023-11-18T20:43:34.113384+0000","last_became_active":"2023-11-18T20:43:32.788717+0000","last_became_peered":"2023-11-18T20:43:32.788717+0000","last_unstale":"2023-11-18T20:43:34.113384+0000","last_undegraded":"2023-11-18T20:43:34.113384+0000","last_fullsized":"2023-11-18T20:43:34.113384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:05:53.021163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.804929+0000","last_change":"2023-11-18T20:43:33.796585+0000","last_active":"2023-11-18T20:43:33.804929+0000","last_peered":"2023-11-18T20:43:33.804929+0000","last_clean":"2023-11-18T20:43:33.804929+0000","last_became_active":"2023-11-18T20:43:33.796276+0000","last_became_peered":"2023-11-18T20:43:33.796276+0000","last_unstale":"2023-11-18T20:43:33.804929+0000","last_undegraded":"2023-11-18T20:43:33.804929+0000","last_fullsized":"2023-11-18T20:43:33.804929+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:59:35.675350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.111976+0000","last_change":"2023-11-18T20:43:32.789162+0000","last_active":"2023-11-18T20:43:34.111976+0000","last_peered":"2023-11-18T20:43:34.111976+0000","last_clean":"2023-11-18T20:43:34.111976+0000","last_became_active":"2023-11-18T20:43:32.788613+0000","last_became_peered":"2023-11-18T20:43:32.788613+0000","last_unstale":"2023-11-18T20:43:34.111976+0000","last_undegraded":"2023-11-18T20:43:34.111976+0000","last_fullsized":"2023-11-18T20:43:34.111976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:04:50.354776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807823+0000","last_change":"2023-11-18T20:43:33.805816+0000","last_active":"2023-11-18T20:43:33.807823+0000","last_peered":"2023-11-18T20:43:33.807823+0000","last_clean":"2023-11-18T20:43:33.807823+0000","last_became_active":"2023-11-18T20:43:33.805545+0000","last_became_peered":"2023-11-18T20:43:33.805545+0000","last_unstale":"2023-11-18T20:43:33.807823+0000","last_undegraded":"2023-11-18T20:43:33.807823+0000","last_fullsized":"2023-11-18T20:43:33.807823+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:58:12.405990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.115965+0000","last_change":"2023-11-18T20:43:32.792303+0000","last_active":"2023-11-18T20:43:34.115965+0000","last_peered":"2023-11-18T20:43:34.115965+0000","last_clean":"2023-11-18T20:43:34.115965+0000","last_became_active":"2023-11-18T20:43:32.790966+0000","last_became_peered":"2023-11-18T20:43:32.790966+0000","last_unstale":"2023-11-18T20:43:34.115965+0000","last_undegraded":"2023-11-18T20:43:34.115965+0000","last_fullsized":"2023-11-18T20:43:34.115965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:32:54.632975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.794782+0000","last_change":"2023-11-18T20:43:33.787059+0000","last_active":"2023-11-18T20:43:33.794782+0000","last_peered":"2023-11-18T20:43:33.794782+0000","last_clean":"2023-11-18T20:43:33.794782+0000","last_became_active":"2023-11-18T20:43:33.786697+0000","last_became_peered":"2023-11-18T20:43:33.786697+0000","last_unstale":"2023-11-18T20:43:33.794782+0000","last_undegraded":"2023-11-18T20:43:33.794782+0000","last_fullsized":"2023-11-18T20:43:33.794782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:42:24.039703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.115829+0000","last_change":"2023-11-18T20:43:32.788325+0000","last_active":"2023-11-18T20:43:34.115829+0000","last_peered":"2023-11-18T20:43:34.115829+0000","last_clean":"2023-11-18T20:43:34.115829+0000","last_became_active":"2023-11-18T20:43:32.787872+0000","last_became_peered":"2023-11-18T20:43:32.787872+0000","last_unstale":"2023-11-18T20:43:34.115829+0000","last_undegraded":"2023-11-18T20:43:34.115829+0000","last_fullsized":"2023-11-18T20:43:34.115829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:42:23.035593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.796289+0000","last_change":"2023-11-18T20:43:33.794602+0000","last_active":"2023-11-18T20:43:33.796289+0000","last_peered":"2023-11-18T20:43:33.796289+0000","last_clean":"2023-11-18T20:43:33.796289+0000","last_became_active":"2023-11-18T20:43:33.793716+0000","last_became_peered":"2023-11-18T20:43:33.793716+0000","last_unstale":"2023-11-18T20:43:33.796289+0000","last_undegraded":"2023-11-18T20:43:33.796289+0000","last_fullsized":"2023-11-18T20:43:33.796289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:45:21.602664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.783762+0000","last_change":"2023-11-18T20:43:32.789855+0000","last_active":"2023-11-18T20:43:33.783762+0000","last_peered":"2023-11-18T20:43:33.783762+0000","last_clean":"2023-11-18T20:43:33.783762+0000","last_became_active":"2023-11-18T20:43:32.789541+0000","last_became_peered":"2023-11-18T20:43:32.789541+0000","last_unstale":"2023-11-18T20:43:33.783762+0000","last_undegraded":"2023-11-18T20:43:33.783762+0000","last_fullsized":"2023-11-18T20:43:33.783762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:12:52.155157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.787861+0000","last_change":"2023-11-18T20:43:33.786320+0000","last_active":"2023-11-18T20:43:33.787861+0000","last_peered":"2023-11-18T20:43:33.787861+0000","last_clean":"2023-11-18T20:43:33.787861+0000","last_became_active":"2023-11-18T20:43:33.785918+0000","last_became_peered":"2023-11-18T20:43:33.785918+0000","last_unstale":"2023-11-18T20:43:33.787861+0000","last_undegraded":"2023-11-18T20:43:33.787861+0000","last_fullsized":"2023-11-18T20:43:33.787861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:42:10.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.111518+0000","last_change":"2023-11-18T20:43:32.788060+0000","last_active":"2023-11-18T20:43:34.111518+0000","last_peered":"2023-11-18T20:43:34.111518+0000","last_clean":"2023-11-18T20:43:34.111518+0000","last_became_active":"2023-11-18T20:43:32.787744+0000","last_became_peered":"2023-11-18T20:43:32.787744+0000","last_unstale":"2023-11-18T20:43:34.111518+0000","last_undegraded":"2023-11-18T20:43:34.111518+0000","last_fullsized":"2023-11-18T20:43:34.111518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:12.977297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.793252+0000","last_change":"2023-11-18T20:43:33.789461+0000","last_active":"2023-11-18T20:43:33.793252+0000","last_peered":"2023-11-18T20:43:33.793252+0000","last_clean":"2023-11-18T20:43:33.793252+0000","last_became_active":"2023-11-18T20:43:33.789177+0000","last_became_peered":"2023-11-18T20:43:33.789177+0000","last_unstale":"2023-11-18T20:43:33.793252+0000","last_undegraded":"2023-11-18T20:43:33.793252+0000","last_fullsized":"2023-11-18T20:43:33.793252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:16:01.555895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.787905+0000","last_change":"2023-11-18T20:43:32.789919+0000","last_active":"2023-11-18T20:43:33.787905+0000","last_peered":"2023-11-18T20:43:33.787905+0000","last_clean":"2023-11-18T20:43:33.787905+0000","last_became_active":"2023-11-18T20:43:32.789645+0000","last_became_peered":"2023-11-18T20:43:32.789645+0000","last_unstale":"2023-11-18T20:43:33.787905+0000","last_undegraded":"2023-11-18T20:43:33.787905+0000","last_fullsized":"2023-11-18T20:43:33.787905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:17.706929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.791213+0000","last_change":"2023-11-18T20:43:33.788805+0000","last_active":"2023-11-18T20:43:33.791213+0000","last_peered":"2023-11-18T20:43:33.791213+0000","last_clean":"2023-11-18T20:43:33.791213+0000","last_became_active":"2023-11-18T20:43:33.788552+0000","last_became_peered":"2023-11-18T20:43:33.788552+0000","last_unstale":"2023-11-18T20:43:33.791213+0000","last_undegraded":"2023-11-18T20:43:33.791213+0000","last_fullsized":"2023-11-18T20:43:33.791213+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:48:59.977613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.783983+0000","last_change":"2023-11-18T20:43:32.786191+0000","last_active":"2023-11-18T20:43:33.783983+0000","last_peered":"2023-11-18T20:43:33.783983+0000","last_clean":"2023-11-18T20:43:33.783983+0000","last_became_active":"2023-11-18T20:43:32.785923+0000","last_became_peered":"2023-11-18T20:43:32.785923+0000","last_unstale":"2023-11-18T20:43:33.783983+0000","last_undegraded":"2023-11-18T20:43:33.783983+0000","last_fullsized":"2023-11-18T20:43:33.783983+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:17.706929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.799995+0000","last_change":"2023-11-18T20:43:33.797837+0000","last_active":"2023-11-18T20:43:33.799995+0000","last_peered":"2023-11-18T20:43:33.799995+0000","last_clean":"2023-11-18T20:43:33.799995+0000","last_became_active":"2023-11-18T20:43:33.797519+0000","last_became_peered":"2023-11-18T20:43:33.797519+0000","last_unstale":"2023-11-18T20:43:33.799995+0000","last_undegraded":"2023-11-18T20:43:33.799995+0000","last_fullsized":"2023-11-18T20:43:33.799995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:05:54.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.787087+0000","last_change":"2023-11-18T20:43:32.789154+0000","last_active":"2023-11-18T20:43:33.787087+0000","last_peered":"2023-11-18T20:43:33.787087+0000","last_clean":"2023-11-18T20:43:33.787087+0000","last_became_active":"2023-11-18T20:43:32.788885+0000","last_became_peered":"2023-11-18T20:43:32.788885+0000","last_unstale":"2023-11-18T20:43:33.787087+0000","last_undegraded":"2023-11-18T20:43:33.787087+0000","last_fullsized":"2023-11-18T20:43:33.787087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:12.977297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800483+0000","last_change":"2023-11-18T20:43:33.796869+0000","last_active":"2023-11-18T20:43:33.800483+0000","last_peered":"2023-11-18T20:43:33.800483+0000","last_clean":"2023-11-18T20:43:33.800483+0000","last_became_active":"2023-11-18T20:43:33.796534+0000","last_became_peered":"2023-11-18T20:43:33.796534+0000","last_unstale":"2023-11-18T20:43:33.800483+0000","last_undegraded":"2023-11-18T20:43:33.800483+0000","last_fullsized":"2023-11-18T20:43:33.800483+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:36:53.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.113546+0000","last_change":"2023-11-18T20:43:32.791936+0000","last_active":"2023-11-18T20:43:34.113546+0000","last_peered":"2023-11-18T20:43:34.113546+0000","last_clean":"2023-11-18T20:43:34.113546+0000","last_became_active":"2023-11-18T20:43:32.791417+0000","last_became_peered":"2023-11-18T20:43:32.791417+0000","last_unstale":"2023-11-18T20:43:34.113546+0000","last_undegraded":"2023-11-18T20:43:34.113546+0000","last_fullsized":"2023-11-18T20:43:34.113546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:41.878375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807563+0000","last_change":"2023-11-18T20:43:33.801148+0000","last_active":"2023-11-18T20:43:33.807563+0000","last_peered":"2023-11-18T20:43:33.807563+0000","last_clean":"2023-11-18T20:43:33.807563+0000","last_became_active":"2023-11-18T20:43:33.800838+0000","last_became_peered":"2023-11-18T20:43:33.800838+0000","last_unstale":"2023-11-18T20:43:33.807563+0000","last_undegraded":"2023-11-18T20:43:33.807563+0000","last_fullsized":"2023-11-18T20:43:33.807563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:42.882485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.786407+0000","last_change":"2023-11-18T20:43:32.789195+0000","last_active":"2023-11-18T20:43:33.786407+0000","last_peered":"2023-11-18T20:43:33.786407+0000","last_clean":"2023-11-18T20:43:33.786407+0000","last_became_active":"2023-11-18T20:43:32.788462+0000","last_became_peered":"2023-11-18T20:43:32.788462+0000","last_unstale":"2023-11-18T20:43:33.786407+0000","last_undegraded":"2023-11-18T20:43:33.786407+0000","last_fullsized":"2023-11-18T20:43:33.786407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:36.755249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.797690+0000","last_change":"2023-11-18T20:43:33.795963+0000","last_active":"2023-11-18T20:43:33.797690+0000","last_peered":"2023-11-18T20:43:33.797690+0000","last_clean":"2023-11-18T20:43:33.797690+0000","last_became_active":"2023-11-18T20:43:33.794640+0000","last_became_peered":"2023-11-18T20:43:33.794640+0000","last_unstale":"2023-11-18T20:43:33.797690+0000","last_undegraded":"2023-11-18T20:43:33.797690+0000","last_fullsized":"2023-11-18T20:43:33.797690+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:12:53.159267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.784135+0000","last_change":"2023-11-18T20:43:32.787804+0000","last_active":"2023-11-18T20:43:33.784135+0000","last_peered":"2023-11-18T20:43:33.784135+0000","last_clean":"2023-11-18T20:43:33.784135+0000","last_became_active":"2023-11-18T20:43:32.787427+0000","last_became_peered":"2023-11-18T20:43:32.787427+0000","last_unstale":"2023-11-18T20:43:33.784135+0000","last_undegraded":"2023-11-18T20:43:33.784135+0000","last_fullsized":"2023-11-18T20:43:33.784135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:12.977297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.797982+0000","last_change":"2023-11-18T20:43:33.794711+0000","last_active":"2023-11-18T20:43:33.797982+0000","last_peered":"2023-11-18T20:43:33.797982+0000","last_clean":"2023-11-18T20:43:33.797982+0000","last_became_active":"2023-11-18T20:43:33.794422+0000","last_became_peered":"2023-11-18T20:43:33.794422+0000","last_unstale":"2023-11-18T20:43:33.797982+0000","last_undegraded":"2023-11-18T20:43:33.797982+0000","last_fullsized":"2023-11-18T20:43:33.797982+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:42:24.039703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.786505+0000","last_change":"2023-11-18T20:43:32.791063+0000","last_active":"2023-11-18T20:43:33.786505+0000","last_peered":"2023-11-18T20:43:33.786505+0000","last_clean":"2023-11-18T20:43:33.786505+0000","last_became_active":"2023-11-18T20:43:32.790779+0000","last_became_peered":"2023-11-18T20:43:32.790779+0000","last_unstale":"2023-11-18T20:43:33.786505+0000","last_undegraded":"2023-11-18T20:43:33.786505+0000","last_fullsized":"2023-11-18T20:43:33.786505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:56:22.470085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798645+0000","last_change":"2023-11-18T20:43:33.797248+0000","last_active":"2023-11-18T20:43:33.798645+0000","last_peered":"2023-11-18T20:43:33.798645+0000","last_clean":"2023-11-18T20:43:33.798645+0000","last_became_active":"2023-11-18T20:43:33.796993+0000","last_became_peered":"2023-11-18T20:43:33.796993+0000","last_unstale":"2023-11-18T20:43:33.798645+0000","last_undegraded":"2023-11-18T20:43:33.798645+0000","last_fullsized":"2023-11-18T20:43:33.798645+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:42:24.039703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.784667+0000","last_change":"2023-11-18T20:43:32.791620+0000","last_active":"2023-11-18T20:43:33.784667+0000","last_peered":"2023-11-18T20:43:33.784667+0000","last_clean":"2023-11-18T20:43:33.784667+0000","last_became_active":"2023-11-18T20:43:32.791314+0000","last_became_peered":"2023-11-18T20:43:32.791314+0000","last_unstale":"2023-11-18T20:43:33.784667+0000","last_undegraded":"2023-11-18T20:43:33.784667+0000","last_fullsized":"2023-11-18T20:43:33.784667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:34:55.635591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.797752+0000","last_change":"2023-11-18T20:43:33.794620+0000","last_active":"2023-11-18T20:43:33.797752+0000","last_peered":"2023-11-18T20:43:33.797752+0000","last_clean":"2023-11-18T20:43:33.797752+0000","last_became_active":"2023-11-18T20:43:33.794327+0000","last_became_peered":"2023-11-18T20:43:33.794327+0000","last_unstale":"2023-11-18T20:43:33.797752+0000","last_undegraded":"2023-11-18T20:43:33.797752+0000","last_fullsized":"2023-11-18T20:43:33.797752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:18.711038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.784135+0000","last_change":"2023-11-18T20:43:32.786778+0000","last_active":"2023-11-18T20:43:33.784135+0000","last_peered":"2023-11-18T20:43:33.784135+0000","last_clean":"2023-11-18T20:43:33.784135+0000","last_became_active":"2023-11-18T20:43:32.786509+0000","last_became_peered":"2023-11-18T20:43:32.786509+0000","last_unstale":"2023-11-18T20:43:33.784135+0000","last_undegraded":"2023-11-18T20:43:33.784135+0000","last_fullsized":"2023-11-18T20:43:33.784135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:31.358885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807633+0000","last_change":"2023-11-18T20:43:33.801524+0000","last_active":"2023-11-18T20:43:33.807633+0000","last_peered":"2023-11-18T20:43:33.807633+0000","last_clean":"2023-11-18T20:43:33.807633+0000","last_became_active":"2023-11-18T20:43:33.801068+0000","last_became_peered":"2023-11-18T20:43:33.801068+0000","last_unstale":"2023-11-18T20:43:33.807633+0000","last_undegraded":"2023-11-18T20:43:33.807633+0000","last_fullsized":"2023-11-18T20:43:33.807633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:56:06.214458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.791360+0000","last_change":"2023-11-18T20:43:32.788614+0000","last_active":"2023-11-18T20:43:33.791360+0000","last_peered":"2023-11-18T20:43:33.791360+0000","last_clean":"2023-11-18T20:43:33.791360+0000","last_became_active":"2023-11-18T20:43:32.788171+0000","last_became_peered":"2023-11-18T20:43:32.788171+0000","last_unstale":"2023-11-18T20:43:33.791360+0000","last_undegraded":"2023-11-18T20:43:33.791360+0000","last_fullsized":"2023-11-18T20:43:33.791360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:04:24.315192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.799980+0000","last_change":"2023-11-18T20:43:33.798571+0000","last_active":"2023-11-18T20:43:33.799980+0000","last_peered":"2023-11-18T20:43:33.799980+0000","last_clean":"2023-11-18T20:43:33.799980+0000","last_became_active":"2023-11-18T20:43:33.798309+0000","last_became_peered":"2023-11-18T20:43:33.798309+0000","last_unstale":"2023-11-18T20:43:33.799980+0000","last_undegraded":"2023-11-18T20:43:33.799980+0000","last_fullsized":"2023-11-18T20:43:33.799980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:32:05.273998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.786854+0000","last_change":"2023-11-18T20:43:32.791542+0000","last_active":"2023-11-18T20:43:33.786854+0000","last_peered":"2023-11-18T20:43:33.786854+0000","last_clean":"2023-11-18T20:43:33.786854+0000","last_became_active":"2023-11-18T20:43:32.791233+0000","last_became_peered":"2023-11-18T20:43:32.791233+0000","last_unstale":"2023-11-18T20:43:33.786854+0000","last_undegraded":"2023-11-18T20:43:33.786854+0000","last_fullsized":"2023-11-18T20:43:33.786854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:56:17.823126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-18T20:43:32.783982+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:43:32.783982+0000","last_peered":"2023-11-18T20:43:32.783982+0000","last_clean":"2023-11-18T20:43:32.783982+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:43:32.783982+0000","last_undegraded":"2023-11-18T20:43:32.783982+0000","last_fullsized":"2023-11-18T20:43:32.783982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":954368,"data_stored":728556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.496}]},{"osd":1,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.432}]},{"osd":2,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51600000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55000000000000004}]},{"osd":4,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.33400000000000002}]},{"osd":5,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.315},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.40799999999999997}]},{"osd":6,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.56799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.376}]},{"osd":1,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42599999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.35299999999999998}]},{"osd":7,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.628}]},{"osd":1,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47799999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.52000000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.502},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.66200000000000003}]},{"osd":6,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56100000000000005}]},{"osd":7,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.67800000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57599999999999996}]},{"osd":1,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.621}]},{"osd":2,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.504}]},{"osd":3,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.46800000000000003}]},{"osd":5,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.51900000000000002}]},{"osd":6,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.44800000000000001}]},{"osd":7,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.314},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.252}]},{"osd":1,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.314},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.38100000000000001}]},{"osd":2,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.41599999999999998}]},{"osd":4,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57899999999999996}]},{"osd":5,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72799999999999998}]},{"osd":6,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.58799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.311},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.35399999999999998}]},{"osd":1,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.254},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.27100000000000002}]},{"osd":3,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33300000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.54800000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.29899999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.246},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34799999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.40799999999999997}]},{"osd":4,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.436},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47499999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56499999999999995}]},{"osd":6,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.502}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":962560,"data_stored":734712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.217},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.33900000000000002}]},{"osd":2,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.247},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.28100000000000003}]},{"osd":3,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.376}]},{"osd":4,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57399999999999995}]},{"osd":5,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:43:37.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:43:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.243+0000 7fca76061640 1 Processor -- start 2023-11-18T20:43:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.243+0000 7fca76061640 1 -- start start 2023-11-18T20:43:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca76061640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fca70140310 0x7fca701618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca76061640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca7006abe0 0x7fca7006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca76061640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca701406f0 con 0x7fca701409d0 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca76061640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca70161e20 con 0x7fca70140310 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca76061640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca70143960 con 0x7fca7006abe0 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca6f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fca70140310 0x7fca701618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca6effd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fca701409d0 0x7fca7006a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37006/0 (socket says 172.21.15.60:37006) 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca6f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fca70140310 0x7fca701618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44696/0 (socket says 172.21.15.60:44696) 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca6effd640 1 -- 172.21.15.60:0/917324205 learned_addr learned my addr 172.21.15.60:0/917324205 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca6ffff640 1 --2- 172.21.15.60:0/917324205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca7006abe0 0x7fca7006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca6f7fe640 1 -- 172.21.15.60:0/917324205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca7006abe0 msgr2=0x7fca7006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca6f7fe640 1 --2- 172.21.15.60:0/917324205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca7006abe0 0x7fca7006efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca6f7fe640 1 -- 172.21.15.60:0/917324205 >> v1:172.21.15.60:6789/0 conn(0x7fca701409d0 legacy=0x7fca7006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca6f7fe640 1 -- 172.21.15.60:0/917324205 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca70142de0 con 0x7fca70140310 2023-11-18T20:43:37.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.244+0000 7fca6e7fc640 1 -- 172.21.15.60:0/917324205 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477735636 0 0) 0x7fca701406f0 con 0x7fca701409d0 2023-11-18T20:43:37.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.245+0000 7fca6f7fe640 1 --2- 172.21.15.60:0/917324205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 0x7fca701618e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fca600097a0 tx=0x7fca6002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3652db601a713ff server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:37.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.245+0000 7fca6e7fc640 1 -- 172.21.15.60:0/917324205 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6002e070 con 0x7fca70140310 2023-11-18T20:43:37.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.245+0000 7fca6e7fc640 1 -- 172.21.15.60:0/917324205 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca600315a0 con 0x7fca70140310 2023-11-18T20:43:37.248 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.245+0000 7fca6e7fc640 1 -- 172.21.15.60:0/917324205 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca60031930 con 0x7fca70140310 2023-11-18T20:43:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.246+0000 7fca76061640 1 -- 172.21.15.60:0/917324205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 msgr2=0x7fca701618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.246+0000 7fca76061640 1 --2- 172.21.15.60:0/917324205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 0x7fca701618e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fca600097a0 tx=0x7fca6002b840 comp rx=0 tx=0).stop 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.246+0000 7fca76061640 1 -- 172.21.15.60:0/917324205 shutdown_connections 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.246+0000 7fca76061640 1 --2- 172.21.15.60:0/917324205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca7006abe0 0x7fca7006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.246+0000 7fca76061640 1 --2- 172.21.15.60:0/917324205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 0x7fca701618e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.246+0000 7fca76061640 1 -- 172.21.15.60:0/917324205 >> 172.21.15.60:0/917324205 conn(0x7fca700b1e50 msgr2=0x7fca700b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.246+0000 7fca76061640 1 -- 172.21.15.60:0/917324205 shutdown_connections 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.246+0000 7fca76061640 1 -- 172.21.15.60:0/917324205 wait complete. 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.247+0000 7fca76061640 1 Processor -- start 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.247+0000 7fca76061640 1 -- start start 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.248+0000 7fca76061640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca7006abe0 0x7fca7013e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.248+0000 7fca76061640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 0x7fca70156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.248+0000 7fca76061640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca701409d0 0x7fca70144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.248+0000 7fca76061640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca700ad560 con 0x7fca70140310 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.248+0000 7fca76061640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca700ad840 con 0x7fca701409d0 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.248+0000 7fca76061640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca700adb20 con 0x7fca7006abe0 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.248+0000 7fca6effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 0x7fca70156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.248+0000 7fca6effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 0x7fca70156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44702/0 (socket says 172.21.15.60:44702) 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.248+0000 7fca6effd640 1 -- 172.21.15.60:0/4123758075 learned_addr learned my addr 172.21.15.60:0/4123758075 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca6f7fe640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca7006abe0 0x7fca7013e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca6ffff640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca701409d0 0x7fca70144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca6effd640 1 -- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca7006abe0 msgr2=0x7fca7013e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca6effd640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca7006abe0 0x7fca7013e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca6effd640 1 -- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca701409d0 msgr2=0x7fca70144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca6effd640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca701409d0 0x7fca70144760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca6effd640 1 -- 172.21.15.60:0/4123758075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca60009400 con 0x7fca70140310 2023-11-18T20:43:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca6effd640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 0x7fca70156bf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fca6400c4d0 tx=0x7fca6400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca57fff640 1 -- 172.21.15.60:0/4123758075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca640074e0 con 0x7fca70140310 2023-11-18T20:43:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca57fff640 1 -- 172.21.15.60:0/4123758075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca64007660 con 0x7fca70140310 2023-11-18T20:43:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca76061640 1 -- 172.21.15.60:0/4123758075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca700ade00 con 0x7fca70140310 2023-11-18T20:43:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca57fff640 1 -- 172.21.15.60:0/4123758075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64004a20 con 0x7fca70140310 2023-11-18T20:43:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.249+0000 7fca76061640 1 -- 172.21.15.60:0/4123758075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7013eeb0 con 0x7fca70140310 2023-11-18T20:43:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.250+0000 7fca76061640 1 -- 172.21.15.60:0/4123758075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca700ae320 con 0x7fca70140310 2023-11-18T20:43:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.251+0000 7fca57fff640 1 -- 172.21.15.60:0/4123758075 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca64004b80 con 0x7fca70140310 2023-11-18T20:43:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.251+0000 7fca57fff640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca5c06ae70 0x7fca5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.251+0000 7fca57fff640 1 -- 172.21.15.60:0/4123758075 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca64088b00 con 0x7fca70140310 2023-11-18T20:43:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.251+0000 7fca6f7fe640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca5c06ae70 0x7fca5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.252+0000 7fca6f7fe640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca5c06ae70 0x7fca5c06d330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fca60036000 tx=0x7fca6003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:37.256 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.255+0000 7fca57fff640 1 -- 172.21.15.60:0/4123758075 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca700ae320 con 0x7fca70140310 2023-11-18T20:43:37.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.430+0000 7fca76061640 1 -- 172.21.15.60:0/4123758075 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca700b2350 con 0x7fca5c06ae70 2023-11-18T20:43:37.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.438+0000 7fca57fff640 1 -- 172.21.15.60:0/4123758075 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289515 (secure 0 0 0) 0x7fca700b2350 con 0x7fca5c06ae70 2023-11-18T20:43:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:37.443 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:43:37.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 -- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca5c06ae70 msgr2=0x7fca5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca5c06ae70 0x7fca5c06d330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fca60036000 tx=0x7fca6003d040 comp rx=0 tx=0).stop 2023-11-18T20:43:37.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 -- 172.21.15.60:0/4123758075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 msgr2=0x7fca70156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 0x7fca70156bf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fca6400c4d0 tx=0x7fca6400c990 comp rx=0 tx=0).stop 2023-11-18T20:43:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 -- 172.21.15.60:0/4123758075 shutdown_connections 2023-11-18T20:43:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca5c06ae70 0x7fca5c06d330 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca701409d0 0x7fca70144760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca70140310 0x7fca70156bf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 --2- 172.21.15.60:0/4123758075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca7006abe0 0x7fca7013e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 -- 172.21.15.60:0/4123758075 >> 172.21.15.60:0/4123758075 conn(0x7fca700b1e50 msgr2=0x7fca7015dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 -- 172.21.15.60:0/4123758075 shutdown_connections 2023-11-18T20:43:37.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.441+0000 7fca76061640 1 -- 172.21.15.60:0/4123758075 wait complete. 2023-11-18T20:43:37.453 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-18T20:43:35.449986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236792,"kb_used_data":3984,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737928,"statfs":{"total":773094113280,"available":772851638272,"internally_reserved":0,"allocated":4079616,"data_stored":2292396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000798"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.804790+0000","last_change":"2023-11-18T20:43:33.800432+0000","last_active":"2023-11-18T20:43:33.804790+0000","last_peered":"2023-11-18T20:43:33.804790+0000","last_clean":"2023-11-18T20:43:33.804790+0000","last_became_active":"2023-11-18T20:43:33.800158+0000","last_became_peered":"2023-11-18T20:43:33.800158+0000","last_unstale":"2023-11-18T20:43:33.804790+0000","last_undegraded":"2023-11-18T20:43:33.804790+0000","last_fullsized":"2023-11-18T20:43:33.804790+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:12:53.159267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.803544+0000","last_change":"2023-11-18T20:43:33.802355+0000","last_active":"2023-11-18T20:43:33.803544+0000","last_peered":"2023-11-18T20:43:33.803544+0000","last_clean":"2023-11-18T20:43:33.803544+0000","last_became_active":"2023-11-18T20:43:33.801934+0000","last_became_peered":"2023-11-18T20:43:33.801934+0000","last_unstale":"2023-11-18T20:43:33.803544+0000","last_undegraded":"2023-11-18T20:43:33.803544+0000","last_fullsized":"2023-11-18T20:43:33.803544+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:40:23.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807355+0000","last_change":"2023-11-18T20:43:33.800584+0000","last_active":"2023-11-18T20:43:33.807355+0000","last_peered":"2023-11-18T20:43:33.807355+0000","last_clean":"2023-11-18T20:43:33.807355+0000","last_became_active":"2023-11-18T20:43:33.800326+0000","last_became_peered":"2023-11-18T20:43:33.800326+0000","last_unstale":"2023-11-18T20:43:33.807355+0000","last_undegraded":"2023-11-18T20:43:33.807355+0000","last_fullsized":"2023-11-18T20:43:33.807355+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:34:56.639700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.801083+0000","last_change":"2023-11-18T20:43:33.798327+0000","last_active":"2023-11-18T20:43:33.801083+0000","last_peered":"2023-11-18T20:43:33.801083+0000","last_clean":"2023-11-18T20:43:33.801083+0000","last_became_active":"2023-11-18T20:43:33.790560+0000","last_became_peered":"2023-11-18T20:43:33.790560+0000","last_unstale":"2023-11-18T20:43:33.801083+0000","last_undegraded":"2023-11-18T20:43:33.801083+0000","last_fullsized":"2023-11-18T20:43:33.801083+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:34.983437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.795043+0000","last_change":"2023-11-18T20:43:33.791163+0000","last_active":"2023-11-18T20:43:33.795043+0000","last_peered":"2023-11-18T20:43:33.795043+0000","last_clean":"2023-11-18T20:43:33.795043+0000","last_became_active":"2023-11-18T20:43:33.790799+0000","last_became_peered":"2023-11-18T20:43:33.790799+0000","last_unstale":"2023-11-18T20:43:33.795043+0000","last_undegraded":"2023-11-18T20:43:33.795043+0000","last_fullsized":"2023-11-18T20:43:33.795043+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:53:10.088474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.801008+0000","last_change":"2023-11-18T20:43:33.798089+0000","last_active":"2023-11-18T20:43:33.801008+0000","last_peered":"2023-11-18T20:43:33.801008+0000","last_clean":"2023-11-18T20:43:33.801008+0000","last_became_active":"2023-11-18T20:43:33.797769+0000","last_became_peered":"2023-11-18T20:43:33.797769+0000","last_unstale":"2023-11-18T20:43:33.801008+0000","last_undegraded":"2023-11-18T20:43:33.801008+0000","last_fullsized":"2023-11-18T20:43:33.801008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:13.981406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.799975+0000","last_change":"2023-11-18T20:43:33.795891+0000","last_active":"2023-11-18T20:43:33.799975+0000","last_peered":"2023-11-18T20:43:33.799975+0000","last_clean":"2023-11-18T20:43:33.799975+0000","last_became_active":"2023-11-18T20:43:33.795604+0000","last_became_peered":"2023-11-18T20:43:33.795604+0000","last_unstale":"2023-11-18T20:43:33.799975+0000","last_undegraded":"2023-11-18T20:43:33.799975+0000","last_fullsized":"2023-11-18T20:43:33.799975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:58:12.405990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.799935+0000","last_change":"2023-11-18T20:43:33.796971+0000","last_active":"2023-11-18T20:43:33.799935+0000","last_peered":"2023-11-18T20:43:33.799935+0000","last_clean":"2023-11-18T20:43:33.799935+0000","last_became_active":"2023-11-18T20:43:33.796667+0000","last_became_peered":"2023-11-18T20:43:33.796667+0000","last_unstale":"2023-11-18T20:43:33.799935+0000","last_undegraded":"2023-11-18T20:43:33.799935+0000","last_fullsized":"2023-11-18T20:43:33.799935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:34:56.639700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798604+0000","last_change":"2023-11-18T20:43:33.792060+0000","last_active":"2023-11-18T20:43:33.798604+0000","last_peered":"2023-11-18T20:43:33.798604+0000","last_clean":"2023-11-18T20:43:33.798604+0000","last_became_active":"2023-11-18T20:43:33.791790+0000","last_became_peered":"2023-11-18T20:43:33.791790+0000","last_unstale":"2023-11-18T20:43:33.798604+0000","last_undegraded":"2023-11-18T20:43:33.798604+0000","last_fullsized":"2023-11-18T20:43:33.798604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:47:54.978135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.801198+0000","last_change":"2023-11-18T20:43:33.800118+0000","last_active":"2023-11-18T20:43:33.801198+0000","last_peered":"2023-11-18T20:43:33.801198+0000","last_clean":"2023-11-18T20:43:33.801198+0000","last_became_active":"2023-11-18T20:43:33.799816+0000","last_became_peered":"2023-11-18T20:43:33.799816+0000","last_unstale":"2023-11-18T20:43:33.801198+0000","last_undegraded":"2023-11-18T20:43:33.801198+0000","last_fullsized":"2023-11-18T20:43:33.801198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:18.711038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807413+0000","last_change":"2023-11-18T20:43:33.801605+0000","last_active":"2023-11-18T20:43:33.807413+0000","last_peered":"2023-11-18T20:43:33.807413+0000","last_clean":"2023-11-18T20:43:33.807413+0000","last_became_active":"2023-11-18T20:43:33.801337+0000","last_became_peered":"2023-11-18T20:43:33.801337+0000","last_unstale":"2023-11-18T20:43:33.807413+0000","last_undegraded":"2023-11-18T20:43:33.807413+0000","last_fullsized":"2023-11-18T20:43:33.807413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:13.981406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798521+0000","last_change":"2023-11-18T20:43:33.795597+0000","last_active":"2023-11-18T20:43:33.798521+0000","last_peered":"2023-11-18T20:43:33.798521+0000","last_clean":"2023-11-18T20:43:33.798521+0000","last_became_active":"2023-11-18T20:43:33.795334+0000","last_became_peered":"2023-11-18T20:43:33.795334+0000","last_unstale":"2023-11-18T20:43:33.798521+0000","last_undegraded":"2023-11-18T20:43:33.798521+0000","last_fullsized":"2023-11-18T20:43:33.798521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:56:23.474195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.786675+0000","last_change":"2023-11-18T20:43:33.786019+0000","last_active":"2023-11-18T20:43:33.786675+0000","last_peered":"2023-11-18T20:43:33.786675+0000","last_clean":"2023-11-18T20:43:33.786675+0000","last_became_active":"2023-11-18T20:43:33.785684+0000","last_became_peered":"2023-11-18T20:43:33.785684+0000","last_unstale":"2023-11-18T20:43:33.786675+0000","last_undegraded":"2023-11-18T20:43:33.786675+0000","last_fullsized":"2023-11-18T20:43:33.786675+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:05:56.809560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798812+0000","last_change":"2023-11-18T20:43:33.795983+0000","last_active":"2023-11-18T20:43:33.798812+0000","last_peered":"2023-11-18T20:43:33.798812+0000","last_clean":"2023-11-18T20:43:33.798812+0000","last_became_active":"2023-11-18T20:43:33.795689+0000","last_became_peered":"2023-11-18T20:43:33.795689+0000","last_unstale":"2023-11-18T20:43:33.798812+0000","last_undegraded":"2023-11-18T20:43:33.798812+0000","last_fullsized":"2023-11-18T20:43:33.798812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:04:51.358886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.804774+0000","last_change":"2023-11-18T20:43:33.798893+0000","last_active":"2023-11-18T20:43:33.804774+0000","last_peered":"2023-11-18T20:43:33.804774+0000","last_clean":"2023-11-18T20:43:33.804774+0000","last_became_active":"2023-11-18T20:43:33.798516+0000","last_became_peered":"2023-11-18T20:43:33.798516+0000","last_unstale":"2023-11-18T20:43:33.804774+0000","last_undegraded":"2023-11-18T20:43:33.804774+0000","last_fullsized":"2023-11-18T20:43:33.804774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:32:05.273998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.804621+0000","last_change":"2023-11-18T20:43:33.799828+0000","last_active":"2023-11-18T20:43:33.804621+0000","last_peered":"2023-11-18T20:43:33.804621+0000","last_clean":"2023-11-18T20:43:33.804621+0000","last_became_active":"2023-11-18T20:43:33.799564+0000","last_became_peered":"2023-11-18T20:43:33.799564+0000","last_unstale":"2023-11-18T20:43:33.804621+0000","last_undegraded":"2023-11-18T20:43:33.804621+0000","last_fullsized":"2023-11-18T20:43:33.804621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:40:23.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800028+0000","last_change":"2023-11-18T20:43:33.795938+0000","last_active":"2023-11-18T20:43:33.800028+0000","last_peered":"2023-11-18T20:43:33.800028+0000","last_clean":"2023-11-18T20:43:33.800028+0000","last_became_active":"2023-11-18T20:43:33.795569+0000","last_became_peered":"2023-11-18T20:43:33.795569+0000","last_unstale":"2023-11-18T20:43:33.800028+0000","last_undegraded":"2023-11-18T20:43:33.800028+0000","last_fullsized":"2023-11-18T20:43:33.800028+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:56:23.474195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.804905+0000","last_change":"2023-11-18T20:43:33.804285+0000","last_active":"2023-11-18T20:43:33.804905+0000","last_peered":"2023-11-18T20:43:33.804905+0000","last_clean":"2023-11-18T20:43:33.804905+0000","last_became_active":"2023-11-18T20:43:33.803960+0000","last_became_peered":"2023-11-18T20:43:33.803960+0000","last_unstale":"2023-11-18T20:43:33.804905+0000","last_undegraded":"2023-11-18T20:43:33.804905+0000","last_fullsized":"2023-11-18T20:43:33.804905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:42.882485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800963+0000","last_change":"2023-11-18T20:43:33.798064+0000","last_active":"2023-11-18T20:43:33.800963+0000","last_peered":"2023-11-18T20:43:33.800963+0000","last_clean":"2023-11-18T20:43:33.800963+0000","last_became_active":"2023-11-18T20:43:33.797789+0000","last_became_peered":"2023-11-18T20:43:33.797789+0000","last_unstale":"2023-11-18T20:43:33.800963+0000","last_undegraded":"2023-11-18T20:43:33.800963+0000","last_fullsized":"2023-11-18T20:43:33.800963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:32.362994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.796757+0000","last_change":"2023-11-18T20:43:33.796121+0000","last_active":"2023-11-18T20:43:33.796757+0000","last_peered":"2023-11-18T20:43:33.796757+0000","last_clean":"2023-11-18T20:43:33.796757+0000","last_became_active":"2023-11-18T20:43:33.795860+0000","last_became_peered":"2023-11-18T20:43:33.795860+0000","last_unstale":"2023-11-18T20:43:33.796757+0000","last_undegraded":"2023-11-18T20:43:33.796757+0000","last_fullsized":"2023-11-18T20:43:33.796757+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:49:45.680897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.792586+0000","last_change":"2023-11-18T20:43:33.791329+0000","last_active":"2023-11-18T20:43:33.792586+0000","last_peered":"2023-11-18T20:43:33.792586+0000","last_clean":"2023-11-18T20:43:33.792586+0000","last_became_active":"2023-11-18T20:43:33.790929+0000","last_became_peered":"2023-11-18T20:43:33.790929+0000","last_unstale":"2023-11-18T20:43:33.792586+0000","last_undegraded":"2023-11-18T20:43:33.792586+0000","last_fullsized":"2023-11-18T20:43:33.792586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:11:31.306866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807575+0000","last_change":"2023-11-18T20:43:33.800433+0000","last_active":"2023-11-18T20:43:33.807575+0000","last_peered":"2023-11-18T20:43:33.807575+0000","last_clean":"2023-11-18T20:43:33.807575+0000","last_became_active":"2023-11-18T20:43:33.800148+0000","last_became_peered":"2023-11-18T20:43:33.800148+0000","last_unstale":"2023-11-18T20:43:33.807575+0000","last_undegraded":"2023-11-18T20:43:33.807575+0000","last_fullsized":"2023-11-18T20:43:33.807575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:26:40.774256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.797702+0000","last_change":"2023-11-18T20:43:33.794332+0000","last_active":"2023-11-18T20:43:33.797702+0000","last_peered":"2023-11-18T20:43:33.797702+0000","last_clean":"2023-11-18T20:43:33.797702+0000","last_became_active":"2023-11-18T20:43:33.794022+0000","last_became_peered":"2023-11-18T20:43:33.794022+0000","last_unstale":"2023-11-18T20:43:33.797702+0000","last_undegraded":"2023-11-18T20:43:33.797702+0000","last_fullsized":"2023-11-18T20:43:33.797702+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:32:55.637084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.794282+0000","last_change":"2023-11-18T20:43:33.791594+0000","last_active":"2023-11-18T20:43:33.794282+0000","last_peered":"2023-11-18T20:43:33.794282+0000","last_clean":"2023-11-18T20:43:33.794282+0000","last_became_active":"2023-11-18T20:43:33.791279+0000","last_became_peered":"2023-11-18T20:43:33.791279+0000","last_unstale":"2023-11-18T20:43:33.794282+0000","last_undegraded":"2023-11-18T20:43:33.794282+0000","last_fullsized":"2023-11-18T20:43:33.794282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:18.711038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800342+0000","last_change":"2023-11-18T20:43:33.796577+0000","last_active":"2023-11-18T20:43:33.800342+0000","last_peered":"2023-11-18T20:43:33.800342+0000","last_clean":"2023-11-18T20:43:33.800342+0000","last_became_active":"2023-11-18T20:43:33.796312+0000","last_became_peered":"2023-11-18T20:43:33.796312+0000","last_unstale":"2023-11-18T20:43:33.800342+0000","last_undegraded":"2023-11-18T20:43:33.800342+0000","last_fullsized":"2023-11-18T20:43:33.800342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:32.362994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.797713+0000","last_change":"2023-11-18T20:43:33.794492+0000","last_active":"2023-11-18T20:43:33.797713+0000","last_peered":"2023-11-18T20:43:33.797713+0000","last_clean":"2023-11-18T20:43:33.797713+0000","last_became_active":"2023-11-18T20:43:33.794213+0000","last_became_peered":"2023-11-18T20:43:33.794213+0000","last_unstale":"2023-11-18T20:43:33.797713+0000","last_undegraded":"2023-11-18T20:43:33.797713+0000","last_fullsized":"2023-11-18T20:43:33.797713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:56:23.474195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.792632+0000","last_change":"2023-11-18T20:43:33.790591+0000","last_active":"2023-11-18T20:43:33.792632+0000","last_peered":"2023-11-18T20:43:33.792632+0000","last_clean":"2023-11-18T20:43:33.792632+0000","last_became_active":"2023-11-18T20:43:33.790246+0000","last_became_peered":"2023-11-18T20:43:33.790246+0000","last_unstale":"2023-11-18T20:43:33.792632+0000","last_undegraded":"2023-11-18T20:43:33.792632+0000","last_fullsized":"2023-11-18T20:43:33.792632+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:32:05.273998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798501+0000","last_change":"2023-11-18T20:43:33.794844+0000","last_active":"2023-11-18T20:43:33.798501+0000","last_peered":"2023-11-18T20:43:33.798501+0000","last_clean":"2023-11-18T20:43:33.798501+0000","last_became_active":"2023-11-18T20:43:33.794564+0000","last_became_peered":"2023-11-18T20:43:33.794564+0000","last_unstale":"2023-11-18T20:43:33.798501+0000","last_undegraded":"2023-11-18T20:43:33.798501+0000","last_fullsized":"2023-11-18T20:43:33.798501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:37.759359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807616+0000","last_change":"2023-11-18T20:43:33.800946+0000","last_active":"2023-11-18T20:43:33.807616+0000","last_peered":"2023-11-18T20:43:33.807616+0000","last_clean":"2023-11-18T20:43:33.807616+0000","last_became_active":"2023-11-18T20:43:33.800397+0000","last_became_peered":"2023-11-18T20:43:33.800397+0000","last_unstale":"2023-11-18T20:43:33.807616+0000","last_undegraded":"2023-11-18T20:43:33.807616+0000","last_fullsized":"2023-11-18T20:43:33.807616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:56:18.827236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.799239+0000","last_change":"2023-11-18T20:43:33.795175+0000","last_active":"2023-11-18T20:43:33.799239+0000","last_peered":"2023-11-18T20:43:33.799239+0000","last_clean":"2023-11-18T20:43:33.799239+0000","last_became_active":"2023-11-18T20:43:33.794845+0000","last_became_peered":"2023-11-18T20:43:33.794845+0000","last_unstale":"2023-11-18T20:43:33.799239+0000","last_undegraded":"2023-11-18T20:43:33.799239+0000","last_fullsized":"2023-11-18T20:43:33.799239+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:32.362994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800676+0000","last_change":"2023-11-18T20:43:33.797636+0000","last_active":"2023-11-18T20:43:33.800676+0000","last_peered":"2023-11-18T20:43:33.800676+0000","last_clean":"2023-11-18T20:43:33.800676+0000","last_became_active":"2023-11-18T20:43:33.797311+0000","last_became_peered":"2023-11-18T20:43:33.797311+0000","last_unstale":"2023-11-18T20:43:33.800676+0000","last_undegraded":"2023-11-18T20:43:33.800676+0000","last_fullsized":"2023-11-18T20:43:33.800676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:32.362994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.813018+0000","last_change":"2023-11-18T20:43:33.806648+0000","last_active":"2023-11-18T20:43:33.813018+0000","last_peered":"2023-11-18T20:43:33.813018+0000","last_clean":"2023-11-18T20:43:33.813018+0000","last_became_active":"2023-11-18T20:43:33.806113+0000","last_became_peered":"2023-11-18T20:43:33.806113+0000","last_unstale":"2023-11-18T20:43:33.813018+0000","last_undegraded":"2023-11-18T20:43:33.813018+0000","last_fullsized":"2023-11-18T20:43:33.813018+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:04:51.358886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.801663+0000","last_change":"2023-11-18T20:43:33.800284+0000","last_active":"2023-11-18T20:43:33.801663+0000","last_peered":"2023-11-18T20:43:33.801663+0000","last_clean":"2023-11-18T20:43:33.801663+0000","last_became_active":"2023-11-18T20:43:33.800020+0000","last_became_peered":"2023-11-18T20:43:33.800020+0000","last_unstale":"2023-11-18T20:43:33.801663+0000","last_undegraded":"2023-11-18T20:43:33.801663+0000","last_fullsized":"2023-11-18T20:43:33.801663+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:34.983437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.115774+0000","last_change":"2023-11-18T20:43:32.789148+0000","last_active":"2023-11-18T20:43:34.115774+0000","last_peered":"2023-11-18T20:43:34.115774+0000","last_clean":"2023-11-18T20:43:34.115774+0000","last_became_active":"2023-11-18T20:43:32.788721+0000","last_became_peered":"2023-11-18T20:43:32.788721+0000","last_unstale":"2023-11-18T20:43:34.115774+0000","last_undegraded":"2023-11-18T20:43:34.115774+0000","last_fullsized":"2023-11-18T20:43:34.115774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:33.979328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.812956+0000","last_change":"2023-11-18T20:43:33.805898+0000","last_active":"2023-11-18T20:43:33.812956+0000","last_peered":"2023-11-18T20:43:33.812956+0000","last_clean":"2023-11-18T20:43:33.812956+0000","last_became_active":"2023-11-18T20:43:33.805576+0000","last_became_peered":"2023-11-18T20:43:33.805576+0000","last_unstale":"2023-11-18T20:43:33.812956+0000","last_undegraded":"2023-11-18T20:43:33.812956+0000","last_fullsized":"2023-11-18T20:43:33.812956+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:40:08.752775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.113073+0000","last_change":"2023-11-18T20:43:32.786552+0000","last_active":"2023-11-18T20:43:34.113073+0000","last_peered":"2023-11-18T20:43:34.113073+0000","last_clean":"2023-11-18T20:43:34.113073+0000","last_became_active":"2023-11-18T20:43:32.786271+0000","last_became_peered":"2023-11-18T20:43:32.786271+0000","last_unstale":"2023-11-18T20:43:34.113073+0000","last_undegraded":"2023-11-18T20:43:34.113073+0000","last_fullsized":"2023-11-18T20:43:34.113073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:17.706929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807686+0000","last_change":"2023-11-18T20:43:33.799693+0000","last_active":"2023-11-18T20:43:33.807686+0000","last_peered":"2023-11-18T20:43:33.807686+0000","last_clean":"2023-11-18T20:43:33.807686+0000","last_became_active":"2023-11-18T20:43:33.799375+0000","last_became_peered":"2023-11-18T20:43:33.799375+0000","last_unstale":"2023-11-18T20:43:33.807686+0000","last_undegraded":"2023-11-18T20:43:33.807686+0000","last_fullsized":"2023-11-18T20:43:33.807686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:35:36.747539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.112923+0000","last_change":"2023-11-18T20:43:32.786101+0000","last_active":"2023-11-18T20:43:34.112923+0000","last_peered":"2023-11-18T20:43:34.112923+0000","last_clean":"2023-11-18T20:43:34.112923+0000","last_became_active":"2023-11-18T20:43:32.785754+0000","last_became_peered":"2023-11-18T20:43:32.785754+0000","last_unstale":"2023-11-18T20:43:34.112923+0000","last_undegraded":"2023-11-18T20:43:34.112923+0000","last_fullsized":"2023-11-18T20:43:34.112923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:36:53.103501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.790778+0000","last_change":"2023-11-18T20:43:33.788363+0000","last_active":"2023-11-18T20:43:33.790778+0000","last_peered":"2023-11-18T20:43:33.790778+0000","last_clean":"2023-11-18T20:43:33.790778+0000","last_became_active":"2023-11-18T20:43:33.788104+0000","last_became_peered":"2023-11-18T20:43:33.788104+0000","last_unstale":"2023-11-18T20:43:33.790778+0000","last_undegraded":"2023-11-18T20:43:33.790778+0000","last_fullsized":"2023-11-18T20:43:33.790778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:12:53.159267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.111039+0000","last_change":"2023-11-18T20:43:32.787841+0000","last_active":"2023-11-18T20:43:34.111039+0000","last_peered":"2023-11-18T20:43:34.111039+0000","last_clean":"2023-11-18T20:43:34.111039+0000","last_became_active":"2023-11-18T20:43:32.787545+0000","last_became_peered":"2023-11-18T20:43:32.787545+0000","last_unstale":"2023-11-18T20:43:34.111039+0000","last_undegraded":"2023-11-18T20:43:34.111039+0000","last_fullsized":"2023-11-18T20:43:34.111039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:18:42.906284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.813002+0000","last_change":"2023-11-18T20:43:33.805980+0000","last_active":"2023-11-18T20:43:33.813002+0000","last_peered":"2023-11-18T20:43:33.813002+0000","last_clean":"2023-11-18T20:43:33.813002+0000","last_became_active":"2023-11-18T20:43:33.805671+0000","last_became_peered":"2023-11-18T20:43:33.805671+0000","last_unstale":"2023-11-18T20:43:33.813002+0000","last_undegraded":"2023-11-18T20:43:33.813002+0000","last_fullsized":"2023-11-18T20:43:33.813002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:37.759359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.115729+0000","last_change":"2023-11-18T20:43:32.790506+0000","last_active":"2023-11-18T20:43:34.115729+0000","last_peered":"2023-11-18T20:43:34.115729+0000","last_clean":"2023-11-18T20:43:34.115729+0000","last_became_active":"2023-11-18T20:43:32.790228+0000","last_became_peered":"2023-11-18T20:43:32.790228+0000","last_unstale":"2023-11-18T20:43:34.115729+0000","last_undegraded":"2023-11-18T20:43:34.115729+0000","last_fullsized":"2023-11-18T20:43:34.115729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:12.977297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807486+0000","last_change":"2023-11-18T20:43:33.801617+0000","last_active":"2023-11-18T20:43:33.807486+0000","last_peered":"2023-11-18T20:43:33.807486+0000","last_clean":"2023-11-18T20:43:33.807486+0000","last_became_active":"2023-11-18T20:43:33.801286+0000","last_became_peered":"2023-11-18T20:43:33.801286+0000","last_unstale":"2023-11-18T20:43:33.807486+0000","last_undegraded":"2023-11-18T20:43:33.807486+0000","last_fullsized":"2023-11-18T20:43:33.807486+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:05:54.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.112861+0000","last_change":"2023-11-18T20:43:32.789438+0000","last_active":"2023-11-18T20:43:34.112861+0000","last_peered":"2023-11-18T20:43:34.112861+0000","last_clean":"2023-11-18T20:43:34.112861+0000","last_became_active":"2023-11-18T20:43:32.788839+0000","last_became_peered":"2023-11-18T20:43:32.788839+0000","last_unstale":"2023-11-18T20:43:34.112861+0000","last_undegraded":"2023-11-18T20:43:34.112861+0000","last_fullsized":"2023-11-18T20:43:34.112861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:52:33.979328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807550+0000","last_change":"2023-11-18T20:43:33.800718+0000","last_active":"2023-11-18T20:43:33.807550+0000","last_peered":"2023-11-18T20:43:33.807550+0000","last_clean":"2023-11-18T20:43:33.807550+0000","last_became_active":"2023-11-18T20:43:33.798535+0000","last_became_peered":"2023-11-18T20:43:33.798535+0000","last_unstale":"2023-11-18T20:43:33.807550+0000","last_undegraded":"2023-11-18T20:43:33.807550+0000","last_fullsized":"2023-11-18T20:43:33.807550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:19:33.538932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.791205+0000","last_change":"2023-11-18T20:43:32.784541+0000","last_active":"2023-11-18T20:43:33.791205+0000","last_peered":"2023-11-18T20:43:33.791205+0000","last_clean":"2023-11-18T20:43:33.791205+0000","last_became_active":"2023-11-18T20:43:32.784259+0000","last_became_peered":"2023-11-18T20:43:32.784259+0000","last_unstale":"2023-11-18T20:43:33.791205+0000","last_undegraded":"2023-11-18T20:43:33.791205+0000","last_fullsized":"2023-11-18T20:43:33.791205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:58:11.401881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800364+0000","last_change":"2023-11-18T20:43:33.795628+0000","last_active":"2023-11-18T20:43:33.800364+0000","last_peered":"2023-11-18T20:43:33.800364+0000","last_clean":"2023-11-18T20:43:33.800364+0000","last_became_active":"2023-11-18T20:43:33.795259+0000","last_became_peered":"2023-11-18T20:43:33.795259+0000","last_unstale":"2023-11-18T20:43:33.800364+0000","last_undegraded":"2023-11-18T20:43:33.800364+0000","last_fullsized":"2023-11-18T20:43:33.800364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:32:55.637084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.783620+0000","last_change":"2023-11-18T20:43:32.791435+0000","last_active":"2023-11-18T20:43:33.783620+0000","last_peered":"2023-11-18T20:43:33.783620+0000","last_clean":"2023-11-18T20:43:33.783620+0000","last_became_active":"2023-11-18T20:43:32.791075+0000","last_became_peered":"2023-11-18T20:43:32.791075+0000","last_unstale":"2023-11-18T20:43:33.783620+0000","last_undegraded":"2023-11-18T20:43:33.783620+0000","last_fullsized":"2023-11-18T20:43:33.783620+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:40:07.748665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800693+0000","last_change":"2023-11-18T20:43:33.796504+0000","last_active":"2023-11-18T20:43:33.800693+0000","last_peered":"2023-11-18T20:43:33.800693+0000","last_clean":"2023-11-18T20:43:33.800693+0000","last_became_active":"2023-11-18T20:43:33.796239+0000","last_became_peered":"2023-11-18T20:43:33.796239+0000","last_unstale":"2023-11-18T20:43:33.800693+0000","last_undegraded":"2023-11-18T20:43:33.800693+0000","last_fullsized":"2023-11-18T20:43:33.800693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:32.362994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.791735+0000","last_change":"2023-11-18T20:43:32.787724+0000","last_active":"2023-11-18T20:43:33.791735+0000","last_peered":"2023-11-18T20:43:33.791735+0000","last_clean":"2023-11-18T20:43:33.791735+0000","last_became_active":"2023-11-18T20:43:32.787451+0000","last_became_peered":"2023-11-18T20:43:32.787451+0000","last_unstale":"2023-11-18T20:43:33.791735+0000","last_undegraded":"2023-11-18T20:43:33.791735+0000","last_fullsized":"2023-11-18T20:43:33.791735+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:04:35.722671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.794868+0000","last_change":"2023-11-18T20:43:33.793661+0000","last_active":"2023-11-18T20:43:33.794868+0000","last_peered":"2023-11-18T20:43:33.794868+0000","last_clean":"2023-11-18T20:43:33.794868+0000","last_became_active":"2023-11-18T20:43:33.793410+0000","last_became_peered":"2023-11-18T20:43:33.793410+0000","last_unstale":"2023-11-18T20:43:33.794868+0000","last_undegraded":"2023-11-18T20:43:33.794868+0000","last_fullsized":"2023-11-18T20:43:33.794868+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:37.759359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.786256+0000","last_change":"2023-11-18T20:43:32.789378+0000","last_active":"2023-11-18T20:43:33.786256+0000","last_peered":"2023-11-18T20:43:33.786256+0000","last_clean":"2023-11-18T20:43:33.786256+0000","last_became_active":"2023-11-18T20:43:32.788879+0000","last_became_peered":"2023-11-18T20:43:32.788879+0000","last_unstale":"2023-11-18T20:43:33.786256+0000","last_undegraded":"2023-11-18T20:43:33.786256+0000","last_fullsized":"2023-11-18T20:43:33.786256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:31.358885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798794+0000","last_change":"2023-11-18T20:43:33.795955+0000","last_active":"2023-11-18T20:43:33.798794+0000","last_peered":"2023-11-18T20:43:33.798794+0000","last_clean":"2023-11-18T20:43:33.798794+0000","last_became_active":"2023-11-18T20:43:33.795679+0000","last_became_peered":"2023-11-18T20:43:33.795679+0000","last_unstale":"2023-11-18T20:43:33.798794+0000","last_undegraded":"2023-11-18T20:43:33.798794+0000","last_fullsized":"2023-11-18T20:43:33.798794+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:32:05.273998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.113092+0000","last_change":"2023-11-18T20:43:32.792377+0000","last_active":"2023-11-18T20:43:34.113092+0000","last_peered":"2023-11-18T20:43:34.113092+0000","last_clean":"2023-11-18T20:43:34.113092+0000","last_became_active":"2023-11-18T20:43:32.792087+0000","last_became_peered":"2023-11-18T20:43:32.792087+0000","last_unstale":"2023-11-18T20:43:34.113092+0000","last_undegraded":"2023-11-18T20:43:34.113092+0000","last_fullsized":"2023-11-18T20:43:34.113092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:40:22.845618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807589+0000","last_change":"2023-11-18T20:43:33.802336+0000","last_active":"2023-11-18T20:43:33.807589+0000","last_peered":"2023-11-18T20:43:33.807589+0000","last_clean":"2023-11-18T20:43:33.807589+0000","last_became_active":"2023-11-18T20:43:33.802058+0000","last_became_peered":"2023-11-18T20:43:33.802058+0000","last_unstale":"2023-11-18T20:43:33.807589+0000","last_undegraded":"2023-11-18T20:43:33.807589+0000","last_fullsized":"2023-11-18T20:43:33.807589+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:04:51.358886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.117642+0000","last_change":"2023-11-18T20:43:32.788567+0000","last_active":"2023-11-18T20:43:34.117642+0000","last_peered":"2023-11-18T20:43:34.117642+0000","last_clean":"2023-11-18T20:43:34.117642+0000","last_became_active":"2023-11-18T20:43:32.788211+0000","last_became_peered":"2023-11-18T20:43:32.788211+0000","last_unstale":"2023-11-18T20:43:34.117642+0000","last_undegraded":"2023-11-18T20:43:34.117642+0000","last_fullsized":"2023-11-18T20:43:34.117642+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:15:30.342072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800333+0000","last_change":"2023-11-18T20:43:33.795595+0000","last_active":"2023-11-18T20:43:33.800333+0000","last_peered":"2023-11-18T20:43:33.800333+0000","last_clean":"2023-11-18T20:43:33.800333+0000","last_became_active":"2023-11-18T20:43:33.795177+0000","last_became_peered":"2023-11-18T20:43:33.795177+0000","last_unstale":"2023-11-18T20:43:33.800333+0000","last_undegraded":"2023-11-18T20:43:33.800333+0000","last_fullsized":"2023-11-18T20:43:33.800333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:37.759359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.116057+0000","last_change":"2023-11-18T20:43:32.791294+0000","last_active":"2023-11-18T20:43:34.116057+0000","last_peered":"2023-11-18T20:43:34.116057+0000","last_clean":"2023-11-18T20:43:34.116057+0000","last_became_active":"2023-11-18T20:43:32.790980+0000","last_became_peered":"2023-11-18T20:43:32.790980+0000","last_unstale":"2023-11-18T20:43:34.116057+0000","last_undegraded":"2023-11-18T20:43:34.116057+0000","last_fullsized":"2023-11-18T20:43:34.116057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:36.755249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.801010+0000","last_change":"2023-11-18T20:43:33.798954+0000","last_active":"2023-11-18T20:43:33.801010+0000","last_peered":"2023-11-18T20:43:33.801010+0000","last_clean":"2023-11-18T20:43:33.801010+0000","last_became_active":"2023-11-18T20:43:33.798420+0000","last_became_peered":"2023-11-18T20:43:33.798420+0000","last_unstale":"2023-11-18T20:43:33.801010+0000","last_undegraded":"2023-11-18T20:43:33.801010+0000","last_fullsized":"2023-11-18T20:43:33.801010+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:34:56.639700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.784526+0000","last_change":"2023-11-18T20:43:32.789906+0000","last_active":"2023-11-18T20:43:33.784526+0000","last_peered":"2023-11-18T20:43:33.784526+0000","last_clean":"2023-11-18T20:43:33.784526+0000","last_became_active":"2023-11-18T20:43:32.789582+0000","last_became_peered":"2023-11-18T20:43:32.789582+0000","last_unstale":"2023-11-18T20:43:33.784526+0000","last_undegraded":"2023-11-18T20:43:33.784526+0000","last_fullsized":"2023-11-18T20:43:33.784526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:56:22.470085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.812996+0000","last_change":"2023-11-18T20:43:33.805882+0000","last_active":"2023-11-18T20:43:33.812996+0000","last_peered":"2023-11-18T20:43:33.812996+0000","last_clean":"2023-11-18T20:43:33.812996+0000","last_became_active":"2023-11-18T20:43:33.805599+0000","last_became_peered":"2023-11-18T20:43:33.805599+0000","last_unstale":"2023-11-18T20:43:33.812996+0000","last_undegraded":"2023-11-18T20:43:33.812996+0000","last_fullsized":"2023-11-18T20:43:33.812996+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:42.882485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.791358+0000","last_change":"2023-11-18T20:43:32.783841+0000","last_active":"2023-11-18T20:43:33.791358+0000","last_peered":"2023-11-18T20:43:33.791358+0000","last_clean":"2023-11-18T20:43:33.791358+0000","last_became_active":"2023-11-18T20:43:32.783556+0000","last_became_peered":"2023-11-18T20:43:32.783556+0000","last_unstale":"2023-11-18T20:43:33.791358+0000","last_undegraded":"2023-11-18T20:43:33.791358+0000","last_fullsized":"2023-11-18T20:43:33.791358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:42:11.680696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807555+0000","last_change":"2023-11-18T20:43:33.805482+0000","last_active":"2023-11-18T20:43:33.807555+0000","last_peered":"2023-11-18T20:43:33.807555+0000","last_clean":"2023-11-18T20:43:33.807555+0000","last_became_active":"2023-11-18T20:43:33.805179+0000","last_became_peered":"2023-11-18T20:43:33.805179+0000","last_unstale":"2023-11-18T20:43:33.807555+0000","last_undegraded":"2023-11-18T20:43:33.807555+0000","last_fullsized":"2023-11-18T20:43:33.807555+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:32:55.637084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.113384+0000","last_change":"2023-11-18T20:43:32.788989+0000","last_active":"2023-11-18T20:43:34.113384+0000","last_peered":"2023-11-18T20:43:34.113384+0000","last_clean":"2023-11-18T20:43:34.113384+0000","last_became_active":"2023-11-18T20:43:32.788717+0000","last_became_peered":"2023-11-18T20:43:32.788717+0000","last_unstale":"2023-11-18T20:43:34.113384+0000","last_undegraded":"2023-11-18T20:43:34.113384+0000","last_fullsized":"2023-11-18T20:43:34.113384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:05:53.021163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.804929+0000","last_change":"2023-11-18T20:43:33.796585+0000","last_active":"2023-11-18T20:43:33.804929+0000","last_peered":"2023-11-18T20:43:33.804929+0000","last_clean":"2023-11-18T20:43:33.804929+0000","last_became_active":"2023-11-18T20:43:33.796276+0000","last_became_peered":"2023-11-18T20:43:33.796276+0000","last_unstale":"2023-11-18T20:43:33.804929+0000","last_undegraded":"2023-11-18T20:43:33.804929+0000","last_fullsized":"2023-11-18T20:43:33.804929+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:59:35.675350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.111976+0000","last_change":"2023-11-18T20:43:32.789162+0000","last_active":"2023-11-18T20:43:34.111976+0000","last_peered":"2023-11-18T20:43:34.111976+0000","last_clean":"2023-11-18T20:43:34.111976+0000","last_became_active":"2023-11-18T20:43:32.788613+0000","last_became_peered":"2023-11-18T20:43:32.788613+0000","last_unstale":"2023-11-18T20:43:34.111976+0000","last_undegraded":"2023-11-18T20:43:34.111976+0000","last_fullsized":"2023-11-18T20:43:34.111976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:04:50.354776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807823+0000","last_change":"2023-11-18T20:43:33.805816+0000","last_active":"2023-11-18T20:43:33.807823+0000","last_peered":"2023-11-18T20:43:33.807823+0000","last_clean":"2023-11-18T20:43:33.807823+0000","last_became_active":"2023-11-18T20:43:33.805545+0000","last_became_peered":"2023-11-18T20:43:33.805545+0000","last_unstale":"2023-11-18T20:43:33.807823+0000","last_undegraded":"2023-11-18T20:43:33.807823+0000","last_fullsized":"2023-11-18T20:43:33.807823+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:58:12.405990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.115965+0000","last_change":"2023-11-18T20:43:32.792303+0000","last_active":"2023-11-18T20:43:34.115965+0000","last_peered":"2023-11-18T20:43:34.115965+0000","last_clean":"2023-11-18T20:43:34.115965+0000","last_became_active":"2023-11-18T20:43:32.790966+0000","last_became_peered":"2023-11-18T20:43:32.790966+0000","last_unstale":"2023-11-18T20:43:34.115965+0000","last_undegraded":"2023-11-18T20:43:34.115965+0000","last_fullsized":"2023-11-18T20:43:34.115965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:32:54.632975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.794782+0000","last_change":"2023-11-18T20:43:33.787059+0000","last_active":"2023-11-18T20:43:33.794782+0000","last_peered":"2023-11-18T20:43:33.794782+0000","last_clean":"2023-11-18T20:43:33.794782+0000","last_became_active":"2023-11-18T20:43:33.786697+0000","last_became_peered":"2023-11-18T20:43:33.786697+0000","last_unstale":"2023-11-18T20:43:33.794782+0000","last_undegraded":"2023-11-18T20:43:33.794782+0000","last_fullsized":"2023-11-18T20:43:33.794782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:42:24.039703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.115829+0000","last_change":"2023-11-18T20:43:32.788325+0000","last_active":"2023-11-18T20:43:34.115829+0000","last_peered":"2023-11-18T20:43:34.115829+0000","last_clean":"2023-11-18T20:43:34.115829+0000","last_became_active":"2023-11-18T20:43:32.787872+0000","last_became_peered":"2023-11-18T20:43:32.787872+0000","last_unstale":"2023-11-18T20:43:34.115829+0000","last_undegraded":"2023-11-18T20:43:34.115829+0000","last_fullsized":"2023-11-18T20:43:34.115829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:42:23.035593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.796289+0000","last_change":"2023-11-18T20:43:33.794602+0000","last_active":"2023-11-18T20:43:33.796289+0000","last_peered":"2023-11-18T20:43:33.796289+0000","last_clean":"2023-11-18T20:43:33.796289+0000","last_became_active":"2023-11-18T20:43:33.793716+0000","last_became_peered":"2023-11-18T20:43:33.793716+0000","last_unstale":"2023-11-18T20:43:33.796289+0000","last_undegraded":"2023-11-18T20:43:33.796289+0000","last_fullsized":"2023-11-18T20:43:33.796289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:45:21.602664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.783762+0000","last_change":"2023-11-18T20:43:32.789855+0000","last_active":"2023-11-18T20:43:33.783762+0000","last_peered":"2023-11-18T20:43:33.783762+0000","last_clean":"2023-11-18T20:43:33.783762+0000","last_became_active":"2023-11-18T20:43:32.789541+0000","last_became_peered":"2023-11-18T20:43:32.789541+0000","last_unstale":"2023-11-18T20:43:33.783762+0000","last_undegraded":"2023-11-18T20:43:33.783762+0000","last_fullsized":"2023-11-18T20:43:33.783762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:12:52.155157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.787861+0000","last_change":"2023-11-18T20:43:33.786320+0000","last_active":"2023-11-18T20:43:33.787861+0000","last_peered":"2023-11-18T20:43:33.787861+0000","last_clean":"2023-11-18T20:43:33.787861+0000","last_became_active":"2023-11-18T20:43:33.785918+0000","last_became_peered":"2023-11-18T20:43:33.785918+0000","last_unstale":"2023-11-18T20:43:33.787861+0000","last_undegraded":"2023-11-18T20:43:33.787861+0000","last_fullsized":"2023-11-18T20:43:33.787861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:42:10.376052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.111518+0000","last_change":"2023-11-18T20:43:32.788060+0000","last_active":"2023-11-18T20:43:34.111518+0000","last_peered":"2023-11-18T20:43:34.111518+0000","last_clean":"2023-11-18T20:43:34.111518+0000","last_became_active":"2023-11-18T20:43:32.787744+0000","last_became_peered":"2023-11-18T20:43:32.787744+0000","last_unstale":"2023-11-18T20:43:34.111518+0000","last_undegraded":"2023-11-18T20:43:34.111518+0000","last_fullsized":"2023-11-18T20:43:34.111518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:12.977297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.793252+0000","last_change":"2023-11-18T20:43:33.789461+0000","last_active":"2023-11-18T20:43:33.793252+0000","last_peered":"2023-11-18T20:43:33.793252+0000","last_clean":"2023-11-18T20:43:33.793252+0000","last_became_active":"2023-11-18T20:43:33.789177+0000","last_became_peered":"2023-11-18T20:43:33.789177+0000","last_unstale":"2023-11-18T20:43:33.793252+0000","last_undegraded":"2023-11-18T20:43:33.793252+0000","last_fullsized":"2023-11-18T20:43:33.793252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:16:01.555895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.787905+0000","last_change":"2023-11-18T20:43:32.789919+0000","last_active":"2023-11-18T20:43:33.787905+0000","last_peered":"2023-11-18T20:43:33.787905+0000","last_clean":"2023-11-18T20:43:33.787905+0000","last_became_active":"2023-11-18T20:43:32.789645+0000","last_became_peered":"2023-11-18T20:43:32.789645+0000","last_unstale":"2023-11-18T20:43:33.787905+0000","last_undegraded":"2023-11-18T20:43:33.787905+0000","last_fullsized":"2023-11-18T20:43:33.787905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:17.706929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.791213+0000","last_change":"2023-11-18T20:43:33.788805+0000","last_active":"2023-11-18T20:43:33.791213+0000","last_peered":"2023-11-18T20:43:33.791213+0000","last_clean":"2023-11-18T20:43:33.791213+0000","last_became_active":"2023-11-18T20:43:33.788552+0000","last_became_peered":"2023-11-18T20:43:33.788552+0000","last_unstale":"2023-11-18T20:43:33.791213+0000","last_undegraded":"2023-11-18T20:43:33.791213+0000","last_fullsized":"2023-11-18T20:43:33.791213+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:48:59.977613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.783983+0000","last_change":"2023-11-18T20:43:32.786191+0000","last_active":"2023-11-18T20:43:33.783983+0000","last_peered":"2023-11-18T20:43:33.783983+0000","last_clean":"2023-11-18T20:43:33.783983+0000","last_became_active":"2023-11-18T20:43:32.785923+0000","last_became_peered":"2023-11-18T20:43:32.785923+0000","last_unstale":"2023-11-18T20:43:33.783983+0000","last_undegraded":"2023-11-18T20:43:33.783983+0000","last_fullsized":"2023-11-18T20:43:33.783983+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:17.706929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.799995+0000","last_change":"2023-11-18T20:43:33.797837+0000","last_active":"2023-11-18T20:43:33.799995+0000","last_peered":"2023-11-18T20:43:33.799995+0000","last_clean":"2023-11-18T20:43:33.799995+0000","last_became_active":"2023-11-18T20:43:33.797519+0000","last_became_peered":"2023-11-18T20:43:33.797519+0000","last_unstale":"2023-11-18T20:43:33.799995+0000","last_undegraded":"2023-11-18T20:43:33.799995+0000","last_fullsized":"2023-11-18T20:43:33.799995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:05:54.025273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.787087+0000","last_change":"2023-11-18T20:43:32.789154+0000","last_active":"2023-11-18T20:43:33.787087+0000","last_peered":"2023-11-18T20:43:33.787087+0000","last_clean":"2023-11-18T20:43:33.787087+0000","last_became_active":"2023-11-18T20:43:32.788885+0000","last_became_peered":"2023-11-18T20:43:32.788885+0000","last_unstale":"2023-11-18T20:43:33.787087+0000","last_undegraded":"2023-11-18T20:43:33.787087+0000","last_fullsized":"2023-11-18T20:43:33.787087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:12.977297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.800483+0000","last_change":"2023-11-18T20:43:33.796869+0000","last_active":"2023-11-18T20:43:33.800483+0000","last_peered":"2023-11-18T20:43:33.800483+0000","last_clean":"2023-11-18T20:43:33.800483+0000","last_became_active":"2023-11-18T20:43:33.796534+0000","last_became_peered":"2023-11-18T20:43:33.796534+0000","last_unstale":"2023-11-18T20:43:33.800483+0000","last_undegraded":"2023-11-18T20:43:33.800483+0000","last_fullsized":"2023-11-18T20:43:33.800483+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:36:53.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:34.113546+0000","last_change":"2023-11-18T20:43:32.791936+0000","last_active":"2023-11-18T20:43:34.113546+0000","last_peered":"2023-11-18T20:43:34.113546+0000","last_clean":"2023-11-18T20:43:34.113546+0000","last_became_active":"2023-11-18T20:43:32.791417+0000","last_became_peered":"2023-11-18T20:43:32.791417+0000","last_unstale":"2023-11-18T20:43:34.113546+0000","last_undegraded":"2023-11-18T20:43:34.113546+0000","last_fullsized":"2023-11-18T20:43:34.113546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:41.878375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807563+0000","last_change":"2023-11-18T20:43:33.801148+0000","last_active":"2023-11-18T20:43:33.807563+0000","last_peered":"2023-11-18T20:43:33.807563+0000","last_clean":"2023-11-18T20:43:33.807563+0000","last_became_active":"2023-11-18T20:43:33.800838+0000","last_became_peered":"2023-11-18T20:43:33.800838+0000","last_unstale":"2023-11-18T20:43:33.807563+0000","last_undegraded":"2023-11-18T20:43:33.807563+0000","last_fullsized":"2023-11-18T20:43:33.807563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:30:42.882485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.786407+0000","last_change":"2023-11-18T20:43:32.789195+0000","last_active":"2023-11-18T20:43:33.786407+0000","last_peered":"2023-11-18T20:43:33.786407+0000","last_clean":"2023-11-18T20:43:33.786407+0000","last_became_active":"2023-11-18T20:43:32.788462+0000","last_became_peered":"2023-11-18T20:43:32.788462+0000","last_unstale":"2023-11-18T20:43:33.786407+0000","last_undegraded":"2023-11-18T20:43:33.786407+0000","last_fullsized":"2023-11-18T20:43:33.786407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:12:36.755249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.797690+0000","last_change":"2023-11-18T20:43:33.795963+0000","last_active":"2023-11-18T20:43:33.797690+0000","last_peered":"2023-11-18T20:43:33.797690+0000","last_clean":"2023-11-18T20:43:33.797690+0000","last_became_active":"2023-11-18T20:43:33.794640+0000","last_became_peered":"2023-11-18T20:43:33.794640+0000","last_unstale":"2023-11-18T20:43:33.797690+0000","last_undegraded":"2023-11-18T20:43:33.797690+0000","last_fullsized":"2023-11-18T20:43:33.797690+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:12:53.159267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.784135+0000","last_change":"2023-11-18T20:43:32.787804+0000","last_active":"2023-11-18T20:43:33.784135+0000","last_peered":"2023-11-18T20:43:33.784135+0000","last_clean":"2023-11-18T20:43:33.784135+0000","last_became_active":"2023-11-18T20:43:32.787427+0000","last_became_peered":"2023-11-18T20:43:32.787427+0000","last_unstale":"2023-11-18T20:43:33.784135+0000","last_undegraded":"2023-11-18T20:43:33.784135+0000","last_fullsized":"2023-11-18T20:43:33.784135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:22:12.977297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.797982+0000","last_change":"2023-11-18T20:43:33.794711+0000","last_active":"2023-11-18T20:43:33.797982+0000","last_peered":"2023-11-18T20:43:33.797982+0000","last_clean":"2023-11-18T20:43:33.797982+0000","last_became_active":"2023-11-18T20:43:33.794422+0000","last_became_peered":"2023-11-18T20:43:33.794422+0000","last_unstale":"2023-11-18T20:43:33.797982+0000","last_undegraded":"2023-11-18T20:43:33.797982+0000","last_fullsized":"2023-11-18T20:43:33.797982+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:42:24.039703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.786505+0000","last_change":"2023-11-18T20:43:32.791063+0000","last_active":"2023-11-18T20:43:33.786505+0000","last_peered":"2023-11-18T20:43:33.786505+0000","last_clean":"2023-11-18T20:43:33.786505+0000","last_became_active":"2023-11-18T20:43:32.790779+0000","last_became_peered":"2023-11-18T20:43:32.790779+0000","last_unstale":"2023-11-18T20:43:33.786505+0000","last_undegraded":"2023-11-18T20:43:33.786505+0000","last_fullsized":"2023-11-18T20:43:33.786505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:56:22.470085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.798645+0000","last_change":"2023-11-18T20:43:33.797248+0000","last_active":"2023-11-18T20:43:33.798645+0000","last_peered":"2023-11-18T20:43:33.798645+0000","last_clean":"2023-11-18T20:43:33.798645+0000","last_became_active":"2023-11-18T20:43:33.796993+0000","last_became_peered":"2023-11-18T20:43:33.796993+0000","last_unstale":"2023-11-18T20:43:33.798645+0000","last_undegraded":"2023-11-18T20:43:33.798645+0000","last_fullsized":"2023-11-18T20:43:33.798645+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:42:24.039703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.784667+0000","last_change":"2023-11-18T20:43:32.791620+0000","last_active":"2023-11-18T20:43:33.784667+0000","last_peered":"2023-11-18T20:43:33.784667+0000","last_clean":"2023-11-18T20:43:33.784667+0000","last_became_active":"2023-11-18T20:43:32.791314+0000","last_became_peered":"2023-11-18T20:43:32.791314+0000","last_unstale":"2023-11-18T20:43:33.784667+0000","last_undegraded":"2023-11-18T20:43:33.784667+0000","last_fullsized":"2023-11-18T20:43:33.784667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:34:55.635591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.797752+0000","last_change":"2023-11-18T20:43:33.794620+0000","last_active":"2023-11-18T20:43:33.797752+0000","last_peered":"2023-11-18T20:43:33.797752+0000","last_clean":"2023-11-18T20:43:33.797752+0000","last_became_active":"2023-11-18T20:43:33.794327+0000","last_became_peered":"2023-11-18T20:43:33.794327+0000","last_unstale":"2023-11-18T20:43:33.797752+0000","last_undegraded":"2023-11-18T20:43:33.797752+0000","last_fullsized":"2023-11-18T20:43:33.797752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:19:18.711038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.784135+0000","last_change":"2023-11-18T20:43:32.786778+0000","last_active":"2023-11-18T20:43:33.784135+0000","last_peered":"2023-11-18T20:43:33.784135+0000","last_clean":"2023-11-18T20:43:33.784135+0000","last_became_active":"2023-11-18T20:43:32.786509+0000","last_became_peered":"2023-11-18T20:43:32.786509+0000","last_unstale":"2023-11-18T20:43:33.784135+0000","last_undegraded":"2023-11-18T20:43:33.784135+0000","last_fullsized":"2023-11-18T20:43:33.784135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:53:31.358885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.807633+0000","last_change":"2023-11-18T20:43:33.801524+0000","last_active":"2023-11-18T20:43:33.807633+0000","last_peered":"2023-11-18T20:43:33.807633+0000","last_clean":"2023-11-18T20:43:33.807633+0000","last_became_active":"2023-11-18T20:43:33.801068+0000","last_became_peered":"2023-11-18T20:43:33.801068+0000","last_unstale":"2023-11-18T20:43:33.807633+0000","last_undegraded":"2023-11-18T20:43:33.807633+0000","last_fullsized":"2023-11-18T20:43:33.807633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:56:06.214458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.791360+0000","last_change":"2023-11-18T20:43:32.788614+0000","last_active":"2023-11-18T20:43:33.791360+0000","last_peered":"2023-11-18T20:43:33.791360+0000","last_clean":"2023-11-18T20:43:33.791360+0000","last_became_active":"2023-11-18T20:43:32.788171+0000","last_became_peered":"2023-11-18T20:43:32.788171+0000","last_unstale":"2023-11-18T20:43:33.791360+0000","last_undegraded":"2023-11-18T20:43:33.791360+0000","last_fullsized":"2023-11-18T20:43:33.791360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:04:24.315192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.799980+0000","last_change":"2023-11-18T20:43:33.798571+0000","last_active":"2023-11-18T20:43:33.799980+0000","last_peered":"2023-11-18T20:43:33.799980+0000","last_clean":"2023-11-18T20:43:33.799980+0000","last_became_active":"2023-11-18T20:43:33.798309+0000","last_became_peered":"2023-11-18T20:43:33.798309+0000","last_unstale":"2023-11-18T20:43:33.799980+0000","last_undegraded":"2023-11-18T20:43:33.799980+0000","last_fullsized":"2023-11-18T20:43:33.799980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:32.767586+0000","last_clean_scrub_stamp":"2023-11-18T20:43:32.767586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:32:05.273998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-18T20:43:33.786854+0000","last_change":"2023-11-18T20:43:32.791542+0000","last_active":"2023-11-18T20:43:33.786854+0000","last_peered":"2023-11-18T20:43:33.786854+0000","last_clean":"2023-11-18T20:43:33.786854+0000","last_became_active":"2023-11-18T20:43:32.791233+0000","last_became_peered":"2023-11-18T20:43:32.791233+0000","last_unstale":"2023-11-18T20:43:33.786854+0000","last_undegraded":"2023-11-18T20:43:33.786854+0000","last_fullsized":"2023-11-18T20:43:33.786854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:43:31.763476+0000","last_clean_scrub_stamp":"2023-11-18T20:43:31.763476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:56:17.823126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-18T20:43:32.783982+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:43:32.783982+0000","last_peered":"2023-11-18T20:43:32.783982+0000","last_clean":"2023-11-18T20:43:32.783982+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:43:32.783982+0000","last_undegraded":"2023-11-18T20:43:32.783982+0000","last_fullsized":"2023-11-18T20:43:32.783982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":954368,"data_stored":728556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.496}]},{"osd":1,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.432}]},{"osd":2,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51600000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55000000000000004}]},{"osd":4,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.33400000000000002}]},{"osd":5,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.315},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.40799999999999997}]},{"osd":6,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.56799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.376}]},{"osd":1,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42599999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.35299999999999998}]},{"osd":7,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.628}]},{"osd":1,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47799999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.52000000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.502},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.66200000000000003}]},{"osd":6,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56100000000000005}]},{"osd":7,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.67800000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57599999999999996}]},{"osd":1,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.621}]},{"osd":2,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.504}]},{"osd":3,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.46800000000000003}]},{"osd":5,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.51900000000000002}]},{"osd":6,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.44800000000000001}]},{"osd":7,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.314},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.252}]},{"osd":1,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.314},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.38100000000000001}]},{"osd":2,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.41599999999999998}]},{"osd":4,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57899999999999996}]},{"osd":5,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.72799999999999998}]},{"osd":6,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Sat Nov 18 20:43:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.58799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.311},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.35399999999999998}]},{"osd":1,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.254},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.27100000000000002}]},{"osd":3,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33300000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.54800000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":360448,"data_stored":138188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.29899999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.246},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34799999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.40799999999999997}]},{"osd":4,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.436},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47499999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56499999999999995}]},{"osd":6,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Nov 18 20:43:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.502}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":962560,"data_stored":734712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.217},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.33900000000000002}]},{"osd":2,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.247},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.28100000000000003}]},{"osd":3,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.376}]},{"osd":4,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57399999999999995}]},{"osd":5,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:43:26 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:43:37.458 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-18T20:43:37.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-18T20:43:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.605+0000 7f868dbe9640 1 Processor -- start 2023-11-18T20:43:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.605+0000 7f868dbe9640 1 -- start start 2023-11-18T20:43:37.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f868dbe9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8688146010 0x7f86880cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f868dbe9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8688146d70 0x7f86880b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f868dbe9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8688161170 con 0x7f86880b4060 2023-11-18T20:43:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f868dbe9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8688161450 con 0x7f8688146d70 2023-11-18T20:43:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f868dbe9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8688161730 con 0x7f8688146010 2023-11-18T20:43:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f8686ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8688146d70 0x7f86880b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f8686ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8688146d70 0x7f86880b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44712/0 (socket says 172.21.15.60:44712) 2023-11-18T20:43:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f8686ffd640 1 -- 172.21.15.60:0/405221398 learned_addr learned my addr 172.21.15.60:0/405221398 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f86877fe640 1 --2- 172.21.15.60:0/405221398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8688146010 0x7f86880cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f8686ffd640 1 -- 172.21.15.60:0/405221398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8688146010 msgr2=0x7f86880cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f8686ffd640 1 --2- 172.21.15.60:0/405221398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8688146010 0x7f86880cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f8686ffd640 1 -- 172.21.15.60:0/405221398 >> v1:172.21.15.100:6789/0 conn(0x7f86880b4060 legacy=0x7f86880d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.606+0000 7f8686ffd640 1 -- 172.21.15.60:0/405221398 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8688161a10 con 0x7f8688146d70 2023-11-18T20:43:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.607+0000 7f8686ffd640 1 --2- 172.21.15.60:0/405221398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146d70 0x7f86880b2cf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f867c009870 tx=0x7f867c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32a028fc9773298 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.607+0000 7f86867fc640 1 -- 172.21.15.60:0/405221398 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c030840 con 0x7f8688146d70 2023-11-18T20:43:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.607+0000 7f86867fc640 1 -- 172.21.15.60:0/405221398 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c0309a0 con 0x7f8688146d70 2023-11-18T20:43:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.607+0000 7f86867fc640 1 -- 172.21.15.60:0/405221398 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c030d30 con 0x7f8688146d70 2023-11-18T20:43:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.608+0000 7f868dbe9640 1 -- 172.21.15.60:0/405221398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146d70 msgr2=0x7f86880b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.608+0000 7f868dbe9640 1 --2- 172.21.15.60:0/405221398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146d70 0x7f86880b2cf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f867c009870 tx=0x7f867c02b980 comp rx=0 tx=0).stop 2023-11-18T20:43:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.608+0000 7f868dbe9640 1 -- 172.21.15.60:0/405221398 shutdown_connections 2023-11-18T20:43:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.608+0000 7f868dbe9640 1 --2- 172.21.15.60:0/405221398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146d70 0x7f86880b2cf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.608+0000 7f868dbe9640 1 --2- 172.21.15.60:0/405221398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8688146010 0x7f86880cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.608+0000 7f868dbe9640 1 -- 172.21.15.60:0/405221398 >> 172.21.15.60:0/405221398 conn(0x7f868806a4d0 msgr2=0x7f868806a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.608+0000 7f868dbe9640 1 -- 172.21.15.60:0/405221398 shutdown_connections 2023-11-18T20:43:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.608+0000 7f868dbe9640 1 -- 172.21.15.60:0/405221398 wait complete. 2023-11-18T20:43:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.609+0000 7f868dbe9640 1 Processor -- start 2023-11-18T20:43:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.609+0000 7f868dbe9640 1 -- start start 2023-11-18T20:43:37.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f868dbe9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86880b4060 0x7f86881f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f868dbe9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146010 0x7f86881f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f868dbe9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8688146d70 0x7f868815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f868dbe9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86881f31a0 con 0x7f8688146010 2023-11-18T20:43:37.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f868dbe9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868815f2b0 con 0x7f8688146d70 2023-11-18T20:43:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f868dbe9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868815e6a0 con 0x7f86880b4060 2023-11-18T20:43:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f8686ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146010 0x7f86881f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f8686ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146010 0x7f86881f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44718/0 (socket says 172.21.15.60:44718) 2023-11-18T20:43:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f8686ffd640 1 -- 172.21.15.60:0/3140651599 learned_addr learned my addr 172.21.15.60:0/3140651599 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f8687fff640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8688146d70 0x7f868815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.610+0000 7f86877fe640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86880b4060 0x7f86881f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f8686ffd640 1 -- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86880b4060 msgr2=0x7f86881f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f8686ffd640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86880b4060 0x7f86881f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f8686ffd640 1 -- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8688146d70 msgr2=0x7f868815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f8686ffd640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8688146d70 0x7f868815da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f8686ffd640 1 -- 172.21.15.60:0/3140651599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c0094d0 con 0x7f8688146010 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f86877fe640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86880b4060 0x7f86881f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f8686ffd640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146010 0x7f86881f68a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f867c0090b0 tx=0x7f867c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f866ffff640 1 -- 172.21.15.60:0/3140651599 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c03e040 con 0x7f8688146010 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f866ffff640 1 -- 172.21.15.60:0/3140651599 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c039cb0 con 0x7f8688146010 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f868dbe9640 1 -- 172.21.15.60:0/3140651599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8688146570 con 0x7f8688146010 2023-11-18T20:43:37.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f866ffff640 1 -- 172.21.15.60:0/3140651599 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c043680 con 0x7f8688146010 2023-11-18T20:43:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.611+0000 7f868dbe9640 1 -- 172.21.15.60:0/3140651599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868815ec60 con 0x7f8688146010 2023-11-18T20:43:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.612+0000 7f868dbe9640 1 -- 172.21.15.60:0/3140651599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868815f3f0 con 0x7f8688146010 2023-11-18T20:43:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.612+0000 7f866ffff640 1 -- 172.21.15.60:0/3140651599 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f867c0423e0 con 0x7f8688146010 2023-11-18T20:43:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.613+0000 7f866ffff640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f867406ae40 0x7f867406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.613+0000 7f866ffff640 1 -- 172.21.15.60:0/3140651599 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f867c0abb00 con 0x7f8688146010 2023-11-18T20:43:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.613+0000 7f86877fe640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f867406ae40 0x7f867406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.614+0000 7f86877fe640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f867406ae40 0x7f867406d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8678006fd0 tx=0x7f8678008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:37.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.617+0000 7f866ffff640 1 -- 172.21.15.60:0/3140651599 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f868815f3f0 con 0x7f8688146010 2023-11-18T20:43:37.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.862+0000 7f868dbe9640 1 -- 172.21.15.60:0/3140651599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f868815f3f0 con 0x7f8688146010 2023-11-18T20:43:37.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.864+0000 7f866ffff640 1 -- 172.21.15.60:0/3140651599 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f868815f3f0 con 0x7f8688146010 2023-11-18T20:43:37.865 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-11-18T20:43:37.865 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA== 2023-11-18T20:43:37.865 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-11-18T20:43:37.865 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-11-18T20:43:37.865 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-18T20:43:37.866 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-11-18T20:43:37.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 -- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f867406ae40 msgr2=0x7f867406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f867406ae40 0x7f867406d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8678006fd0 tx=0x7f8678008040 comp rx=0 tx=0).stop 2023-11-18T20:43:37.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 -- 172.21.15.60:0/3140651599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146010 msgr2=0x7f86881f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146010 0x7f86881f68a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f867c0090b0 tx=0x7f867c004730 comp rx=0 tx=0).stop 2023-11-18T20:43:37.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 -- 172.21.15.60:0/3140651599 shutdown_connections 2023-11-18T20:43:37.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f867406ae40 0x7f867406d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8688146d70 0x7f868815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8688146010 0x7f86881f68a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 --2- 172.21.15.60:0/3140651599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86880b4060 0x7f86881f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 -- 172.21.15.60:0/3140651599 >> 172.21.15.60:0/3140651599 conn(0x7f868806a4d0 msgr2=0x7f86880b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:37.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 -- 172.21.15.60:0/3140651599 shutdown_connections 2023-11-18T20:43:37.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.866+0000 7f868dbe9640 1 -- 172.21.15.60:0/3140651599 wait complete. 2023-11-18T20:43:37.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:43:37.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.983+0000 7fd038941640 1 Processor -- start 2023-11-18T20:43:37.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.983+0000 7fd038941640 1 -- start start 2023-11-18T20:43:37.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd038941640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd034140310 0x7fd0341618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd038941640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0341406f0 con 0x7fd03406abe0 2023-11-18T20:43:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd038941640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd034143960 con 0x7fd0341409d0 2023-11-18T20:43:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd038941640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd034161e20 con 0x7fd034140310 2023-11-18T20:43:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd03259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd034140310 0x7fd0341618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd03259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd034140310 0x7fd0341618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44728/0 (socket says 172.21.15.60:44728) 2023-11-18T20:43:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd032d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd03406abe0 0x7fd03406efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37008/0 (socket says 172.21.15.60:37008) 2023-11-18T20:43:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd03259c640 1 -- 172.21.15.60:0/468482578 learned_addr learned my addr 172.21.15.60:0/468482578 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd03259c640 1 -- 172.21.15.60:0/468482578 >> v1:172.21.15.100:6789/0 conn(0x7fd0341409d0 legacy=0x7fd03406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd03259c640 1 -- 172.21.15.60:0/468482578 >> v1:172.21.15.60:6789/0 conn(0x7fd03406abe0 legacy=0x7fd03406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd031d9b640 1 -- 172.21.15.60:0/468482578 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485372675 0 0) 0x7fd0341406f0 con 0x7fd03406abe0 2023-11-18T20:43:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.984+0000 7fd03259c640 1 -- 172.21.15.60:0/468482578 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd034142de0 con 0x7fd034140310 2023-11-18T20:43:37.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.985+0000 7fd03259c640 1 --2- 172.21.15.60:0/468482578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 0x7fd0341618e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd024009870 tx=0x7fd02402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ce1fb42fd04cdd server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:37.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.985+0000 7fd031d9b640 1 -- 172.21.15.60:0/468482578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024030860 con 0x7fd034140310 2023-11-18T20:43:37.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.985+0000 7fd031d9b640 1 -- 172.21.15.60:0/468482578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0240309c0 con 0x7fd034140310 2023-11-18T20:43:37.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.985+0000 7fd031d9b640 1 -- 172.21.15.60:0/468482578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024030d50 con 0x7fd034140310 2023-11-18T20:43:37.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.985+0000 7fd038941640 1 -- 172.21.15.60:0/468482578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 msgr2=0x7fd0341618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.985+0000 7fd038941640 1 --2- 172.21.15.60:0/468482578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 0x7fd0341618e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd024009870 tx=0x7fd02402b9a0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.986+0000 7fd038941640 1 -- 172.21.15.60:0/468482578 shutdown_connections 2023-11-18T20:43:37.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.986+0000 7fd038941640 1 --2- 172.21.15.60:0/468482578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 0x7fd0341618e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.986+0000 7fd038941640 1 -- 172.21.15.60:0/468482578 >> 172.21.15.60:0/468482578 conn(0x7fd0340b1e50 msgr2=0x7fd0340b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:37.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.986+0000 7fd038941640 1 -- 172.21.15.60:0/468482578 shutdown_connections 2023-11-18T20:43:37.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.986+0000 7fd038941640 1 -- 172.21.15.60:0/468482578 wait complete. 2023-11-18T20:43:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.987+0000 7fd038941640 1 Processor -- start 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.987+0000 7fd038941640 1 -- start start 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd038941640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03406abe0 0x7fd034154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd038941640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 0x7fd03413f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd038941640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0341409d0 0x7fd034157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd038941640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd034154ff0 con 0x7fd034140310 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd038941640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0341552d0 con 0x7fd03406abe0 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd038941640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0341555b0 con 0x7fd0341409d0 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd029d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 0x7fd03413f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd029d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 0x7fd03413f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44734/0 (socket says 172.21.15.60:44734) 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd029d9b640 1 -- 172.21.15.60:0/3283757438 learned_addr learned my addr 172.21.15.60:0/3283757438 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:37.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd032d9d640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0341409d0 0x7fd034157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd03259c640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03406abe0 0x7fd034154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd029d9b640 1 -- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0341409d0 msgr2=0x7fd034157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd029d9b640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0341409d0 0x7fd034157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd029d9b640 1 -- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03406abe0 msgr2=0x7fd034154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd029d9b640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03406abe0 0x7fd034154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd029d9b640 1 -- 172.21.15.60:0/3283757438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0240094d0 con 0x7fd034140310 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd03259c640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03406abe0 0x7fd034154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.988+0000 7fd029d9b640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 0x7fd03413f6e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c450 tx=0x7fd01c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.989+0000 7fd02b7fe640 1 -- 172.21.15.60:0/3283757438 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0052d0 con 0x7fd034140310 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.989+0000 7fd02b7fe640 1 -- 172.21.15.60:0/3283757438 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c005430 con 0x7fd034140310 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.989+0000 7fd038941640 1 -- 172.21.15.60:0/3283757438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd034155890 con 0x7fd034140310 2023-11-18T20:43:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.989+0000 7fd02b7fe640 1 -- 172.21.15.60:0/3283757438 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c012870 con 0x7fd034140310 2023-11-18T20:43:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.989+0000 7fd038941640 1 -- 172.21.15.60:0/3283757438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0341578a0 con 0x7fd034140310 2023-11-18T20:43:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.990+0000 7fd038941640 1 -- 172.21.15.60:0/3283757438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd034161030 con 0x7fd034140310 2023-11-18T20:43:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.990+0000 7fd02b7fe640 1 -- 172.21.15.60:0/3283757438 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd01c0129d0 con 0x7fd034140310 2023-11-18T20:43:37.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.991+0000 7fd02b7fe640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd01406ae40 0x7fd01406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:37.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.991+0000 7fd02b7fe640 1 -- 172.21.15.60:0/3283757438 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd01c0891e0 con 0x7fd034140310 2023-11-18T20:43:37.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.991+0000 7fd03259c640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd01406ae40 0x7fd01406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:37.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.991+0000 7fd03259c640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd01406ae40 0x7fd01406d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd024036000 tx=0x7fd02403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:37.995+0000 7fd02b7fe640 1 -- 172.21.15.60:0/3283757438 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd01c054210 con 0x7fd034140310 2023-11-18T20:43:38.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.207+0000 7fd038941640 1 -- 172.21.15.60:0/3283757438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd034069400 con 0x7fd034140310 2023-11-18T20:43:38.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.208+0000 7fd02b7fe640 1 -- 172.21.15.60:0/3283757438 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fd01c057830 con 0x7fd034140310 2023-11-18T20:43:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:38.209 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4814,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3885363174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3885363174},{"type":"v1","addr":"172.21.15.100:6837","nonce":3885363174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1102236552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1102236552},{"type":"v1","addr":"172.21.15.60:6835","nonce":1102236552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2680685354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2680685354},{"type":"v1","addr":"172.21.15.100:6836","nonce":2680685354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:43:33.146258+0000","modified":"2023-11-18T20:43:35.454446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5065":{"gid":5065,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3762312847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3762312847},{"type":"v1","addr":"172.21.15.60:6834","nonce":3762312847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-18T20:43:38.209 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 13 2023-11-18T20:43:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.209+0000 7fd038941640 1 -- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd01406ae40 msgr2=0x7fd01406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.209+0000 7fd038941640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd01406ae40 0x7fd01406d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd024036000 tx=0x7fd02403b040 comp rx=0 tx=0).stop 2023-11-18T20:43:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.210+0000 7fd038941640 1 -- 172.21.15.60:0/3283757438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 msgr2=0x7fd03413f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.210+0000 7fd038941640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 0x7fd03413f6e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c450 tx=0x7fd01c00c910 comp rx=0 tx=0).stop 2023-11-18T20:43:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.210+0000 7fd038941640 1 -- 172.21.15.60:0/3283757438 shutdown_connections 2023-11-18T20:43:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.210+0000 7fd038941640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd01406ae40 0x7fd01406d300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.210+0000 7fd038941640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0341409d0 0x7fd034157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.210+0000 7fd038941640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd034140310 0x7fd03413f6e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.210+0000 7fd038941640 1 --2- 172.21.15.60:0/3283757438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03406abe0 0x7fd034154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.210+0000 7fd038941640 1 -- 172.21.15.60:0/3283757438 >> 172.21.15.60:0/3283757438 conn(0x7fd0340b1e50 msgr2=0x7fd0340afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.210+0000 7fd038941640 1 -- 172.21.15.60:0/3283757438 shutdown_connections 2023-11-18T20:43:38.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.210+0000 7fd038941640 1 -- 172.21.15.60:0/3283757438 wait complete. 2023-11-18T20:43:38.624 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-18T20:43:33.146258+0000', 'modified': '2023-11-18T20:43:35.454446+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5065}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5065': {'gid': 5065, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6834/3762312847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 3762312847}, {'type': 'v1', 'addr': '172.21.15.60:6834', 'nonce': 3762312847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-18T20:43:38.624 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-18T20:43:38.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-18T20:43:38.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.737+0000 7f1bbec44640 1 Processor -- start 2023-11-18T20:43:38.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.737+0000 7f1bbec44640 1 -- start start 2023-11-18T20:43:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbec44640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 0x7f1bb806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbec44640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1bb80aeb80 0x7f1bb806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbec44640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb80adae0 con 0x7f1bb80ad640 2023-11-18T20:43:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbec44640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb80addc0 con 0x7f1bb80aeb80 2023-11-18T20:43:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbec44640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb80ae0a0 con 0x7f1bb815d1d0 2023-11-18T20:43:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbc9e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 0x7f1bb806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbd1e1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1bb80aeb80 0x7f1bb806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bb7fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1bb80ad640 0x7f1bb8160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:35022/0 (socket says 172.21.15.60:35022) 2023-11-18T20:43:38.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bb7fff640 1 -- 172.21.15.60:0/3343729229 learned_addr learned my addr 172.21.15.60:0/3343729229 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:38.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/3343729229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 msgr2=0x7f1bb806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:38.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/3343729229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 0x7f1bb806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/3343729229 >> v1:172.21.15.100:6789/0 conn(0x7f1bb80ad640 legacy=0x7f1bb8160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/3343729229 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb80ae380 con 0x7f1bb80aeb80 2023-11-18T20:43:38.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.738+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/3343729229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 0x7f1bb806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:38.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.739+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/3343729229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb80aeb80 0x7f1bb806ba20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009d90 tx=0x7f1bb002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75aa6c70b8c2ec9b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.739+0000 7f1bb77fe640 1 -- 172.21.15.60:0/3343729229 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb002cc50 con 0x7f1bb80aeb80 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.739+0000 7f1bb77fe640 1 -- 172.21.15.60:0/3343729229 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb002cdb0 con 0x7f1bb80aeb80 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.739+0000 7f1bb77fe640 1 -- 172.21.15.60:0/3343729229 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0034660 con 0x7f1bb80aeb80 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.740+0000 7f1bbec44640 1 -- 172.21.15.60:0/3343729229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb80aeb80 msgr2=0x7f1bb806ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.740+0000 7f1bbec44640 1 --2- 172.21.15.60:0/3343729229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb80aeb80 0x7f1bb806ba20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009d90 tx=0x7f1bb002bea0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.740+0000 7f1bbec44640 1 -- 172.21.15.60:0/3343729229 shutdown_connections 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.740+0000 7f1bbec44640 1 --2- 172.21.15.60:0/3343729229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb80aeb80 0x7f1bb806ba20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.740+0000 7f1bbec44640 1 --2- 172.21.15.60:0/3343729229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 0x7f1bb806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.740+0000 7f1bbec44640 1 -- 172.21.15.60:0/3343729229 >> 172.21.15.60:0/3343729229 conn(0x7f1bb8070e70 msgr2=0x7f1bb8071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.740+0000 7f1bbec44640 1 -- 172.21.15.60:0/3343729229 shutdown_connections 2023-11-18T20:43:38.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.740+0000 7f1bbec44640 1 -- 172.21.15.60:0/3343729229 wait complete. 2023-11-18T20:43:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.741+0000 7f1bbec44640 1 Processor -- start 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.741+0000 7f1bbec44640 1 -- start start 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bbec44640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb80ad640 0x7f1bb80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bbec44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bb80aeb80 0x7f1bb8160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bbec44640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 0x7f1bb8149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bbec44640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8149ab0 con 0x7f1bb80aeb80 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bbec44640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb80ccbb0 con 0x7f1bb80ad640 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bbec44640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb80cce90 con 0x7f1bb815d1d0 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bb80aeb80 0x7f1bb8160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bb80aeb80 0x7f1bb8160400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44748/0 (socket says 172.21.15.60:44748) 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bb7fff640 1 -- 172.21.15.60:0/2934863822 learned_addr learned my addr 172.21.15.60:0/2934863822 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb80ad640 0x7f1bb80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 0x7f1bb8149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bb7fff640 1 -- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 msgr2=0x7f1bb8149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 0x7f1bb8149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.742+0000 7f1bb7fff640 1 -- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb80ad640 msgr2=0x7f1bb80cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.743+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb80ad640 0x7f1bb80cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.743+0000 7f1bb7fff640 1 -- 172.21.15.60:0/2934863822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac009400 con 0x7f1bb80aeb80 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.743+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bb80aeb80 0x7f1bb8160400 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400c4a0 tx=0x7f1ba400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.743+0000 7f1bb57fa640 1 -- 172.21.15.60:0/2934863822 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4005ce0 con 0x7f1bb80aeb80 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.743+0000 7f1bb57fa640 1 -- 172.21.15.60:0/2934863822 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba4005e40 con 0x7f1bb80aeb80 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.743+0000 7f1bbec44640 1 -- 172.21.15.60:0/2934863822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb0009a70 con 0x7f1bb80aeb80 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.743+0000 7f1bb57fa640 1 -- 172.21.15.60:0/2934863822 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4007270 con 0x7f1bb80aeb80 2023-11-18T20:43:38.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.743+0000 7f1bbec44640 1 -- 172.21.15.60:0/2934863822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb80cd450 con 0x7f1bb80aeb80 2023-11-18T20:43:38.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.745+0000 7f1bb57fa640 1 -- 172.21.15.60:0/2934863822 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ba4007490 con 0x7f1bb80aeb80 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.745+0000 7f1bb57fa640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1b9c06ae40 0x7f1b9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.745+0000 7f1bb57fa640 1 -- 172.21.15.60:0/2934863822 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ba4088b80 con 0x7f1bb80aeb80 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.746+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1b9c06ae40 0x7f1b9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.746+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1b9c06ae40 0x7f1b9c06d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009c30 tx=0x7f1bac009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.746+0000 7f1bc17cd740 1 Processor -- start 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.746+0000 7f1bc17cd740 1 -- start start 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32b90 0x558e08e05b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32f70 0x558e08e35920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e06070 0x558e08e35e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08dc68c0 con 0x558e08e32b90 2023-11-18T20:43:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08d78b90 con 0x558e08e06070 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e08d76280 con 0x558e08e32f70 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbc9e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32b90 0x558e08e05b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbc9e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32b90 0x558e08e05b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44762/0 (socket says 172.21.15.60:44762) 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbc9e0640 1 -- 172.21.15.60:0/124946244 learned_addr learned my addr 172.21.15.60:0/124946244 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32f70 0x558e08e35920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e06070 0x558e08e35e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbc9e0640 1 -- 172.21.15.60:0/124946244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32f70 msgr2=0x558e08e35920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32f70 0x558e08e35920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbc9e0640 1 -- 172.21.15.60:0/124946244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e06070 msgr2=0x558e08e35e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e06070 0x558e08e35e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbc9e0640 1 -- 172.21.15.60:0/124946244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e08d37500 con 0x558e08e32b90 2023-11-18T20:43:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32f70 0x558e08e35920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e06070 0x558e08e35e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.747+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32b90 0x558e08e05b30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0099a0 tx=0x7f1bac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b5fb457030f49a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1ba27fc640 1 -- 172.21.15.60:0/124946244 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac014d50 con 0x558e08e32b90 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1ba27fc640 1 -- 172.21.15.60:0/124946244 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac0213f0 con 0x558e08e32b90 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1ba27fc640 1 -- 172.21.15.60:0/124946244 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac021760 con 0x558e08e32b90 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1bc17cd740 1 -- 172.21.15.60:0/124946244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32b90 msgr2=0x558e08e05b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32b90 0x558e08e05b30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0099a0 tx=0x7f1bac014000 comp rx=0 tx=0).stop 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1bc17cd740 1 -- 172.21.15.60:0/124946244 shutdown_connections 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e06070 0x558e08e35e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32f70 0x558e08e35920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/124946244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32b90 0x558e08e05b30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1bc17cd740 1 -- 172.21.15.60:0/124946244 >> 172.21.15.60:0/124946244 conn(0x558e08e07da0 msgr2=0x558e08e50920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:38.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1bc17cd740 1 -- 172.21.15.60:0/124946244 shutdown_connections 2023-11-18T20:43:38.753 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.748+0000 7f1bc17cd740 1 -- 172.21.15.60:0/124946244 wait complete. 2023-11-18T20:43:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.749+0000 7f1bc17cd740 1 -- start start 2023-11-18T20:43:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.749+0000 7f1bc17cd740 1 Processor -- start 2023-11-18T20:43:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32b90 0x558e08e1ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e2f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e2fe70 0x558e08e3a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08d71b70 con 0x558e08e32f70 2023-11-18T20:43:38.756 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08d2be00 con 0x558e08e2fe70 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e08d2baa0 con 0x558e08e32b90 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bc17cd740 10 client.0 _ll_register_callbacks cb 0x558e08a83ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e2f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e2f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44764/0 (socket says 172.21.15.60:44764) 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bb7fff640 1 -- 172.21.15.60:0/807893503 learned_addr learned my addr 172.21.15.60:0/807893503 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32b90 0x558e08e1ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e2fe70 0x558e08e3a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bb7fff640 1 -- 172.21.15.60:0/807893503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32b90 msgr2=0x558e08e1ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32b90 0x558e08e1ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bb7fff640 1 -- 172.21.15.60:0/807893503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e2fe70 msgr2=0x558e08e3a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e2fe70 0x558e08e3a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.757 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.750+0000 7f1bb7fff640 1 -- 172.21.15.60:0/807893503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac009c60 con 0x558e08e32f70 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e2f930 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4086f80 tx=0x7f1ba4057930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f2612cffccd56b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1b977fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1b977fe640 1 -- 172.21.15.60:0/807893503 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4056f60 con 0x558e08e32f70 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1b977fe640 1 -- 172.21.15.60:0/807893503 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ba405f6c0 con 0x558e08e32f70 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bc17cd740 1 -- 172.21.15.60:0/807893503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e08d6ab50 con 0x558e08e32f70 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1b977fe640 1 -- 172.21.15.60:0/807893503 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ba406f180 con 0x558e08e32f70 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bc17cd740 10 client.5421 fetch_fsmap learned FSMap version 13 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bc17cd740 1 -- 172.21.15.60:0/807893503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x558e08d701f0 con 0x558e08e32f70 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1b977fe640 1 -- 172.21.15.60:0/807893503 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba4074020 con 0x558e08e32f70 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1b977fe640 1 -- 172.21.15.60:0/807893503 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1ba4058af0 con 0x558e08e32f70 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bc17cd740 10 client.5421 fetch_fsmap finished waiting for FSMap version 13 2023-11-18T20:43:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bc17cd740 10 client.5421 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bc17cd740 20 client.5421 populate_metadata read hostname 'smithi060' 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.60:6832/3762312847,v1:172.21.15.60:6834/3762312847] conn(0x558e08e1f520 0x558e08e1f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bc17cd740 4 client.5421 mds_command: new command op to 5065 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bc17cd740 1 -- 172.21.15.60:0/807893503 --> [v2:172.21.15.60:6832/3762312847,v1:172.21.15.60:6834/3762312847] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e08d6a870 con 0x558e08e1f520 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.751+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.60:6832/3762312847,v1:172.21.15.60:6834/3762312847] conn(0x558e08e1f520 0x558e08e1f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.60:6832/3762312847,v1:172.21.15.60:6834/3762312847] conn(0x558e08e1f520 0x558e08e1f900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4134f4e6f51cb8 server_cookie=36fbb1ac30ad5d10 in_seq=0 out_seq=0 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1b977fe640 10 client.5421 ms_handle_connect on v2:172.21.15.60:6832/3762312847 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1b977fe640 1 -- 172.21.15.60:0/807893503 <== mds.0 v2:172.21.15.60:6832/3762312847 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e08d6a870 con 0x558e08e1f520 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1b977fe640 10 client.5421 handle_command_reply: tid=0 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 client.5421 shutdown 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 -- 172.21.15.60:0/807893503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 msgr2=0x558e08e2f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e2f930 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4086f80 tx=0x7f1ba4057930 comp rx=0 tx=0).stop 2023-11-18T20:43:38.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 -- 172.21.15.60:0/807893503 shutdown_connections 2023-11-18T20:43:38.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.60:6832/3762312847,v1:172.21.15.60:6834/3762312847] conn(0x558e08e1f520 0x558e08e1f900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e2fe70 0x558e08e3a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e2f930 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1b977fe640 0 client.5421 ms_handle_reset on v2:172.21.15.60:6832/3762312847 2023-11-18T20:43:38.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/807893503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32b90 0x558e08e1ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 -- 172.21.15.60:0/807893503 >> 172.21.15.60:0/807893503 conn(0x558e08e06070 msgr2=0x558e08e09b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:38.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 -- 172.21.15.60:0/807893503 shutdown_connections 2023-11-18T20:43:38.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.752+0000 7f1bc17cd740 1 -- 172.21.15.60:0/807893503 wait complete. 2023-11-18T20:43:38.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.753+0000 7f1bc17cd740 20 client.5421 trim_cache size 0 max 16384 2023-11-18T20:43:38.767 INFO:tasks.ceph.mds.c.smithi060.stderr:2023-11-18T20:43:38.766+0000 7f7b99893640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-18T20:43:38.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.768+0000 7f1bc17cd740 1 Processor -- start 2023-11-18T20:43:38.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.768+0000 7f1bc17cd740 1 -- start start 2023-11-18T20:43:38.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.769+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e2fe70 0x558e08e51200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.769+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e32b90 0x558e08e2df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.769+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e53e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.769+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08cf1270 con 0x558e08e32f70 2023-11-18T20:43:38.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.769+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08e32060 con 0x558e08e32b90 2023-11-18T20:43:38.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.769+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e08d442a0 con 0x558e08e2fe70 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.769+0000 7f1bbd1e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e53e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.769+0000 7f1bbd1e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e53e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44770/0 (socket says 172.21.15.60:44770) 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.769+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/1837306345 learned_addr learned my addr 172.21.15.60:0/1837306345 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.769+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e2fe70 0x558e08e51200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e32b90 0x558e08e2df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/1837306345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e2fe70 msgr2=0x558e08e51200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e2fe70 0x558e08e51200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/1837306345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e32b90 msgr2=0x558e08e2df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e32b90 0x558e08e2df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/1837306345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e08e327a0 con 0x558e08e32f70 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e2fe70 0x558e08e51200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e32b90 0x558e08e2df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e53e30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009eb0 tx=0x7f1bb002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a414248fbc8ab579 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1ba2ffd640 1 -- 172.21.15.60:0/1837306345 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0044070 con 0x558e08e32f70 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1ba2ffd640 1 -- 172.21.15.60:0/1837306345 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb002cad0 con 0x558e08e32f70 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1ba2ffd640 1 -- 172.21.15.60:0/1837306345 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb003f640 con 0x558e08e32f70 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bc17cd740 1 -- 172.21.15.60:0/1837306345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 msgr2=0x558e08e53e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.770+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e53e30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009eb0 tx=0x7f1bb002e000 comp rx=0 tx=0).stop 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.771+0000 7f1bc17cd740 1 -- 172.21.15.60:0/1837306345 shutdown_connections 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.771+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08e53e30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.771+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e32b90 0x558e08e2df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.771+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/1837306345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e2fe70 0x558e08e51200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.771+0000 7f1bc17cd740 1 -- 172.21.15.60:0/1837306345 >> 172.21.15.60:0/1837306345 conn(0x558e08e06070 msgr2=0x558e08e09850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:38.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.771+0000 7f1bc17cd740 1 -- 172.21.15.60:0/1837306345 shutdown_connections 2023-11-18T20:43:38.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.771+0000 7f1bc17cd740 1 -- 172.21.15.60:0/1837306345 wait complete. 2023-11-18T20:43:38.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.771+0000 7f1bc17cd740 1 -- start start 2023-11-18T20:43:38.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.771+0000 7f1bc17cd740 1 Processor -- start 2023-11-18T20:43:38.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e2fe70 0x558e08ebae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32b90 0x558e08e51d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bc17cd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08ebbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08ebc9a0 con 0x558e08e32f70 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08e51200 con 0x558e08e2fe70 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bc17cd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e08e1b3b0 con 0x558e08e32b90 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bc17cd740 10 client.0 _ll_register_callbacks cb 0x7f1bac011f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bbd1e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08ebbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bbd1e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08ebbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44786/0 (socket says 172.21.15.60:44786) 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/2065524269 learned_addr learned my addr 172.21.15.60:0/2065524269 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e2fe70 0x558e08ebae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bb7fff640 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32b90 0x558e08e51d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/2065524269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32b90 msgr2=0x558e08e51d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32b90 0x558e08e51d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/2065524269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e2fe70 msgr2=0x558e08ebae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e2fe70 0x558e08ebae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.772+0000 7f1bbd1e1640 1 -- 172.21.15.60:0/2065524269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb80ae380 con 0x558e08e32f70 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bbd1e1640 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08ebbdd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1bb004a040 tx=0x7f1bb0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759a1b41da8067f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1b977fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1b977fe640 1 -- 172.21.15.60:0/2065524269 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0004800 con 0x558e08e32f70 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1b977fe640 1 -- 172.21.15.60:0/2065524269 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bb0003e40 con 0x558e08e32f70 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bc17cd740 1 -- 172.21.15.60:0/2065524269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bac0213f0 con 0x558e08e32f70 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1b977fe640 1 -- 172.21.15.60:0/2065524269 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bb0034440 con 0x558e08e32f70 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bc17cd740 10 client.5427 fetch_fsmap learned FSMap version 13 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bc17cd740 1 -- 172.21.15.60:0/2065524269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x558e08d0dd00 con 0x558e08e32f70 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1b977fe640 1 -- 172.21.15.60:0/2065524269 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac0213f0 con 0x558e08e32f70 2023-11-18T20:43:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1b977fe640 1 -- 172.21.15.60:0/2065524269 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1bb00345a0 con 0x558e08e32f70 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bc17cd740 10 client.5427 fetch_fsmap finished waiting for FSMap version 13 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bc17cd740 10 client.5427 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bc17cd740 20 client.5427 populate_metadata read hostname 'smithi060' 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.60:6832/3762312847,v1:172.21.15.60:6834/3762312847] conn(0x558e08e362d0 0x558e08e50050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bc17cd740 4 client.5427 mds_command: new command op to 5065 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bc17cd740 1 -- 172.21.15.60:0/2065524269 --> [v2:172.21.15.60:6832/3762312847,v1:172.21.15.60:6834/3762312847] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558e08a726c0 con 0x558e08e362d0 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.773+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.60:6832/3762312847,v1:172.21.15.60:6834/3762312847] conn(0x558e08e362d0 0x558e08e50050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1b977fe640 10 client.5427 ms_handle_connect on v2:172.21.15.60:6832/3762312847 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bbc9e0640 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.60:6832/3762312847,v1:172.21.15.60:6834/3762312847] conn(0x558e08e362d0 0x558e08e50050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f691b74fda7288 server_cookie=f34c5963e5d3ce9b in_seq=0 out_seq=0 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1b977fe640 1 -- 172.21.15.60:0/2065524269 <== mds.0 v2:172.21.15.60:6832/3762312847 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x558e08a726c0 con 0x558e08e362d0 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1b977fe640 10 client.5427 handle_command_reply: tid=0 2023-11-18T20:43:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 client.5427 shutdown 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 -- 172.21.15.60:0/2065524269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 msgr2=0x558e08ebbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08ebbdd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1bb004a040 tx=0x7f1bb0004ab0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 -- 172.21.15.60:0/2065524269 shutdown_connections 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.60:6832/3762312847,v1:172.21.15.60:6834/3762312847] conn(0x558e08e362d0 0x558e08e50050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558e08e32f70 0x558e08ebbdd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558e08e32b90 0x558e08e51d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 --2- 172.21.15.60:0/2065524269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e08e2fe70 0x558e08ebae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1b977fe640 0 client.5427 ms_handle_reset on v2:172.21.15.60:6832/3762312847 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 -- 172.21.15.60:0/2065524269 >> 172.21.15.60:0/2065524269 conn(0x558e08e06070 msgr2=0x558e08e0bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 -- 172.21.15.60:0/2065524269 shutdown_connections 2023-11-18T20:43:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 1 -- 172.21.15.60:0/2065524269 wait complete. 2023-11-18T20:43:38.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.774+0000 7f1bc17cd740 20 client.5427 trim_cache size 0 max 16384 2023-11-18T20:43:38.783 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:43:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "ae4682cb-f89a-4edd-b6d6-55eae03fd1b8", 2023-11-18T20:43:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-11-18T20:43:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5065, 2023-11-18T20:43:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-11-18T20:43:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-11-18T20:43:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-11-18T20:43:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.6743424249999999, 2023-11-18T20:43:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 13, 2023-11-18T20:43:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 34, 2023-11-18T20:43:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 34, 2023-11-18T20:43:38.784 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.864717865999999 2023-11-18T20:43:38.784 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:43:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 -- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1b9c06ae40 msgr2=0x7f1b9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1b9c06ae40 0x7f1b9c06d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009c30 tx=0x7f1bac009190 comp rx=0 tx=0).stop 2023-11-18T20:43:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 -- 172.21.15.60:0/2934863822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bb80aeb80 msgr2=0x7f1bb8160400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bb80aeb80 0x7f1bb8160400 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400c4a0 tx=0x7f1ba400c960 comp rx=0 tx=0).stop 2023-11-18T20:43:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 -- 172.21.15.60:0/2934863822 shutdown_connections 2023-11-18T20:43:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1b9c06ae40 0x7f1b9c06d300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bb815d1d0 0x7f1bb8149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bb80aeb80 0x7f1bb8160400 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 --2- 172.21.15.60:0/2934863822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bb80ad640 0x7f1bb80cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 -- 172.21.15.60:0/2934863822 >> 172.21.15.60:0/2934863822 conn(0x7f1bb8070e70 msgr2=0x7f1bb815a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 -- 172.21.15.60:0/2934863822 shutdown_connections 2023-11-18T20:43:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.775+0000 7f1bbec44640 1 -- 172.21.15.60:0/2934863822 wait complete. 2023-11-18T20:43:38.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.892+0000 7f9f01bfb640 1 Processor -- start 2023-11-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.892+0000 7f9f01bfb640 1 -- start start 2023-11-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.892+0000 7f9f01bfb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc146460 0x7f9efc0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9f01bfb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9efc146840 0x7f9efc06baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9f01bfb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9efc146fd0 con 0x7f9efc06eab0 2023-11-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9f01bfb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9efc06c180 con 0x7f9efc146840 2023-11-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9f01bfb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc06c460 con 0x7f9efc146460 2023-11-18T20:43:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9efaffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9efc146840 0x7f9efc06baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc146460 0x7f9efc0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9efaffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9efc146840 0x7f9efc06baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:48444/0 (socket says 172.21.15.60:48444) 2023-11-18T20:43:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc146460 0x7f9efc0b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37530/0 (socket says 172.21.15.60:37530) 2023-11-18T20:43:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9efaffd640 1 -- 172.21.15.60:0/3356328146 learned_addr learned my addr 172.21.15.60:0/3356328146 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9efaffd640 1 -- 172.21.15.60:0/3356328146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc146460 msgr2=0x7f9efc0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9efaffd640 1 --2- 172.21.15.60:0/3356328146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc146460 0x7f9efc0b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9efaffd640 1 -- 172.21.15.60:0/3356328146 >> v1:172.21.15.100:6789/0 conn(0x7f9efc06eab0 legacy=0x7f9efc160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.893+0000 7f9efaffd640 1 -- 172.21.15.60:0/3356328146 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc06c740 con 0x7f9efc146840 2023-11-18T20:43:38.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.894+0000 7f9efaffd640 1 --2- 172.21.15.60:0/3356328146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9efc146840 0x7f9efc06baf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009870 tx=0x7f9ef002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be51fbf7f7a58e19 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.894+0000 7f9efa7fc640 1 -- 172.21.15.60:0/3356328146 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0030840 con 0x7f9efc146840 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.894+0000 7f9efa7fc640 1 -- 172.21.15.60:0/3356328146 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef00309a0 con 0x7f9efc146840 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.894+0000 7f9efa7fc640 1 -- 172.21.15.60:0/3356328146 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0030d30 con 0x7f9efc146840 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.895+0000 7f9f01bfb640 1 -- 172.21.15.60:0/3356328146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9efc146840 msgr2=0x7f9efc06baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.895+0000 7f9f01bfb640 1 --2- 172.21.15.60:0/3356328146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9efc146840 0x7f9efc06baf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009870 tx=0x7f9ef002b980 comp rx=0 tx=0).stop 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.895+0000 7f9f01bfb640 1 -- 172.21.15.60:0/3356328146 shutdown_connections 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.895+0000 7f9f01bfb640 1 --2- 172.21.15.60:0/3356328146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9efc146840 0x7f9efc06baf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.895+0000 7f9f01bfb640 1 --2- 172.21.15.60:0/3356328146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc146460 0x7f9efc0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.895+0000 7f9f01bfb640 1 -- 172.21.15.60:0/3356328146 >> 172.21.15.60:0/3356328146 conn(0x7f9efc06a6b0 msgr2=0x7f9efc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.895+0000 7f9f01bfb640 1 -- 172.21.15.60:0/3356328146 shutdown_connections 2023-11-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.895+0000 7f9f01bfb640 1 -- 172.21.15.60:0/3356328146 wait complete. 2023-11-18T20:43:38.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.896+0000 7f9f01bfb640 1 Processor -- start 2023-11-18T20:43:38.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.896+0000 7f9f01bfb640 1 -- start start 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9f01bfb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc06eab0 0x7f9efc130f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9f01bfb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9efc146460 0x7f9efc12f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9f01bfb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9efc146840 0x7f9efc15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9f01bfb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc12fef0 con 0x7f9efc146460 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9f01bfb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc1301d0 con 0x7f9efc146840 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9f01bfb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc1304b0 con 0x7f9efc06eab0 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9efaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9efc146460 0x7f9efc12f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9efaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9efc146460 0x7f9efc12f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44796/0 (socket says 172.21.15.60:44796) 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9efaffd640 1 -- 172.21.15.60:0/1753439070 learned_addr learned my addr 172.21.15.60:0/1753439070 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9efb7fe640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc06eab0 0x7f9efc130f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9efbfff640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9efc146840 0x7f9efc15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9efaffd640 1 -- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc06eab0 msgr2=0x7f9efc130f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9efaffd640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc06eab0 0x7f9efc130f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9efaffd640 1 -- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9efc146840 msgr2=0x7f9efc15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9efaffd640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9efc146840 0x7f9efc15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.897+0000 7f9efaffd640 1 -- 172.21.15.60:0/1753439070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00094d0 con 0x7f9efc146460 2023-11-18T20:43:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.898+0000 7f9efaffd640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9efc146460 0x7f9efc12f5f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0000c00 tx=0x7f9ef0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.898+0000 7f9ee3fff640 1 -- 172.21.15.60:0/1753439070 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0039a90 con 0x7f9efc146460 2023-11-18T20:43:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.898+0000 7f9ee3fff640 1 -- 172.21.15.60:0/1753439070 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef0039bf0 con 0x7f9efc146460 2023-11-18T20:43:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.898+0000 7f9f01bfb640 1 -- 172.21.15.60:0/1753439070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc130790 con 0x7f9efc146460 2023-11-18T20:43:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.898+0000 7f9ee3fff640 1 -- 172.21.15.60:0/1753439070 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0043410 con 0x7f9efc146460 2023-11-18T20:43:38.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.898+0000 7f9f01bfb640 1 -- 172.21.15.60:0/1753439070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc12fb30 con 0x7f9efc146460 2023-11-18T20:43:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.899+0000 7f9f01bfb640 1 -- 172.21.15.60:0/1753439070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9efc12fcc0 con 0x7f9efc146460 2023-11-18T20:43:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.899+0000 7f9ee3fff640 1 -- 172.21.15.60:0/1753439070 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ef002f390 con 0x7f9efc146460 2023-11-18T20:43:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.900+0000 7f9ee3fff640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ee806ae40 0x7f9ee806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.900+0000 7f9ee3fff640 1 -- 172.21.15.60:0/1753439070 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ef00ab670 con 0x7f9efc146460 2023-11-18T20:43:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.900+0000 7f9efb7fe640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ee806ae40 0x7f9ee806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.901+0000 7f9efb7fe640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ee806ae40 0x7f9ee806d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009b40 tx=0x7f9eec009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:38.904+0000 7f9ee3fff640 1 -- 172.21.15.60:0/1753439070 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ef00768f0 con 0x7f9efc146460 2023-11-18T20:43:38.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:43:38.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:43:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.117+0000 7f9f01bfb640 1 -- 172.21.15.60:0/1753439070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9efc130cb0 con 0x7f9efc146460 2023-11-18T20:43:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.117+0000 7f9ee3fff640 1 -- 172.21.15.60:0/1753439070 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9efc130cb0 con 0x7f9efc146460 2023-11-18T20:43:39.119 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-18T20:43:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 -- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ee806ae40 msgr2=0x7f9ee806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ee806ae40 0x7f9ee806d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009b40 tx=0x7f9eec009190 comp rx=0 tx=0).stop 2023-11-18T20:43:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 -- 172.21.15.60:0/1753439070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9efc146460 msgr2=0x7f9efc12f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9efc146460 0x7f9efc12f5f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0000c00 tx=0x7f9ef0004730 comp rx=0 tx=0).stop 2023-11-18T20:43:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 -- 172.21.15.60:0/1753439070 shutdown_connections 2023-11-18T20:43:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ee806ae40 0x7f9ee806d300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9efc146840 0x7f9efc15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:39.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9efc146460 0x7f9efc12f5f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:39.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 --2- 172.21.15.60:0/1753439070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9efc06eab0 0x7f9efc130f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:39.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 -- 172.21.15.60:0/1753439070 >> 172.21.15.60:0/1753439070 conn(0x7f9efc06a6b0 msgr2=0x7f9efc0b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:39.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 -- 172.21.15.60:0/1753439070 shutdown_connections 2023-11-18T20:43:39.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:39.120+0000 7f9f01bfb640 1 -- 172.21.15.60:0/1753439070 wait complete. 2023-11-18T20:43:39.132 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-18T20:43:39.132 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-18T20:43:39.132 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-11-18T20:43:39.132 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-18T20:43:39.132 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:39.132 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-18T20:43:39.132 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-18T20:43:39.132 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-18T20:43:39.132 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:43:39.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:39.133 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-11-18T20:43:39.148 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:43:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:43:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:43:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-11-18T20:43:39.148 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:39.148 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42637sec preferred_lft 42637sec 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2023-11-18T20:43:39.149 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::d896:cfff:fef3:88dc/64 scope link 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0a:c3:0c:c2:e6:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8c3:cff:fec2:e672/64 scope link 2023-11-18T20:43:39.150 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:39.151 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:43:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:43:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::40c:deff:fe9d:c1f6/64 scope link 2023-11-18T20:43:39.151 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:39.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:39.151 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-18T20:43:39.217 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-11-18T20:43:39.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:39.219 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-11-18T20:43:39.275 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-11-18T20:43:39.275 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-18T20:43:39.276 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-18T20:43:39.276 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-18T20:43:39.276 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-18T20:43:39.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:39.276 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:43:39.276 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:43:39.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-18T20:43:39.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-18T20:43:39.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-18T20:43:39.277 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:43:39.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:39.382 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-11-18T20:43:39.397 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-18T20:43:39.397 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-18T20:43:39.397 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:43:39.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:39.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-11-18T20:43:39.468 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-18T20:43:39.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:43:39.468 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:39.523 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:43:39.523 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:43:39.523 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:39.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:43:39.649 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:43:39.705 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:43:39.705 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:43:39.705 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:43:39.705 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:43:39.705 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:43:39.706 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:43:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:43:39.708 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:43:39.708 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:43:39.708 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:43:39.708 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:43:39.708 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:43:39.708 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:43:39.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:39.708 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:43:39.760 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-18T20:43:39.760 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-18T20:43:39.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:43:39.833 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:43:39.889 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:43:39.889 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:43:39.889 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:43:39.889 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:43:39.889 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:43:39.889 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:43:39.889 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:43:39.890 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:43:39.891 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:43:39.892 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:43:39.892 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:43:39.892 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:43:39.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:39.892 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:43:39.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-11-18T20:43:39.895+0000 7f362a26d480 -1 init, newargv = 0x555f0a8501a0 newargc=15 2023-11-18T20:43:39.897 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[101199]: starting ceph client 2023-11-18T20:43:39.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[101199]: starting fuse 2023-11-18T20:43:39.912 INFO:teuthology.orchestra.run.smithi060.stdout:41 2023-11-18T20:43:39.912 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-18T20:43:39.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2023-11-18T20:43:39.912 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-11-18T20:43:39.913 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-11-18T20:43:39.914 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-11-18T20:43:40.037 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.101199.asok 2023-11-18T20:43:40.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:43:40.048 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.101199.asok 2023-11-18T20:43:40.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:40.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.101199.asok status 2023-11-18T20:43:40.189 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:43:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-11-18T20:43:40.189 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe", 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-7418-gb0f45a51 (b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe) reef (dev)", 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "101199", 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5436, 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-11-18T20:43:40.190 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 5436 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 361415357 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:43:40.191 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 361415357 2023-11-18T20:43:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.5436 192.168.0.1:0/361415357", 2023-11-18T20:43:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/361415357", 2023-11-18T20:43:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-11-18T20:43:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 13, 2023-11-18T20:43:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 34, 2023-11-18T20:43:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-11-18T20:43:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-11-18T20:43:40.192 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-11-18T20:43:40.192 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:43:40.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:40.198 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-11-18T20:43:40.198 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:40.213 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-11-18T20:43:40.213 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:40.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:40.213 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:40.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2023-11-18T20:43:40.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.432+0000 7f84f8483640 1 Processor -- start 2023-11-18T20:43:40.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.433+0000 7f84f8483640 1 -- start start 2023-11-18T20:43:40.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f8483640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 0x7f84f0141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f8483640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84f0144a00 0x7f84f006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f8483640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f00b2ef0 con 0x7f84f0144440 2023-11-18T20:43:40.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f8483640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f00b3b70 con 0x7f84f0144a00 2023-11-18T20:43:40.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f8483640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00b31d0 con 0x7f84f01415f0 2023-11-18T20:43:40.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f5a1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84f0144a00 0x7f84f006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f5a1e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84f0144a00 0x7f84f006b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44810/0 (socket says 172.21.15.60:44810) 2023-11-18T20:43:40.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f5a1e640 1 -- 172.21.15.60:0/2315968060 learned_addr learned my addr 172.21.15.60:0/2315968060 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:40.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f621f640 1 --2- 172.21.15.60:0/2315968060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 0x7f84f0141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f5a1e640 1 -- 172.21.15.60:0/2315968060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 msgr2=0x7f84f0141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:40.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f5a1e640 1 --2- 172.21.15.60:0/2315968060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 0x7f84f0141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f5a1e640 1 -- 172.21.15.60:0/2315968060 >> v1:172.21.15.100:6789/0 conn(0x7f84f0144440 legacy=0x7f84f00b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:40.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.434+0000 7f84f5a1e640 1 -- 172.21.15.60:0/2315968060 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f00b34b0 con 0x7f84f0144a00 2023-11-18T20:43:40.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.435+0000 7f84f5a1e640 1 --2- 172.21.15.60:0/2315968060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144a00 0x7f84f006b4e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009870 tx=0x7f84e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d20962d7846075d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:40.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.435+0000 7f84f521d640 1 -- 172.21.15.60:0/2315968060 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0030840 con 0x7f84f0144a00 2023-11-18T20:43:40.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.435+0000 7f84f521d640 1 -- 172.21.15.60:0/2315968060 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e00309a0 con 0x7f84f0144a00 2023-11-18T20:43:40.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.435+0000 7f84f521d640 1 -- 172.21.15.60:0/2315968060 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0030d30 con 0x7f84f0144a00 2023-11-18T20:43:40.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.435+0000 7f84f8483640 1 -- 172.21.15.60:0/2315968060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144a00 msgr2=0x7f84f006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:40.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.435+0000 7f84f8483640 1 --2- 172.21.15.60:0/2315968060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144a00 0x7f84f006b4e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009870 tx=0x7f84e002b980 comp rx=0 tx=0).stop 2023-11-18T20:43:40.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.436+0000 7f84f8483640 1 -- 172.21.15.60:0/2315968060 shutdown_connections 2023-11-18T20:43:40.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.436+0000 7f84f8483640 1 --2- 172.21.15.60:0/2315968060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144a00 0x7f84f006b4e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.436+0000 7f84f8483640 1 --2- 172.21.15.60:0/2315968060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 0x7f84f0141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.436+0000 7f84f8483640 1 -- 172.21.15.60:0/2315968060 >> 172.21.15.60:0/2315968060 conn(0x7f84f0070e70 msgr2=0x7f84f0071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:40.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.436+0000 7f84f8483640 1 -- 172.21.15.60:0/2315968060 shutdown_connections 2023-11-18T20:43:40.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.436+0000 7f84f8483640 1 -- 172.21.15.60:0/2315968060 wait complete. 2023-11-18T20:43:40.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.437+0000 7f84f8483640 1 Processor -- start 2023-11-18T20:43:40.439 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.437+0000 7f84f8483640 1 -- start start 2023-11-18T20:43:40.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.437+0000 7f84f8483640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 0x7f84f0135ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f8483640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144440 0x7f84f0154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f8483640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84f0144a00 0x7f84f01f1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f8483640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f0154ee0 con 0x7f84f0144440 2023-11-18T20:43:40.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f8483640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01551c0 con 0x7f84f0144a00 2023-11-18T20:43:40.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f8483640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01554a0 con 0x7f84f01415f0 2023-11-18T20:43:40.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f5a1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144440 0x7f84f0154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f5a1e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144440 0x7f84f0154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44812/0 (socket says 172.21.15.60:44812) 2023-11-18T20:43:40.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f5a1e640 1 -- 172.21.15.60:0/1794493632 learned_addr learned my addr 172.21.15.60:0/1794493632 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:40.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f621f640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 0x7f84f0135ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f6a20640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84f0144a00 0x7f84f01f1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f5a1e640 1 -- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 msgr2=0x7f84f0135ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f5a1e640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 0x7f84f0135ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f5a1e640 1 -- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84f0144a00 msgr2=0x7f84f01f1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f5a1e640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84f0144a00 0x7f84f01f1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f5a1e640 1 -- 172.21.15.60:0/1794493632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e00094d0 con 0x7f84f0144440 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f621f640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 0x7f84f0135ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f5a1e640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144440 0x7f84f0154420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f84e0000c00 tx=0x7f84e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84e6ffd640 1 -- 172.21.15.60:0/1794493632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0039a90 con 0x7f84f0144440 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84e6ffd640 1 -- 172.21.15.60:0/1794493632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e0039bf0 con 0x7f84f0144440 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f8483640 1 -- 172.21.15.60:0/1794493632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f0155780 con 0x7f84f0144440 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84e6ffd640 1 -- 172.21.15.60:0/1794493632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0043410 con 0x7f84f0144440 2023-11-18T20:43:40.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.438+0000 7f84f8483640 1 -- 172.21.15.60:0/1794493632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f01405d0 con 0x7f84f0144440 2023-11-18T20:43:40.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.439+0000 7f84f8483640 1 -- 172.21.15.60:0/1794493632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f0155ca0 con 0x7f84f0144440 2023-11-18T20:43:40.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.440+0000 7f84e6ffd640 1 -- 172.21.15.60:0/1794493632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84e004c3b0 con 0x7f84f0144440 2023-11-18T20:43:40.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.440+0000 7f84e6ffd640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84d406ae40 0x7f84d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.440+0000 7f84e6ffd640 1 -- 172.21.15.60:0/1794493632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84e00aba70 con 0x7f84f0144440 2023-11-18T20:43:40.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.440+0000 7f84f621f640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84d406ae40 0x7f84d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.441+0000 7f84f621f640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84d406ae40 0x7f84d406d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009b90 tx=0x7f84ec008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:40.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.444+0000 7f84e6ffd640 1 -- 172.21.15.60:0/1794493632 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84f0155ca0 con 0x7f84f0144440 2023-11-18T20:43:40.620 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.619+0000 7f84f8483640 1 -- 172.21.15.60:0/1794493632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7f84f0155ca0 con 0x7f84f0144440 2023-11-18T20:43:40.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.806+0000 7f84e6ffd640 1 -- 172.21.15.60:0/1794493632 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v35) v1 ==== 124+0+0 (secure 0 0 0) 0x7f84f0155ca0 con 0x7f84f0144440 2023-11-18T20:43:40.807 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'data_new' created 2023-11-18T20:43:40.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 -- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84d406ae40 msgr2=0x7f84d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84d406ae40 0x7f84d406d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009b90 tx=0x7f84ec008040 comp rx=0 tx=0).stop 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 -- 172.21.15.60:0/1794493632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144440 msgr2=0x7f84f0154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144440 0x7f84f0154420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f84e0000c00 tx=0x7f84e0004730 comp rx=0 tx=0).stop 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 -- 172.21.15.60:0/1794493632 shutdown_connections 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f84d406ae40 0x7f84d406d300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84f0144a00 0x7f84f01f1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84f0144440 0x7f84f0154420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 --2- 172.21.15.60:0/1794493632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84f01415f0 0x7f84f0135ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 -- 172.21.15.60:0/1794493632 >> 172.21.15.60:0/1794493632 conn(0x7f84f0070e70 msgr2=0x7f84f0142f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 -- 172.21.15.60:0/1794493632 shutdown_connections 2023-11-18T20:43:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.808+0000 7f84f8483640 1 -- 172.21.15.60:0/1794493632 wait complete. 2023-11-18T20:43:40.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2023-11-18T20:43:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.932+0000 7f5c9053a640 1 Processor -- start 2023-11-18T20:43:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.932+0000 7f5c9053a640 1 -- start start 2023-11-18T20:43:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c9053a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c88142270 0x7f5c881426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c9053a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c88161d20 con 0x7f5c8806ea50 2023-11-18T20:43:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c9053a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8815e760 con 0x7f5c88160fb0 2023-11-18T20:43:40.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c9053a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c880d1ee0 con 0x7f5c88142270 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c8ead7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c88142270 0x7f5c881426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c8e2d6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5c8806ea50 0x7f5c8806ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37474/0 (socket says 172.21.15.60:37474) 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c8e2d6640 1 -- 172.21.15.60:0/2196660873 learned_addr learned my addr 172.21.15.60:0/2196660873 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c8ead7640 1 -- 172.21.15.60:0/2196660873 >> v1:172.21.15.100:6789/0 conn(0x7f5c88160fb0 legacy=0x7f5c88161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c8ead7640 1 -- 172.21.15.60:0/2196660873 >> v1:172.21.15.60:6789/0 conn(0x7f5c8806ea50 legacy=0x7f5c8806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c8d2d4640 1 -- 172.21.15.60:0/2196660873 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716527190 0 0) 0x7f5c88161d20 con 0x7f5c8806ea50 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.933+0000 7f5c8ead7640 1 -- 172.21.15.60:0/2196660873 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8815d9a0 con 0x7f5c88142270 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.934+0000 7f5c8ead7640 1 --2- 172.21.15.60:0/2196660873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 0x7f5c881426c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009d90 tx=0x7f5c8002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce92fc28edab1ce server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.934+0000 7f5c8d2d4640 1 -- 172.21.15.60:0/2196660873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8002cc60 con 0x7f5c88142270 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.934+0000 7f5c8d2d4640 1 -- 172.21.15.60:0/2196660873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8002cdc0 con 0x7f5c88142270 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.934+0000 7f5c8d2d4640 1 -- 172.21.15.60:0/2196660873 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c80034660 con 0x7f5c88142270 2023-11-18T20:43:40.938 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.935+0000 7f5c9053a640 1 -- 172.21.15.60:0/2196660873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 msgr2=0x7f5c881426c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.935+0000 7f5c9053a640 1 --2- 172.21.15.60:0/2196660873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 0x7f5c881426c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009d90 tx=0x7f5c8002beb0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.935+0000 7f5c9053a640 1 -- 172.21.15.60:0/2196660873 shutdown_connections 2023-11-18T20:43:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.935+0000 7f5c9053a640 1 --2- 172.21.15.60:0/2196660873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 0x7f5c881426c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.935+0000 7f5c9053a640 1 -- 172.21.15.60:0/2196660873 >> 172.21.15.60:0/2196660873 conn(0x7f5c8806a6b0 msgr2=0x7f5c8806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.935+0000 7f5c9053a640 1 -- 172.21.15.60:0/2196660873 shutdown_connections 2023-11-18T20:43:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.935+0000 7f5c9053a640 1 -- 172.21.15.60:0/2196660873 wait complete. 2023-11-18T20:43:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.936+0000 7f5c9053a640 1 Processor -- start 2023-11-18T20:43:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.936+0000 7f5c9053a640 1 -- start start 2023-11-18T20:43:40.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c9053a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c8806ea50 0x7f5c88159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c9053a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 0x7f5c880cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c9053a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c88160fb0 0x7f5c881357b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c9053a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881f33b0 con 0x7f5c88142270 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c9053a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815acc0 con 0x7f5c88160fb0 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c9053a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815a420 con 0x7f5c8806ea50 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8dad5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 0x7f5c880cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8dad5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 0x7f5c880cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57304/0 (socket says 172.21.15.60:57304) 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8dad5640 1 -- 172.21.15.60:0/1662322911 learned_addr learned my addr 172.21.15.60:0/1662322911 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8e2d6640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c8806ea50 0x7f5c88159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8ead7640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c88160fb0 0x7f5c881357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8dad5640 1 -- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c8806ea50 msgr2=0x7f5c88159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8dad5640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c8806ea50 0x7f5c88159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8dad5640 1 -- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c88160fb0 msgr2=0x7f5c881357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8dad5640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c88160fb0 0x7f5c881357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8dad5640 1 -- 172.21.15.60:0/1662322911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c80009a70 con 0x7f5c88142270 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8e2d6640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c8806ea50 0x7f5c88159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c8dad5640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 0x7f5c880cb7d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c450 tx=0x7f5c7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c7effd640 1 -- 172.21.15.60:0/1662322911 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c780052d0 con 0x7f5c88142270 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c7effd640 1 -- 172.21.15.60:0/1662322911 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c78005430 con 0x7f5c88142270 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c9053a640 1 -- 172.21.15.60:0/1662322911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8815a700 con 0x7f5c88142270 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.937+0000 7f5c7effd640 1 -- 172.21.15.60:0/1662322911 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78012870 con 0x7f5c88142270 2023-11-18T20:43:40.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.938+0000 7f5c9053a640 1 -- 172.21.15.60:0/1662322911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c880cbd10 con 0x7f5c88142270 2023-11-18T20:43:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.938+0000 7f5c9053a640 1 -- 172.21.15.60:0/1662322911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c881f3120 con 0x7f5c88142270 2023-11-18T20:43:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.939+0000 7f5c7effd640 1 -- 172.21.15.60:0/1662322911 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c780129d0 con 0x7f5c88142270 2023-11-18T20:43:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.939+0000 7f5c7effd640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5c6c06ae40 0x7f5c6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.940+0000 7f5c7effd640 1 -- 172.21.15.60:0/1662322911 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6072+0+0 (secure 0 0 0) 0x7f5c780892c0 con 0x7f5c88142270 2023-11-18T20:43:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.940+0000 7f5c8e2d6640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5c6c06ae40 0x7f5c6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.940+0000 7f5c8e2d6640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5c6c06ae40 0x7f5c6c06d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c84007fd0 tx=0x7f5c84007dc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:40.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:40.944+0000 7f5c7effd640 1 -- 172.21.15.60:0/1662322911 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c78054200 con 0x7f5c88142270 2023-11-18T20:43:41.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.160+0000 7f5c9053a640 1 -- 172.21.15.60:0/1662322911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) v1 -- 0x7f5c881f34f0 con 0x7f5c88142270 2023-11-18T20:43:41.816 INFO:teuthology.orchestra.run.smithi060.stderr:added data pool 6 to fsmap 2023-11-18T20:43:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.813+0000 7f5c7effd640 1 -- 172.21.15.60:0/1662322911 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 6 to fsmap v14) v1 ==== 131+0+0 (secure 0 0 0) 0x7f5c881f34f0 con 0x7f5c88142270 2023-11-18T20:43:41.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.820+0000 7f5c9053a640 1 -- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5c6c06ae40 msgr2=0x7f5c6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:41.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.820+0000 7f5c9053a640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5c6c06ae40 0x7f5c6c06d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c84007fd0 tx=0x7f5c84007dc0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.821+0000 7f5c9053a640 1 -- 172.21.15.60:0/1662322911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 msgr2=0x7f5c880cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:41.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.821+0000 7f5c9053a640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 0x7f5c880cb7d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c450 tx=0x7f5c7800c910 comp rx=0 tx=0).stop 2023-11-18T20:43:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.822+0000 7f5c9053a640 1 -- 172.21.15.60:0/1662322911 shutdown_connections 2023-11-18T20:43:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.822+0000 7f5c9053a640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5c6c06ae40 0x7f5c6c06d300 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c84007fd0 tx=0x7f5c84007dc0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.822+0000 7f5c9053a640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c88160fb0 0x7f5c881357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.822+0000 7f5c9053a640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c88142270 0x7f5c880cb7d0 secure :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c450 tx=0x7f5c7800c910 comp rx=0 tx=0).stop 2023-11-18T20:43:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.822+0000 7f5c9053a640 1 --2- 172.21.15.60:0/1662322911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c8806ea50 0x7f5c88159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.822+0000 7f5c9053a640 1 -- 172.21.15.60:0/1662322911 >> 172.21.15.60:0/1662322911 conn(0x7f5c8806a6b0 msgr2=0x7f5c88140870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:41.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.823+0000 7f5c9053a640 1 -- 172.21.15.60:0/1662322911 shutdown_connections 2023-11-18T20:43:41.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.824+0000 7f5c9053a640 1 -- 172.21.15.60:0/1662322911 wait complete. 2023-11-18T20:43:41.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:43:41.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.949+0000 7f593a07c640 1 Processor -- start 2023-11-18T20:43:41.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.949+0000 7f593a07c640 1 -- start start 2023-11-18T20:43:41.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f593a07c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f593415d1d0 0x7f593406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:41.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f593a07c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340ad640 0x7f593406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:41.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f593a07c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f59340aeb80 0x7f5934160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:41.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f593a07c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59340adae0 con 0x7f593415d1d0 2023-11-18T20:43:41.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f593a07c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59340addc0 con 0x7f59340aeb80 2023-11-18T20:43:41.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f593a07c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59340ae0a0 con 0x7f59340ad640 2023-11-18T20:43:41.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f59337fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f593415d1d0 0x7f593406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f59337fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f593415d1d0 0x7f593406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57312/0 (socket says 172.21.15.60:57312) 2023-11-18T20:43:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f59337fe640 1 -- 172.21.15.60:0/3911284136 learned_addr learned my addr 172.21.15.60:0/3911284136 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f5932ffd640 1 --2- 172.21.15.60:0/3911284136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340ad640 0x7f593406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f5933fff640 1 --2- 172.21.15.60:0/3911284136 >> v2:172.21.15.100:3300/0 conn(0x7f59340aeb80 0x7f5934160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f59337fe640 1 -- 172.21.15.60:0/3911284136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340ad640 msgr2=0x7f593406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f59337fe640 1 --2- 172.21.15.60:0/3911284136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340ad640 0x7f593406ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f59337fe640 1 -- 172.21.15.60:0/3911284136 >> v2:172.21.15.100:3300/0 conn(0x7f59340aeb80 msgr2=0x7f5934160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f59337fe640 1 --2- 172.21.15.60:0/3911284136 >> v2:172.21.15.100:3300/0 conn(0x7f59340aeb80 0x7f5934160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f59337fe640 1 -- 172.21.15.60:0/3911284136 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340ae380 con 0x7f593415d1d0 2023-11-18T20:43:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f5932ffd640 1 --2- 172.21.15.60:0/3911284136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340ad640 0x7f593406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.950+0000 7f5933fff640 1 --2- 172.21.15.60:0/3911284136 >> v2:172.21.15.100:3300/0 conn(0x7f59340aeb80 0x7f5934160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.951+0000 7f59337fe640 1 --2- 172.21.15.60:0/3911284136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593415d1d0 0x7f593406b4e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f593406f410 tx=0x7f592402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86610024fb1f4f14 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.951+0000 7f59327fc640 1 -- 172.21.15.60:0/3911284136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924031870 con 0x7f593415d1d0 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.951+0000 7f59327fc640 1 -- 172.21.15.60:0/3911284136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59240319d0 con 0x7f593415d1d0 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.951+0000 7f59327fc640 1 -- 172.21.15.60:0/3911284136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924031d60 con 0x7f593415d1d0 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.952+0000 7f593a07c640 1 -- 172.21.15.60:0/3911284136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593415d1d0 msgr2=0x7f593406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.952+0000 7f593a07c640 1 --2- 172.21.15.60:0/3911284136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593415d1d0 0x7f593406b4e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f593406f410 tx=0x7f592402b910 comp rx=0 tx=0).stop 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.952+0000 7f593a07c640 1 -- 172.21.15.60:0/3911284136 shutdown_connections 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.952+0000 7f593a07c640 1 --2- 172.21.15.60:0/3911284136 >> v2:172.21.15.100:3300/0 conn(0x7f59340aeb80 0x7f5934160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.952+0000 7f593a07c640 1 --2- 172.21.15.60:0/3911284136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340ad640 0x7f593406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.952+0000 7f593a07c640 1 --2- 172.21.15.60:0/3911284136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f593415d1d0 0x7f593406b4e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.952+0000 7f593a07c640 1 -- 172.21.15.60:0/3911284136 >> 172.21.15.60:0/3911284136 conn(0x7f5934070e70 msgr2=0x7f5934071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:41.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.952+0000 7f593a07c640 1 -- 172.21.15.60:0/3911284136 shutdown_connections 2023-11-18T20:43:41.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.952+0000 7f593a07c640 1 -- 172.21.15.60:0/3911284136 wait complete. 2023-11-18T20:43:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.953+0000 7f593a07c640 1 Processor -- start 2023-11-18T20:43:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.953+0000 7f593a07c640 1 -- start start 2023-11-18T20:43:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f593a07c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59340ad640 0x7f59341f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f593a07c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340aeb80 0x7f59341f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f593a07c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f593415d1d0 0x7f593415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f593a07c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593415ca10 con 0x7f59340ad640 2023-11-18T20:43:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f593a07c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59341f6e60 con 0x7f593415d1d0 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f593a07c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593415c730 con 0x7f59340aeb80 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f59337fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59340ad640 0x7f59341f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f59337fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59340ad640 0x7f59341f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57324/0 (socket says 172.21.15.60:57324) 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f59337fe640 1 -- 172.21.15.60:0/1002753791 learned_addr learned my addr 172.21.15.60:0/1002753791 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f5932ffd640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340aeb80 0x7f59341f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f5933fff640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f593415d1d0 0x7f593415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f59337fe640 1 -- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340aeb80 msgr2=0x7f59341f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f59337fe640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340aeb80 0x7f59341f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f59337fe640 1 -- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f593415d1d0 msgr2=0x7f593415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f59337fe640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f593415d1d0 0x7f593415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.954+0000 7f59337fe640 1 -- 172.21.15.60:0/1002753791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59240094d0 con 0x7f59340ad640 2023-11-18T20:43:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.955+0000 7f59337fe640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59340ad640 0x7f59341f3b70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f592402b950 tx=0x7f59240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.955+0000 7f591bfff640 1 -- 172.21.15.60:0/1002753791 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924031860 con 0x7f59340ad640 2023-11-18T20:43:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.955+0000 7f591bfff640 1 -- 172.21.15.60:0/1002753791 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59240443f0 con 0x7f59340ad640 2023-11-18T20:43:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.955+0000 7f593a07c640 1 -- 172.21.15.60:0/1002753791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59340af120 con 0x7f59340ad640 2023-11-18T20:43:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.955+0000 7f591bfff640 1 -- 172.21.15.60:0/1002753791 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59240447a0 con 0x7f59340ad640 2023-11-18T20:43:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.955+0000 7f593a07c640 1 -- 172.21.15.60:0/1002753791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59340af6e0 con 0x7f59340ad640 2023-11-18T20:43:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.956+0000 7f593a07c640 1 -- 172.21.15.60:0/1002753791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5934069400 con 0x7f59340ad640 2023-11-18T20:43:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.956+0000 7f591bfff640 1 -- 172.21.15.60:0/1002753791 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f592404d450 con 0x7f59340ad640 2023-11-18T20:43:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.957+0000 7f591bfff640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f592006ae40 0x7f592006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.957+0000 7f591bfff640 1 -- 172.21.15.60:0/1002753791 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f592402d040 con 0x7f59340ad640 2023-11-18T20:43:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.957+0000 7f5932ffd640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f592006ae40 0x7f592006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.958+0000 7f5932ffd640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f592006ae40 0x7f592006d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59340cc610 tx=0x7f5928008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:41.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:41.961+0000 7f591bfff640 1 -- 172.21.15.60:0/1002753791 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5924076e30 con 0x7f59340ad640 2023-11-18T20:43:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.176+0000 7f593a07c640 1 -- 172.21.15.60:0/1002753791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5934151a20 con 0x7f59340ad640 2023-11-18T20:43:42.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.176+0000 7f591bfff640 1 -- 172.21.15.60:0/1002753791 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f5924035090 con 0x7f59340ad640 2023-11-18T20:43:42.178 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:42.178 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4814,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3885363174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3885363174},{"type":"v1","addr":"172.21.15.100:6837","nonce":3885363174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1102236552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1102236552},{"type":"v1","addr":"172.21.15.60:6835","nonce":1102236552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2680685354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2680685354},{"type":"v1","addr":"172.21.15.100:6836","nonce":2680685354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:43:33.146258+0000","modified":"2023-11-18T20:43:41.162003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5065":{"gid":5065,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3762312847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3762312847},{"type":"v1","addr":"172.21.15.60:6834","nonce":3762312847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-18T20:43:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 14 2023-11-18T20:43:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 -- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f592006ae40 msgr2=0x7f592006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f592006ae40 0x7f592006d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59340cc610 tx=0x7f5928008040 comp rx=0 tx=0).stop 2023-11-18T20:43:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 -- 172.21.15.60:0/1002753791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59340ad640 msgr2=0x7f59341f3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59340ad640 0x7f59341f3b70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f592402b950 tx=0x7f59240046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 -- 172.21.15.60:0/1002753791 shutdown_connections 2023-11-18T20:43:42.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f592006ae40 0x7f592006d300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f593415d1d0 0x7f593415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59340aeb80 0x7f59341f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 --2- 172.21.15.60:0/1002753791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59340ad640 0x7f59341f3b70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 -- 172.21.15.60:0/1002753791 >> 172.21.15.60:0/1002753791 conn(0x7f5934070e70 msgr2=0x7f593406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 -- 172.21.15.60:0/1002753791 shutdown_connections 2023-11-18T20:43:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.178+0000 7f593a07c640 1 -- 172.21.15.60:0/1002753791 wait complete. 2023-11-18T20:43:42.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:42.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.295+0000 7f4b01200640 1 Processor -- start 2023-11-18T20:43:42.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.295+0000 7f4b01200640 1 -- start start 2023-11-18T20:43:42.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4b01200640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4afc160880 0x7f4afc0d3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4b01200640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc0cf060 con 0x7f4afc0d1a50 2023-11-18T20:43:42.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4b01200640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc0d3e00 con 0x7f4afc160c60 2023-11-18T20:43:42.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4b01200640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc0ce730 con 0x7f4afc160880 2023-11-18T20:43:42.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4afb59e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4afc0d1a50 0x7f4afc0b2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37482/0 (socket says 172.21.15.60:37482) 2023-11-18T20:43:42.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4afb59e640 1 -- 172.21.15.60:0/3222675244 learned_addr learned my addr 172.21.15.60:0/3222675244 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:42.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4afad9d640 1 --2- 172.21.15.60:0/3222675244 >> v2:172.21.15.100:3300/0 conn(0x7f4afc160880 0x7f4afc0d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452183753 0 0) 0x7f4afc0cf060 con 0x7f4afc0d1a50 2023-11-18T20:43:42.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ae8003520 con 0x7f4afc0d1a50 2023-11-18T20:43:42.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2185210772 0 0) 0x7f4ae8003520 con 0x7f4afc0d1a50 2023-11-18T20:43:42.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 >> v2:172.21.15.100:3300/0 conn(0x7f4afc160880 msgr2=0x7f4afc0d3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4af9d9b640 1 --2- 172.21.15.60:0/3222675244 >> v2:172.21.15.100:3300/0 conn(0x7f4afc160880 0x7f4afc0d3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 >> v1:172.21.15.100:6789/0 conn(0x7f4afc160c60 legacy=0x7f4afc0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0cea10 con 0x7f4afc0d1a50 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f4aec0019f0 con 0x7f4afc0d1a50 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 >> v1:172.21.15.60:6789/0 conn(0x7f4afc0d1a50 legacy=0x7f4afc0b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.296+0000 7f4af9d9b640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4ae8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4af9d9b640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ae80047c0 0x7f4ae8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4af9d9b640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ae8005180 0x7f4ae800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0ce730 con 0x7f4ae8003970 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae800c120 con 0x7f4ae80047c0 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae800c2e0 con 0x7f4ae8005180 2023-11-18T20:43:42.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4afad9d640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4ae8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4afa59c640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ae80047c0 0x7f4ae8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4afb59e640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ae8005180 0x7f4ae800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4afad9d640 1 -- 172.21.15.60:0/3222675244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ae8005180 msgr2=0x7f4ae800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4afad9d640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ae8005180 0x7f4ae800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4afad9d640 1 -- 172.21.15.60:0/3222675244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ae80047c0 msgr2=0x7f4ae8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4afad9d640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ae80047c0 0x7f4ae8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4afad9d640 1 -- 172.21.15.60:0/3222675244 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8003520 con 0x7f4ae8003970 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4afb59e640 1 -- 172.21.15.60:0/3222675244 reap_dead start 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.297+0000 7f4afa59c640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ae80047c0 0x7f4ae8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4afad9d640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4ae8003d90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009ac0 tx=0x7f4ae80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c993e54af591ed server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af402c6b0 con 0x7f4ae8003970 2023-11-18T20:43:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af402c810 con 0x7f4ae8003970 2023-11-18T20:43:42.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4af9d9b640 1 -- 172.21.15.60:0/3222675244 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af402cba0 con 0x7f4ae8003970 2023-11-18T20:43:42.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4b01200640 1 -- 172.21.15.60:0/3222675244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 msgr2=0x7f4ae8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4b01200640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4ae8003d90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009ac0 tx=0x7f4ae80057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4b01200640 1 -- 172.21.15.60:0/3222675244 shutdown_connections 2023-11-18T20:43:42.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4b01200640 1 --2- 172.21.15.60:0/3222675244 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4ae8003d90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4b01200640 1 -- 172.21.15.60:0/3222675244 >> 172.21.15.60:0/3222675244 conn(0x7f4afc09c3b0 msgr2=0x7f4afc0d3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:42.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4b01200640 1 -- 172.21.15.60:0/3222675244 shutdown_connections 2023-11-18T20:43:42.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.298+0000 7f4b01200640 1 -- 172.21.15.60:0/3222675244 wait complete. 2023-11-18T20:43:42.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.300+0000 7f4b01200640 1 Processor -- start 2023-11-18T20:43:42.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.300+0000 7f4b01200640 1 -- start start 2023-11-18T20:43:42.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4b01200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4afc0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4b01200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc14f450 0x7f4afc0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4b01200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4afc0b4400 0x7f4afc0b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4b01200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0b4140 con 0x7f4ae8003970 2023-11-18T20:43:42.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4b01200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0b4dd0 con 0x7f4afc14f450 2023-11-18T20:43:42.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4b01200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0b2b60 con 0x7f4afc0b4400 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4afc0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4afc0b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57344/0 (socket says 172.21.15.60:57344) 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afad9d640 1 -- 172.21.15.60:0/3256710711 learned_addr learned my addr 172.21.15.60:0/3256710711 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afa59c640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc14f450 0x7f4afc0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afb59e640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4afc0b4400 0x7f4afc0b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afad9d640 1 -- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4afc0b4400 msgr2=0x7f4afc0b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afad9d640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4afc0b4400 0x7f4afc0b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afad9d640 1 -- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc14f450 msgr2=0x7f4afc0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afad9d640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc14f450 0x7f4afc0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afad9d640 1 -- 172.21.15.60:0/3256710711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0b2e40 con 0x7f4ae8003970 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4afad9d640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4afc0b3c00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4af402c250 tx=0x7f4af4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:42.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4af37fe640 1 -- 172.21.15.60:0/3256710711 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4003810 con 0x7f4ae8003970 2023-11-18T20:43:42.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4af37fe640 1 -- 172.21.15.60:0/3256710711 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af4003970 con 0x7f4ae8003970 2023-11-18T20:43:42.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.301+0000 7f4b01200640 1 -- 172.21.15.60:0/3256710711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc0b3120 con 0x7f4ae8003970 2023-11-18T20:43:42.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.302+0000 7f4af37fe640 1 -- 172.21.15.60:0/3256710711 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4003d00 con 0x7f4ae8003970 2023-11-18T20:43:42.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.302+0000 7f4b01200640 1 -- 172.21.15.60:0/3256710711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc0b36e0 con 0x7f4ae8003970 2023-11-18T20:43:42.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.302+0000 7f4b01200640 1 -- 172.21.15.60:0/3256710711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc067940 con 0x7f4ae8003970 2023-11-18T20:43:42.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.303+0000 7f4af37fe640 1 -- 172.21.15.60:0/3256710711 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4af4034440 con 0x7f4ae8003970 2023-11-18T20:43:42.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.303+0000 7f4af37fe640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4adc06ae40 0x7f4adc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.304+0000 7f4af37fe640 1 -- 172.21.15.60:0/3256710711 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f4af40ab940 con 0x7f4ae8003970 2023-11-18T20:43:42.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.304+0000 7f4afa59c640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4adc06ae40 0x7f4adc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.304+0000 7f4afa59c640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4adc06ae40 0x7f4adc06d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4000c00 tx=0x7f4ae4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:42.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.308+0000 7f4af37fe640 1 -- 172.21.15.60:0/3256710711 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4af40769a0 con 0x7f4ae8003970 2023-11-18T20:43:42.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.489+0000 7f4b01200640 1 -- 172.21.15.60:0/3256710711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4afc0b3870 con 0x7f4ae8003970 2023-11-18T20:43:42.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.490+0000 7f4af37fe640 1 -- 172.21.15.60:0/3256710711 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18349 (secure 0 0 0) 0x7f4af404c090 con 0x7f4ae8003970 2023-11-18T20:43:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:42.492 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":36,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:41.161791+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-11-18T20:43:40.650315+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 -- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4adc06ae40 msgr2=0x7f4adc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4adc06ae40 0x7f4adc06d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4000c00 tx=0x7f4ae4009040 comp rx=0 tx=0).stop 2023-11-18T20:43:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 -- 172.21.15.60:0/3256710711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 msgr2=0x7f4afc0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4afc0b3c00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4af402c250 tx=0x7f4af4004730 comp rx=0 tx=0).stop 2023-11-18T20:43:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 -- 172.21.15.60:0/3256710711 shutdown_connections 2023-11-18T20:43:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4adc06ae40 0x7f4adc06d300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4afc0b4400 0x7f4afc0b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4afc14f450 0x7f4afc0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 --2- 172.21.15.60:0/3256710711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ae8003970 0x7f4afc0b3c00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 -- 172.21.15.60:0/3256710711 >> 172.21.15.60:0/3256710711 conn(0x7f4afc09c3b0 msgr2=0x7f4afc15df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:42.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 -- 172.21.15.60:0/3256710711 shutdown_connections 2023-11-18T20:43:42.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.492+0000 7f4b01200640 1 -- 172.21.15.60:0/3256710711 wait complete. 2023-11-18T20:43:42.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:42.505 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2023-11-18T20:43:42.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:42.524 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2023-11-18T20:43:42.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2023-11-18T20:43:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.734+0000 7feccee7c640 1 Processor -- start 2023-11-18T20:43:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.734+0000 7feccee7c640 1 -- start start 2023-11-18T20:43:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.734+0000 7feccee7c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fecc80d3850 0x7fecc80b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.734+0000 7feccee7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 0x7fecc80cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.734+0000 7feccee7c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc80d00f0 con 0x7fecc815d420 2023-11-18T20:43:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.734+0000 7feccee7c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc80cff00 con 0x7fecc80d3850 2023-11-18T20:43:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.734+0000 7feccee7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc80d1420 con 0x7fecc80b43a0 2023-11-18T20:43:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7fecccc18640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fecc815d420 0x7fecc80d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37492/0 (socket says 172.21.15.60:37492) 2023-11-18T20:43:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7fecccc18640 1 -- 172.21.15.60:0/3909250302 learned_addr learned my addr 172.21.15.60:0/3909250302 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7feccd419640 1 --2- 172.21.15.60:0/3909250302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 0x7fecc80cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7fecc7fff640 1 --2- 172.21.15.60:0/3909250302 >> v2:172.21.15.100:3300/0 conn(0x7fecc80d3850 0x7fecc80b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7fecc77fe640 1 -- 172.21.15.60:0/3909250302 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728222090 0 0) 0x7fecc80d00f0 con 0x7fecc815d420 2023-11-18T20:43:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7fecc77fe640 1 -- 172.21.15.60:0/3909250302 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb8003520 con 0x7fecc815d420 2023-11-18T20:43:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7fecc77fe640 1 -- 172.21.15.60:0/3909250302 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3998920843 0 0) 0x7fecb8003520 con 0x7fecc815d420 2023-11-18T20:43:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7feccd419640 1 -- 172.21.15.60:0/3909250302 >> v2:172.21.15.100:3300/0 conn(0x7fecc80d3850 msgr2=0x7fecc80b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7feccd419640 1 --2- 172.21.15.60:0/3909250302 >> v2:172.21.15.100:3300/0 conn(0x7fecc80d3850 0x7fecc80b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7feccd419640 1 -- 172.21.15.60:0/3909250302 >> v1:172.21.15.60:6789/0 conn(0x7fecc815d420 legacy=0x7fecc80d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.735+0000 7feccd419640 1 -- 172.21.15.60:0/3909250302 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc80d03d0 con 0x7fecc80b43a0 2023-11-18T20:43:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.736+0000 7feccd419640 1 --2- 172.21.15.60:0/3909250302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 0x7fecc80cf9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009d90 tx=0x7fecc002bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a55bd12ac895ec97 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.736+0000 7fecc77fe640 1 -- 172.21.15.60:0/3909250302 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc001670 con 0x7fecc80b43a0 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.736+0000 7fecc77fe640 1 -- 172.21.15.60:0/3909250302 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc002c810 con 0x7fecc80b43a0 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.736+0000 7fecc77fe640 1 -- 172.21.15.60:0/3909250302 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc002cb80 con 0x7fecc80b43a0 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.737+0000 7feccee7c640 1 -- 172.21.15.60:0/3909250302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 msgr2=0x7fecc80cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.737+0000 7feccee7c640 1 --2- 172.21.15.60:0/3909250302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 0x7fecc80cf9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009d90 tx=0x7fecc002bce0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.737+0000 7feccee7c640 1 -- 172.21.15.60:0/3909250302 shutdown_connections 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.737+0000 7feccee7c640 1 --2- 172.21.15.60:0/3909250302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 0x7fecc80cf9c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.737+0000 7feccee7c640 1 --2- 172.21.15.60:0/3909250302 >> v2:172.21.15.100:3300/0 conn(0x7fecc80d3850 0x7fecc80b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.737+0000 7feccee7c640 1 -- 172.21.15.60:0/3909250302 >> 172.21.15.60:0/3909250302 conn(0x7fecc808b000 msgr2=0x7fecc809c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.737+0000 7feccee7c640 1 -- 172.21.15.60:0/3909250302 shutdown_connections 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.737+0000 7feccee7c640 1 -- 172.21.15.60:0/3909250302 wait complete. 2023-11-18T20:43:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.738+0000 7feccee7c640 1 Processor -- start 2023-11-18T20:43:42.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.738+0000 7feccee7c640 1 -- start start 2023-11-18T20:43:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccee7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 0x7fecc81f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccee7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecc80d3850 0x7fecc80cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccee7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc815d420 0x7fecc815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccee7c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc815e9a0 con 0x7fecc815d420 2023-11-18T20:43:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccee7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc81f2aa0 con 0x7fecc80d3850 2023-11-18T20:43:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccee7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc815ec40 con 0x7fecc80b43a0 2023-11-18T20:43:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccd419640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc815d420 0x7fecc815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccd419640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc815d420 0x7fecc815b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57354/0 (socket says 172.21.15.60:57354) 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccd419640 1 -- 172.21.15.60:0/3590769621 learned_addr learned my addr 172.21.15.60:0/3590769621 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7fecccc18640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 0x7fecc81f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7fecc7fff640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecc80d3850 0x7fecc80cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccd419640 1 -- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 msgr2=0x7fecc81f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccd419640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 0x7fecc81f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccd419640 1 -- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecc80d3850 msgr2=0x7fecc80cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccd419640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecc80d3850 0x7fecc80cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccd419640 1 -- 172.21.15.60:0/3590769621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0009a70 con 0x7fecc815d420 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7fecc7fff640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecc80d3850 0x7fecc80cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.739+0000 7feccd419640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc815d420 0x7fecc815b680 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fecc002c2f0 tx=0x7fecc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.740+0000 7fecc57fa640 1 -- 172.21.15.60:0/3590769621 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0007000 con 0x7fecc815d420 2023-11-18T20:43:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.740+0000 7fecc57fa640 1 -- 172.21.15.60:0/3590769621 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc002de10 con 0x7fecc815d420 2023-11-18T20:43:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.740+0000 7feccee7c640 1 -- 172.21.15.60:0/3590769621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc815ef20 con 0x7fecc815d420 2023-11-18T20:43:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.740+0000 7fecc57fa640 1 -- 172.21.15.60:0/3590769621 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0004490 con 0x7fecc815d420 2023-11-18T20:43:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.740+0000 7feccee7c640 1 -- 172.21.15.60:0/3590769621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc815f4e0 con 0x7fecc815d420 2023-11-18T20:43:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.741+0000 7feccee7c640 1 -- 172.21.15.60:0/3590769621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc8069400 con 0x7fecc815d420 2023-11-18T20:43:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.741+0000 7fecc57fa640 1 -- 172.21.15.60:0/3590769621 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecc00045f0 con 0x7fecc815d420 2023-11-18T20:43:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.742+0000 7fecc57fa640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fecac06ae40 0x7fecac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.742+0000 7fecc57fa640 1 -- 172.21.15.60:0/3590769621 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fecc00ac010 con 0x7fecc815d420 2023-11-18T20:43:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.742+0000 7fecccc18640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fecac06ae40 0x7fecac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:42.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.742+0000 7fecccc18640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fecac06ae40 0x7fecac06d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fecbc007a00 tx=0x7fecbc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.746+0000 7fecc57fa640 1 -- 172.21.15.60:0/3590769621 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fecc0077060 con 0x7fecc815d420 2023-11-18T20:43:42.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.994+0000 7feccee7c640 1 -- 172.21.15.60:0/3590769621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7fecc815d800 con 0x7fecc815d420 2023-11-18T20:43:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.996+0000 7fecc57fa640 1 -- 172.21.15.60:0/3590769621 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v14) v1 ==== 210+0+167 (secure 0 0 0) 0x7fecc007a680 con 0x7fecc815d420 2023-11-18T20:43:42.997 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-11-18T20:43:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA== 2023-11-18T20:43:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow rw" 2023-11-18T20:43:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-11-18T20:43:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2023-11-18T20:43:42.997 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-11-18T20:43:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 -- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fecac06ae40 msgr2=0x7fecac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fecac06ae40 0x7fecac06d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fecbc007a00 tx=0x7fecbc009040 comp rx=0 tx=0).stop 2023-11-18T20:43:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 -- 172.21.15.60:0/3590769621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc815d420 msgr2=0x7fecc815b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc815d420 0x7fecc815b680 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fecc002c2f0 tx=0x7fecc0004a40 comp rx=0 tx=0).stop 2023-11-18T20:43:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 -- 172.21.15.60:0/3590769621 shutdown_connections 2023-11-18T20:43:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fecac06ae40 0x7fecac06d300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fecc815d420 0x7fecc815b680 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fecc80d3850 0x7fecc80cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 --2- 172.21.15.60:0/3590769621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fecc80b43a0 0x7fecc81f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 -- 172.21.15.60:0/3590769621 >> 172.21.15.60:0/3590769621 conn(0x7fecc808b000 msgr2=0x7fecc815c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 -- 172.21.15.60:0/3590769621 shutdown_connections 2023-11-18T20:43:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:42.998+0000 7feccee7c640 1 -- 172.21.15.60:0/3590769621 wait complete. 2023-11-18T20:43:43.009 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:43:43.010 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:43.025 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:43.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.170+0000 7f895a821640 1 Processor -- start 2023-11-18T20:43:43.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.170+0000 7f895a821640 1 -- start start 2023-11-18T20:43:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.171+0000 7f895a821640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89541415f0 0x7f8954141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.171+0000 7f895a821640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8954144a00 0x7f895406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.171+0000 7f895a821640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540b2ef0 con 0x7f8954144440 2023-11-18T20:43:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.171+0000 7f895a821640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89540b3b70 con 0x7f8954144a00 2023-11-18T20:43:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.171+0000 7f895a821640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89540b31d0 con 0x7f89541415f0 2023-11-18T20:43:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f8953fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89541415f0 0x7f8954141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f89537fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8954144a00 0x7f895406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f8953fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89541415f0 0x7f8954141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:35078/0 (socket says 172.21.15.60:35078) 2023-11-18T20:43:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f8953fff640 1 -- 172.21.15.60:0/3351072766 learned_addr learned my addr 172.21.15.60:0/3351072766 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f89537fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8954144a00 0x7f895406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38664/0 (socket says 172.21.15.60:38664) 2023-11-18T20:43:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f8958dbe640 1 --1- 172.21.15.60:0/3351072766 >> v1:172.21.15.100:6789/0 conn(0x7f8954144440 0x7f89540b3f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:33814/0 (socket says 172.21.15.60:33814) 2023-11-18T20:43:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f8953fff640 1 -- 172.21.15.60:0/3351072766 >> v2:172.21.15.100:3300/0 conn(0x7f8954144a00 msgr2=0x7f895406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f8953fff640 1 --2- 172.21.15.60:0/3351072766 >> v2:172.21.15.100:3300/0 conn(0x7f8954144a00 0x7f895406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f8953fff640 1 -- 172.21.15.60:0/3351072766 >> v1:172.21.15.100:6789/0 conn(0x7f8954144440 legacy=0x7f89540b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f8953fff640 1 -- 172.21.15.60:0/3351072766 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540b34b0 con 0x7f89541415f0 2023-11-18T20:43:43.175 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.172+0000 7f89537fe640 1 --2- 172.21.15.60:0/3351072766 >> v2:172.21.15.100:3300/0 conn(0x7f8954144a00 0x7f895406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:43.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.173+0000 7f8953fff640 1 --2- 172.21.15.60:0/3351072766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89541415f0 0x7f8954141e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f893c009870 tx=0x7f893c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2451f5a7f066a31e server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:43.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.173+0000 7f8952ffd640 1 -- 172.21.15.60:0/3351072766 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c030840 con 0x7f89541415f0 2023-11-18T20:43:43.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.173+0000 7f8952ffd640 1 -- 172.21.15.60:0/3351072766 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c0309a0 con 0x7f89541415f0 2023-11-18T20:43:43.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.173+0000 7f8952ffd640 1 -- 172.21.15.60:0/3351072766 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c030d30 con 0x7f89541415f0 2023-11-18T20:43:43.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.173+0000 7f895a821640 1 -- 172.21.15.60:0/3351072766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89541415f0 msgr2=0x7f8954141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.176 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.173+0000 7f895a821640 1 --2- 172.21.15.60:0/3351072766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89541415f0 0x7f8954141e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f893c009870 tx=0x7f893c02b980 comp rx=0 tx=0).stop 2023-11-18T20:43:43.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.174+0000 7f895a821640 1 -- 172.21.15.60:0/3351072766 shutdown_connections 2023-11-18T20:43:43.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.174+0000 7f895a821640 1 --2- 172.21.15.60:0/3351072766 >> v2:172.21.15.100:3300/0 conn(0x7f8954144a00 0x7f895406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.174+0000 7f895a821640 1 --2- 172.21.15.60:0/3351072766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89541415f0 0x7f8954141e00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.174+0000 7f895a821640 1 -- 172.21.15.60:0/3351072766 >> 172.21.15.60:0/3351072766 conn(0x7f8954070e70 msgr2=0x7f8954071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:43.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.174+0000 7f895a821640 1 -- 172.21.15.60:0/3351072766 shutdown_connections 2023-11-18T20:43:43.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.174+0000 7f895a821640 1 -- 172.21.15.60:0/3351072766 wait complete. 2023-11-18T20:43:43.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.175+0000 7f895a821640 1 Processor -- start 2023-11-18T20:43:43.177 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.175+0000 7f895a821640 1 -- start start 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f895a821640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89541415f0 0x7f89541f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f895a821640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8954144440 0x7f89541560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f895a821640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8954144a00 0x7f895406de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f895a821640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89541389e0 con 0x7f89541415f0 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f895a821640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954158190 con 0x7f8954144440 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f895a821640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8954158470 con 0x7f8954144a00 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8953fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89541415f0 0x7f89541f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8953fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89541415f0 0x7f89541f2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57370/0 (socket says 172.21.15.60:57370) 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8953fff640 1 -- 172.21.15.60:0/3834579499 learned_addr learned my addr 172.21.15.60:0/3834579499 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8958dbe640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8954144a00 0x7f895406de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f89537fe640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8954144440 0x7f89541560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8953fff640 1 -- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8954144a00 msgr2=0x7f895406de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8953fff640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8954144a00 0x7f895406de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8953fff640 1 -- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8954144440 msgr2=0x7f89541560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8953fff640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8954144440 0x7f89541560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8953fff640 1 -- 172.21.15.60:0/3834579499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8944009400 con 0x7f89541415f0 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8953fff640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89541415f0 0x7f89541f2e20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f893c000c00 tx=0x7f893c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8950ff9640 1 -- 172.21.15.60:0/3834579499 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c039c90 con 0x7f89541415f0 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.176+0000 7f8950ff9640 1 -- 172.21.15.60:0/3834579499 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c039df0 con 0x7f89541415f0 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.177+0000 7f895a821640 1 -- 172.21.15.60:0/3834579499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c0094d0 con 0x7f89541415f0 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.177+0000 7f8950ff9640 1 -- 172.21.15.60:0/3834579499 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c043640 con 0x7f89541415f0 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.177+0000 7f895a821640 1 -- 172.21.15.60:0/3834579499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8954158a30 con 0x7f89541415f0 2023-11-18T20:43:43.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.177+0000 7f895a821640 1 -- 172.21.15.60:0/3834579499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8954066270 con 0x7f89541415f0 2023-11-18T20:43:43.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.178+0000 7f8950ff9640 1 -- 172.21.15.60:0/3834579499 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f893c0437a0 con 0x7f89541415f0 2023-11-18T20:43:43.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.178+0000 7f8950ff9640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f893406ae40 0x7f893406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.178+0000 7f8950ff9640 1 -- 172.21.15.60:0/3834579499 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f893c03e070 con 0x7f89541415f0 2023-11-18T20:43:43.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.178+0000 7f89537fe640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f893406ae40 0x7f893406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.179+0000 7f89537fe640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f893406ae40 0x7f893406d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f895406eda0 tx=0x7f8944008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:43.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.182+0000 7f8950ff9640 1 -- 172.21.15.60:0/3834579499 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f893c0776f0 con 0x7f89541415f0 2023-11-18T20:43:43.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.362+0000 7f895a821640 1 -- 172.21.15.60:0/3834579499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8954069400 con 0x7f89541415f0 2023-11-18T20:43:43.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.363+0000 7f8950ff9640 1 -- 172.21.15.60:0/3834579499 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18336 (secure 0 0 0) 0x7f893c07ad10 con 0x7f89541415f0 2023-11-18T20:43:43.365 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:43.365 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:42.814654+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-11-18T20:43:40.650315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 -- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f893406ae40 msgr2=0x7f893406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f893406ae40 0x7f893406d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f895406eda0 tx=0x7f8944008040 comp rx=0 tx=0).stop 2023-11-18T20:43:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 -- 172.21.15.60:0/3834579499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89541415f0 msgr2=0x7f89541f2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89541415f0 0x7f89541f2e20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f893c000c00 tx=0x7f893c004730 comp rx=0 tx=0).stop 2023-11-18T20:43:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 -- 172.21.15.60:0/3834579499 shutdown_connections 2023-11-18T20:43:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f893406ae40 0x7f893406d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8954144a00 0x7f895406de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8954144440 0x7f89541560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 --2- 172.21.15.60:0/3834579499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89541415f0 0x7f89541f2e20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 -- 172.21.15.60:0/3834579499 >> 172.21.15.60:0/3834579499 conn(0x7f8954070e70 msgr2=0x7f8954143b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 -- 172.21.15.60:0/3834579499 shutdown_connections 2023-11-18T20:43:43.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.366+0000 7f895a821640 1 -- 172.21.15.60:0/3834579499 wait complete. 2023-11-18T20:43:43.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:43:31.703619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:43:32.121610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-11-18T20:43:40.650315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:43.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:43.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.486+0000 7f17a37c0640 1 Processor -- start 2023-11-18T20:43:43.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.486+0000 7f17a37c0640 1 -- start start 2023-11-18T20:43:43.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.487+0000 7f17a37c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f179c06ea50 0x7f179c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.487+0000 7f17a37c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c0d0ed0 0x7f179c0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.487+0000 7f17a37c0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f179c161d20 con 0x7f179c160fb0 2023-11-18T20:43:43.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.487+0000 7f17a37c0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179c06f370 con 0x7f179c06ea50 2023-11-18T20:43:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.487+0000 7f17a37c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c15e760 con 0x7f179c0d0ed0 2023-11-18T20:43:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.487+0000 7f17a1d5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c0d0ed0 0x7f179c0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.487+0000 7f17a155c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f179c06ea50 0x7f179c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.487+0000 7f17a0d5b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f179c160fb0 0x7f179c161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:33816/0 (socket says 172.21.15.60:33816) 2023-11-18T20:43:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.487+0000 7f17a0d5b640 1 -- 172.21.15.60:0/2948836937 learned_addr learned my addr 172.21.15.60:0/2948836937 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:43.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.488+0000 7f17a155c640 1 -- 172.21.15.60:0/2948836937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c0d0ed0 msgr2=0x7f179c0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.488+0000 7f17a155c640 1 --2- 172.21.15.60:0/2948836937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c0d0ed0 0x7f179c0d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.488+0000 7f17a155c640 1 -- 172.21.15.60:0/2948836937 >> v1:172.21.15.100:6789/0 conn(0x7f179c160fb0 legacy=0x7f179c161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.488+0000 7f17a155c640 1 -- 172.21.15.60:0/2948836937 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f179c15d9a0 con 0x7f179c06ea50 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f17a155c640 1 --2- 172.21.15.60:0/2948836937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c06ea50 0x7f179c06ee30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17940097a0 tx=0x7f179402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adfd52c5ec97fee5 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f179bfff640 1 -- 172.21.15.60:0/2948836937 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794031830 con 0x7f179c06ea50 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f179bfff640 1 -- 172.21.15.60:0/2948836937 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1794031990 con 0x7f179c06ea50 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f179bfff640 1 -- 172.21.15.60:0/2948836937 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1794031d20 con 0x7f179c06ea50 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f17a37c0640 1 -- 172.21.15.60:0/2948836937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c06ea50 msgr2=0x7f179c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f17a37c0640 1 --2- 172.21.15.60:0/2948836937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c06ea50 0x7f179c06ee30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17940097a0 tx=0x7f179402b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f17a37c0640 1 -- 172.21.15.60:0/2948836937 shutdown_connections 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f17a37c0640 1 --2- 172.21.15.60:0/2948836937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c0d0ed0 0x7f179c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f17a37c0640 1 --2- 172.21.15.60:0/2948836937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c06ea50 0x7f179c06ee30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f17a37c0640 1 -- 172.21.15.60:0/2948836937 >> 172.21.15.60:0/2948836937 conn(0x7f179c06a6b0 msgr2=0x7f179c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:43.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f17a37c0640 1 -- 172.21.15.60:0/2948836937 shutdown_connections 2023-11-18T20:43:43.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.489+0000 7f17a37c0640 1 -- 172.21.15.60:0/2948836937 wait complete. 2023-11-18T20:43:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.491+0000 7f17a37c0640 1 Processor -- start 2023-11-18T20:43:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.491+0000 7f17a37c0640 1 -- start start 2023-11-18T20:43:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.491+0000 7f17a37c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c06ea50 0x7f179c132910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.491+0000 7f17a37c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c0d0ed0 0x7f179c1e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.491+0000 7f17a37c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c160fb0 0x7f179c0b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.491+0000 7f17a37c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c132e50 con 0x7f179c160fb0 2023-11-18T20:43:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.491+0000 7f17a37c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1e8000 con 0x7f179c0d0ed0 2023-11-18T20:43:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.491+0000 7f17a37c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f179c1e8490 con 0x7f179c06ea50 2023-11-18T20:43:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.491+0000 7f17a1d5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c160fb0 0x7f179c0b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a1d5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c160fb0 0x7f179c0b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57376/0 (socket says 172.21.15.60:57376) 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a1d5d640 1 -- 172.21.15.60:0/1101539570 learned_addr learned my addr 172.21.15.60:0/1101539570 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a155c640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c06ea50 0x7f179c132910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a0d5b640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c0d0ed0 0x7f179c1e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a1d5d640 1 -- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c06ea50 msgr2=0x7f179c132910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a1d5d640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c06ea50 0x7f179c132910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a1d5d640 1 -- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c0d0ed0 msgr2=0x7f179c1e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a1d5d640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c0d0ed0 0x7f179c1e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a1d5d640 1 -- 172.21.15.60:0/1101539570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1794009400 con 0x7f179c160fb0 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a1d5d640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c160fb0 0x7f179c0b1fd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f178c00efc0 tx=0x7f178c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f1799ffb640 1 -- 172.21.15.60:0/1101539570 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c009b10 con 0x7f179c160fb0 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f1799ffb640 1 -- 172.21.15.60:0/1101539570 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c00f040 con 0x7f179c160fb0 2023-11-18T20:43:43.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f1799ffb640 1 -- 172.21.15.60:0/1101539570 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c0146c0 con 0x7f179c160fb0 2023-11-18T20:43:43.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.492+0000 7f17a37c0640 1 -- 172.21.15.60:0/1101539570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179c1e7150 con 0x7f179c160fb0 2023-11-18T20:43:43.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.493+0000 7f17a37c0640 1 -- 172.21.15.60:0/1101539570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179c1e76d0 con 0x7f179c160fb0 2023-11-18T20:43:43.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.493+0000 7f17a37c0640 1 -- 172.21.15.60:0/1101539570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f179c1e8190 con 0x7f179c160fb0 2023-11-18T20:43:43.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.494+0000 7f1799ffb640 1 -- 172.21.15.60:0/1101539570 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f178c014820 con 0x7f179c160fb0 2023-11-18T20:43:43.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.494+0000 7f1799ffb640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f178806ae40 0x7f178806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.494+0000 7f1799ffb640 1 -- 172.21.15.60:0/1101539570 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f178c08a410 con 0x7f179c160fb0 2023-11-18T20:43:43.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.495+0000 7f17a155c640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f178806ae40 0x7f178806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.495+0000 7f17a155c640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f178806ae40 0x7f178806d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1794036000 tx=0x7f179403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:43.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.499+0000 7f1799ffb640 1 -- 172.21.15.60:0/1101539570 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f178c0553e0 con 0x7f179c160fb0 2023-11-18T20:43:43.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.674+0000 7f17a37c0640 1 -- 172.21.15.60:0/1101539570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f179c069400 con 0x7f179c160fb0 2023-11-18T20:43:43.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.674+0000 7f1799ffb640 1 -- 172.21.15.60:0/1101539570 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f178c058a00 con 0x7f179c160fb0 2023-11-18T20:43:43.676 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 -- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f178806ae40 msgr2=0x7f178806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f178806ae40 0x7f178806d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1794036000 tx=0x7f179403c040 comp rx=0 tx=0).stop 2023-11-18T20:43:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 -- 172.21.15.60:0/1101539570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c160fb0 msgr2=0x7f179c0b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c160fb0 0x7f179c0b1fd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f178c00efc0 tx=0x7f178c00c490 comp rx=0 tx=0).stop 2023-11-18T20:43:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 -- 172.21.15.60:0/1101539570 shutdown_connections 2023-11-18T20:43:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f178806ae40 0x7f178806d300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c160fb0 0x7f179c0b1fd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c0d0ed0 0x7f179c1e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 --2- 172.21.15.60:0/1101539570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c06ea50 0x7f179c132910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 -- 172.21.15.60:0/1101539570 >> 172.21.15.60:0/1101539570 conn(0x7f179c06a6b0 msgr2=0x7f179c15cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:43.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 -- 172.21.15.60:0/1101539570 shutdown_connections 2023-11-18T20:43:43.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.676+0000 7f17a37c0640 1 -- 172.21.15.60:0/1101539570 wait complete. 2023-11-18T20:43:43.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:43.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.796+0000 7efcf6aaf640 1 Processor -- start 2023-11-18T20:43:43.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.796+0000 7efcf6aaf640 1 -- start start 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.797+0000 7efcf6aaf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efcf0160f70 0x7efcf00cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.797+0000 7efcf6aaf640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efcf00ce480 0x7efcf00d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.797+0000 7efcf6aaf640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf01614e0 con 0x7efcf015d2d0 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.797+0000 7efcf6aaf640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf01617c0 con 0x7efcf0160f70 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.797+0000 7efcf6aaf640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf0161aa0 con 0x7efcf00ce480 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.797+0000 7efceffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efcf0160f70 0x7efcf00cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.797+0000 7efcf484b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efcf015d2d0 0x7efcf00b8b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37498/0 (socket says 172.21.15.60:37498) 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.797+0000 7efcf484b640 1 -- 172.21.15.60:0/2826945232 learned_addr learned my addr 172.21.15.60:0/2826945232 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.797+0000 7efcf504c640 1 --2- 172.21.15.60:0/2826945232 >> v2:172.21.15.100:3300/0 conn(0x7efcf00ce480 0x7efcf00d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.798+0000 7efceffff640 1 -- 172.21.15.60:0/2826945232 >> v2:172.21.15.100:3300/0 conn(0x7efcf00ce480 msgr2=0x7efcf00d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.798+0000 7efceffff640 1 --2- 172.21.15.60:0/2826945232 >> v2:172.21.15.100:3300/0 conn(0x7efcf00ce480 0x7efcf00d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.798+0000 7efceffff640 1 -- 172.21.15.60:0/2826945232 >> v1:172.21.15.60:6789/0 conn(0x7efcf015d2d0 legacy=0x7efcf00b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.798+0000 7efcef7fe640 1 -- 172.21.15.60:0/2826945232 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824541466 0 0) 0x7efcf01614e0 con 0x7efcf015d2d0 2023-11-18T20:43:43.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.798+0000 7efceffff640 1 -- 172.21.15.60:0/2826945232 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf0161d80 con 0x7efcf0160f70 2023-11-18T20:43:43.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.798+0000 7efcf504c640 1 --2- 172.21.15.60:0/2826945232 >> v2:172.21.15.100:3300/0 conn(0x7efcf00ce480 0x7efcf00d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:43.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.798+0000 7efceffff640 1 --2- 172.21.15.60:0/2826945232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf0160f70 0x7efcf00cdf40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efcf00cf630 tx=0x7efce002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442705902b219821 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:43.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.798+0000 7efcef7fe640 1 -- 172.21.15.60:0/2826945232 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce002cc80 con 0x7efcf0160f70 2023-11-18T20:43:43.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.798+0000 7efcef7fe640 1 -- 172.21.15.60:0/2826945232 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce002cde0 con 0x7efcf0160f70 2023-11-18T20:43:43.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.799+0000 7efcef7fe640 1 -- 172.21.15.60:0/2826945232 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0034660 con 0x7efcf0160f70 2023-11-18T20:43:43.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.799+0000 7efcf6aaf640 1 -- 172.21.15.60:0/2826945232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf0160f70 msgr2=0x7efcf00cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.799+0000 7efcf6aaf640 1 --2- 172.21.15.60:0/2826945232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf0160f70 0x7efcf00cdf40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efcf00cf630 tx=0x7efce002bed0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.799+0000 7efcf6aaf640 1 -- 172.21.15.60:0/2826945232 shutdown_connections 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.799+0000 7efcf6aaf640 1 --2- 172.21.15.60:0/2826945232 >> v2:172.21.15.100:3300/0 conn(0x7efcf00ce480 0x7efcf00d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.799+0000 7efcf6aaf640 1 --2- 172.21.15.60:0/2826945232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf0160f70 0x7efcf00cdf40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.799+0000 7efcf6aaf640 1 -- 172.21.15.60:0/2826945232 >> 172.21.15.60:0/2826945232 conn(0x7efcf0072230 msgr2=0x7efcf00cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.799+0000 7efcf6aaf640 1 -- 172.21.15.60:0/2826945232 shutdown_connections 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.799+0000 7efcf6aaf640 1 -- 172.21.15.60:0/2826945232 wait complete. 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.800+0000 7efcf6aaf640 1 Processor -- start 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.800+0000 7efcf6aaf640 1 -- start start 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efcf6aaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf00ce480 0x7efcf0157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efcf6aaf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf015d2d0 0x7efcf0158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efcf6aaf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcf0160f70 0x7efcf01f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efcf6aaf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf00b3b50 con 0x7efcf00ce480 2023-11-18T20:43:43.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efcf6aaf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf01ed630 con 0x7efcf0160f70 2023-11-18T20:43:43.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efcf6aaf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf00b2ed0 con 0x7efcf015d2d0 2023-11-18T20:43:43.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efcf484b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf00ce480 0x7efcf0157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efcf484b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf00ce480 0x7efcf0157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57396/0 (socket says 172.21.15.60:57396) 2023-11-18T20:43:43.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efcf484b640 1 -- 172.21.15.60:0/3262012906 learned_addr learned my addr 172.21.15.60:0/3262012906 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:43.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efceffff640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf015d2d0 0x7efcf0158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.801+0000 7efcf504c640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcf0160f70 0x7efcf01f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efcf484b640 1 -- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf015d2d0 msgr2=0x7efcf0158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efcf484b640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf015d2d0 0x7efcf0158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efcf484b640 1 -- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcf0160f70 msgr2=0x7efcf01f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efcf484b640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcf0160f70 0x7efcf01f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efcf484b640 1 -- 172.21.15.60:0/3262012906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce0009a70 con 0x7efcf00ce480 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efcf504c640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcf0160f70 0x7efcf01f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efceffff640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf015d2d0 0x7efcf0158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efcf484b640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf00ce480 0x7efcf0157de0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efce800c4d0 tx=0x7efce800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efced7fa640 1 -- 172.21.15.60:0/3262012906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce80074e0 con 0x7efcf00ce480 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efced7fa640 1 -- 172.21.15.60:0/3262012906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce8007660 con 0x7efcf00ce480 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efcf6aaf640 1 -- 172.21.15.60:0/3262012906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf00b31b0 con 0x7efcf00ce480 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efced7fa640 1 -- 172.21.15.60:0/3262012906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8004a20 con 0x7efcf00ce480 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.802+0000 7efcf6aaf640 1 -- 172.21.15.60:0/3262012906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf00b3770 con 0x7efcf00ce480 2023-11-18T20:43:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.803+0000 7efcf6aaf640 1 -- 172.21.15.60:0/3262012906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf0069400 con 0x7efcf00ce480 2023-11-18T20:43:43.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.803+0000 7efced7fa640 1 -- 172.21.15.60:0/3262012906 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efce8004b80 con 0x7efcf00ce480 2023-11-18T20:43:43.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.804+0000 7efced7fa640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efcd006ae40 0x7efcd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:43.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.804+0000 7efced7fa640 1 -- 172.21.15.60:0/3262012906 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7efce8088d10 con 0x7efcf00ce480 2023-11-18T20:43:43.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.804+0000 7efceffff640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efcd006ae40 0x7efcd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:43.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.805+0000 7efceffff640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efcd006ae40 0x7efcd006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efce0036000 tx=0x7efce0030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:43.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.808+0000 7efced7fa640 1 -- 172.21.15.60:0/3262012906 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efce8053c30 con 0x7efcf00ce480 2023-11-18T20:43:43.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.984+0000 7efcf6aaf640 1 -- 172.21.15.60:0/3262012906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efcf00d0aa0 con 0x7efcf00ce480 2023-11-18T20:43:43.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.984+0000 7efced7fa640 1 -- 172.21.15.60:0/3262012906 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7efce80180a0 con 0x7efcf00ce480 2023-11-18T20:43:43.986 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:43.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 -- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efcd006ae40 msgr2=0x7efcd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efcd006ae40 0x7efcd006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efce0036000 tx=0x7efce0030000 comp rx=0 tx=0).stop 2023-11-18T20:43:43.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 -- 172.21.15.60:0/3262012906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf00ce480 msgr2=0x7efcf0157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:43.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf00ce480 0x7efcf0157de0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efce800c4d0 tx=0x7efce800c990 comp rx=0 tx=0).stop 2023-11-18T20:43:43.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 -- 172.21.15.60:0/3262012906 shutdown_connections 2023-11-18T20:43:43.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efcd006ae40 0x7efcd006d300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcf0160f70 0x7efcf01f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf015d2d0 0x7efcf0158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 --2- 172.21.15.60:0/3262012906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf00ce480 0x7efcf0157de0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:43.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 -- 172.21.15.60:0/3262012906 >> 172.21.15.60:0/3262012906 conn(0x7efcf0072230 msgr2=0x7efcf00b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:43.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 -- 172.21.15.60:0/3262012906 shutdown_connections 2023-11-18T20:43:43.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:43.986+0000 7efcf6aaf640 1 -- 172.21.15.60:0/3262012906 wait complete. 2023-11-18T20:43:43.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:44.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.106+0000 7f5637501640 1 Processor -- start 2023-11-18T20:43:44.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.106+0000 7f5637501640 1 -- start start 2023-11-18T20:43:44.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.107+0000 7f5637501640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f563015d2d0 0x7f56300b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.107+0000 7f5637501640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56300d2040 0x7f56300b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.107+0000 7f5637501640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56300d1bb0 con 0x7f56300cda40 2023-11-18T20:43:44.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.107+0000 7f5637501640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563015d6b0 con 0x7f56300d2040 2023-11-18T20:43:44.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.107+0000 7f5637501640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56300ce700 con 0x7f563015d2d0 2023-11-18T20:43:44.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.107+0000 7f5634a9c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56300d2040 0x7f56300b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.107+0000 7f5634a9c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f56300d2040 0x7f56300b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57400/0 (socket says 172.21.15.60:57400) 2023-11-18T20:43:44.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.107+0000 7f5634a9c640 1 -- 172.21.15.60:0/14570911 learned_addr learned my addr 172.21.15.60:0/14570911 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:44.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.107+0000 7f563529d640 1 --2- 172.21.15.60:0/14570911 >> v2:172.21.15.100:3300/0 conn(0x7f563015d2d0 0x7f56300b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.108+0000 7f5634a9c640 1 -- 172.21.15.60:0/14570911 >> v2:172.21.15.100:3300/0 conn(0x7f563015d2d0 msgr2=0x7f56300b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:44.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.108+0000 7f5634a9c640 1 --2- 172.21.15.60:0/14570911 >> v2:172.21.15.100:3300/0 conn(0x7f563015d2d0 0x7f56300b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.108+0000 7f5634a9c640 1 -- 172.21.15.60:0/14570911 >> v1:172.21.15.100:6789/0 conn(0x7f56300cda40 legacy=0x7f56300cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:44.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.108+0000 7f5634a9c640 1 -- 172.21.15.60:0/14570911 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56300d1280 con 0x7f56300d2040 2023-11-18T20:43:44.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.108+0000 7f563529d640 1 --2- 172.21.15.60:0/14570911 >> v2:172.21.15.100:3300/0 conn(0x7f563015d2d0 0x7f56300b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:44.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.108+0000 7f5634a9c640 1 --2- 172.21.15.60:0/14570911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 0x7f56300b51a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f563006c5d0 tx=0x7f562002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdba0d6e2caa8f4f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:44.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.108+0000 7f5627fff640 1 -- 172.21.15.60:0/14570911 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620031890 con 0x7f56300d2040 2023-11-18T20:43:44.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.109+0000 7f5627fff640 1 -- 172.21.15.60:0/14570911 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56200319f0 con 0x7f56300d2040 2023-11-18T20:43:44.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.109+0000 7f5627fff640 1 -- 172.21.15.60:0/14570911 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620031d80 con 0x7f56300d2040 2023-11-18T20:43:44.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.109+0000 7f5637501640 1 -- 172.21.15.60:0/14570911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 msgr2=0x7f56300b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.109+0000 7f5637501640 1 --2- 172.21.15.60:0/14570911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 0x7f56300b51a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f563006c5d0 tx=0x7f562002b930 comp rx=0 tx=0).stop 2023-11-18T20:43:44.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.109+0000 7f5637501640 1 -- 172.21.15.60:0/14570911 shutdown_connections 2023-11-18T20:43:44.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.109+0000 7f5637501640 1 --2- 172.21.15.60:0/14570911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 0x7f56300b51a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.109+0000 7f5637501640 1 --2- 172.21.15.60:0/14570911 >> v2:172.21.15.100:3300/0 conn(0x7f563015d2d0 0x7f56300b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.109+0000 7f5637501640 1 -- 172.21.15.60:0/14570911 >> 172.21.15.60:0/14570911 conn(0x7f5630072050 msgr2=0x7f5630072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:44.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.109+0000 7f5637501640 1 -- 172.21.15.60:0/14570911 shutdown_connections 2023-11-18T20:43:44.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.109+0000 7f5637501640 1 -- 172.21.15.60:0/14570911 wait complete. 2023-11-18T20:43:44.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.110+0000 7f5637501640 1 Processor -- start 2023-11-18T20:43:44.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.110+0000 7f5637501640 1 -- start start 2023-11-18T20:43:44.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5637501640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56300cda40 0x7f56300ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5637501640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 0x7f56300b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5637501640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f563015d2d0 0x7f56300ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5637501640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56300c7ce0 con 0x7f56300d2040 2023-11-18T20:43:44.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5637501640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56300cd1c0 con 0x7f56300cda40 2023-11-18T20:43:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5637501640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56300c70d0 con 0x7f563015d2d0 2023-11-18T20:43:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5634a9c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 0x7f56300b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5634a9c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 0x7f56300b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57402/0 (socket says 172.21.15.60:57402) 2023-11-18T20:43:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5634a9c640 1 -- 172.21.15.60:0/2306557430 learned_addr learned my addr 172.21.15.60:0/2306557430 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f563529d640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56300cda40 0x7f56300ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5635a9e640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f563015d2d0 0x7f56300ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5634a9c640 1 -- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f563015d2d0 msgr2=0x7f56300ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5634a9c640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f563015d2d0 0x7f56300ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5634a9c640 1 -- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56300cda40 msgr2=0x7f56300ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5634a9c640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56300cda40 0x7f56300ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.111+0000 7f5634a9c640 1 -- 172.21.15.60:0/2306557430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200094d0 con 0x7f56300d2040 2023-11-18T20:43:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.112+0000 7f5634a9c640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 0x7f56300b3aa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f562002b970 tx=0x7f56200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.112+0000 7f5625ffb640 1 -- 172.21.15.60:0/2306557430 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620031860 con 0x7f56300d2040 2023-11-18T20:43:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.112+0000 7f5625ffb640 1 -- 172.21.15.60:0/2306557430 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56200443f0 con 0x7f56300d2040 2023-11-18T20:43:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.112+0000 7f5625ffb640 1 -- 172.21.15.60:0/2306557430 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56200447a0 con 0x7f56300d2040 2023-11-18T20:43:44.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.112+0000 7f5637501640 1 -- 172.21.15.60:0/2306557430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56300c73b0 con 0x7f56300d2040 2023-11-18T20:43:44.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.112+0000 7f5637501640 1 -- 172.21.15.60:0/2306557430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56300c7970 con 0x7f56300d2040 2023-11-18T20:43:44.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.113+0000 7f5637501640 1 -- 172.21.15.60:0/2306557430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5630158af0 con 0x7f56300d2040 2023-11-18T20:43:44.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.113+0000 7f5625ffb640 1 -- 172.21.15.60:0/2306557430 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f562004d450 con 0x7f56300d2040 2023-11-18T20:43:44.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.114+0000 7f5625ffb640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f561406ae40 0x7f561406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.114+0000 7f5625ffb640 1 -- 172.21.15.60:0/2306557430 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f56200abfd0 con 0x7f56300d2040 2023-11-18T20:43:44.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.114+0000 7f563529d640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f561406ae40 0x7f561406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.115+0000 7f563529d640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f561406ae40 0x7f561406d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f562c00b840 tx=0x7f562c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:44.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.118+0000 7f5625ffb640 1 -- 172.21.15.60:0/2306557430 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5620076ef0 con 0x7f56300d2040 2023-11-18T20:43:44.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.297+0000 7f5637501640 1 -- 172.21.15.60:0/2306557430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5630069400 con 0x7f56300d2040 2023-11-18T20:43:44.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.297+0000 7f5625ffb640 1 -- 172.21.15.60:0/2306557430 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f56200350a0 con 0x7f56300d2040 2023-11-18T20:43:44.298 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:44.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 -- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f561406ae40 msgr2=0x7f561406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f561406ae40 0x7f561406d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f562c00b840 tx=0x7f562c008040 comp rx=0 tx=0).stop 2023-11-18T20:43:44.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 -- 172.21.15.60:0/2306557430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 msgr2=0x7f56300b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 0x7f56300b3aa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f562002b970 tx=0x7f56200046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 -- 172.21.15.60:0/2306557430 shutdown_connections 2023-11-18T20:43:44.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f561406ae40 0x7f561406d300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f563015d2d0 0x7f56300ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56300d2040 0x7f56300b3aa0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 --2- 172.21.15.60:0/2306557430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56300cda40 0x7f56300ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 -- 172.21.15.60:0/2306557430 >> 172.21.15.60:0/2306557430 conn(0x7f5630072050 msgr2=0x7f5630072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:44.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 -- 172.21.15.60:0/2306557430 shutdown_connections 2023-11-18T20:43:44.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.299+0000 7f5637501640 1 -- 172.21.15.60:0/2306557430 wait complete. 2023-11-18T20:43:44.311 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-11-18T20:43:44.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.419+0000 7f2ab83a9640 1 Processor -- start 2023-11-18T20:43:44.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.419+0000 7f2ab83a9640 1 -- start start 2023-11-18T20:43:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab83a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ab015c2f0 0x7f2ab006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab83a9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2ab006f040 0x7f2ab0140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab83a9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab015ba40 con 0x7f2ab0140d40 2023-11-18T20:43:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab83a9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab015c6d0 con 0x7f2ab006f040 2023-11-18T20:43:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab83a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab013e530 con 0x7f2ab015c2f0 2023-11-18T20:43:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab6946640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2ab0140d40 0x7f2ab015ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37506/0 (socket says 172.21.15.60:37506) 2023-11-18T20:43:44.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab6946640 1 -- 172.21.15.60:0/2317297999 learned_addr learned my addr 172.21.15.60:0/2317297999 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:44.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab5944640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2ab006f040 0x7f2ab0140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab6145640 1 --2- 172.21.15.60:0/2317297999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ab015c2f0 0x7f2ab006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab5944640 1 -- 172.21.15.60:0/2317297999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ab015c2f0 msgr2=0x7f2ab006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:44.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab5944640 1 --2- 172.21.15.60:0/2317297999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ab015c2f0 0x7f2ab006eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab5944640 1 -- 172.21.15.60:0/2317297999 >> v1:172.21.15.60:6789/0 conn(0x7f2ab0140d40 legacy=0x7f2ab015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab5143640 1 -- 172.21.15.60:0/2317297999 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051260932 0 0) 0x7f2ab015ba40 con 0x7f2ab0140d40 2023-11-18T20:43:44.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.420+0000 7f2ab5944640 1 -- 172.21.15.60:0/2317297999 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab01411d0 con 0x7f2ab006f040 2023-11-18T20:43:44.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.421+0000 7f2ab5944640 1 --2- 172.21.15.60:0/2317297999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 0x7f2ab0140800 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009800 tx=0x7f2aa402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d3b884a4a0d792 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:44.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.421+0000 7f2ab5143640 1 -- 172.21.15.60:0/2317297999 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4031800 con 0x7f2ab006f040 2023-11-18T20:43:44.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.421+0000 7f2ab5143640 1 -- 172.21.15.60:0/2317297999 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa4031960 con 0x7f2ab006f040 2023-11-18T20:43:44.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.421+0000 7f2ab5143640 1 -- 172.21.15.60:0/2317297999 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4031cf0 con 0x7f2ab006f040 2023-11-18T20:43:44.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.422+0000 7f2ab83a9640 1 -- 172.21.15.60:0/2317297999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 msgr2=0x7f2ab0140800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.422+0000 7f2ab83a9640 1 --2- 172.21.15.60:0/2317297999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 0x7f2ab0140800 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009800 tx=0x7f2aa402b8a0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.422+0000 7f2ab83a9640 1 -- 172.21.15.60:0/2317297999 shutdown_connections 2023-11-18T20:43:44.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.422+0000 7f2ab83a9640 1 --2- 172.21.15.60:0/2317297999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 0x7f2ab0140800 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.422+0000 7f2ab83a9640 1 --2- 172.21.15.60:0/2317297999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ab015c2f0 0x7f2ab006eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.422+0000 7f2ab83a9640 1 -- 172.21.15.60:0/2317297999 >> 172.21.15.60:0/2317297999 conn(0x7f2ab006a6b0 msgr2=0x7f2ab006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:44.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.422+0000 7f2ab83a9640 1 -- 172.21.15.60:0/2317297999 shutdown_connections 2023-11-18T20:43:44.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.422+0000 7f2ab83a9640 1 -- 172.21.15.60:0/2317297999 wait complete. 2023-11-18T20:43:44.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.423+0000 7f2ab83a9640 1 Processor -- start 2023-11-18T20:43:44.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.423+0000 7f2ab83a9640 1 -- start start 2023-11-18T20:43:44.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab83a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 0x7f2ab013d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab83a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ab0140d40 0x7f2ab015c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab83a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ab015c2f0 0x7f2ab015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab83a9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0158d80 con 0x7f2ab006f040 2023-11-18T20:43:44.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab83a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0153760 con 0x7f2ab0140d40 2023-11-18T20:43:44.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab83a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0155570 con 0x7f2ab015c2f0 2023-11-18T20:43:44.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab6145640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 0x7f2ab013d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab6145640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 0x7f2ab013d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57428/0 (socket says 172.21.15.60:57428) 2023-11-18T20:43:44.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab6145640 1 -- 172.21.15.60:0/3388356133 learned_addr learned my addr 172.21.15.60:0/3388356133 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:44.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab6946640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ab015c2f0 0x7f2ab015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab5944640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ab0140d40 0x7f2ab015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab6145640 1 -- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ab015c2f0 msgr2=0x7f2ab015cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab6145640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ab015c2f0 0x7f2ab015cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab6145640 1 -- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ab0140d40 msgr2=0x7f2ab015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab6145640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ab0140d40 0x7f2ab015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.424+0000 7f2ab6145640 1 -- 172.21.15.60:0/3388356133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa4009400 con 0x7f2ab006f040 2023-11-18T20:43:44.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.425+0000 7f2ab6145640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 0x7f2ab013d3d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800d620 tx=0x7f2aa800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:44.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.425+0000 7f2aaeffd640 1 -- 172.21.15.60:0/3388356133 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80125f0 con 0x7f2ab006f040 2023-11-18T20:43:44.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.425+0000 7f2aaeffd640 1 -- 172.21.15.60:0/3388356133 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa8012750 con 0x7f2ab006f040 2023-11-18T20:43:44.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.425+0000 7f2ab83a9640 1 -- 172.21.15.60:0/3388356133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab006f4a0 con 0x7f2ab006f040 2023-11-18T20:43:44.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.425+0000 7f2aaeffd640 1 -- 172.21.15.60:0/3388356133 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8012b00 con 0x7f2ab006f040 2023-11-18T20:43:44.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.425+0000 7f2ab83a9640 1 -- 172.21.15.60:0/3388356133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab0153f30 con 0x7f2ab006f040 2023-11-18T20:43:44.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.426+0000 7f2ab83a9640 1 -- 172.21.15.60:0/3388356133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab0153a80 con 0x7f2ab006f040 2023-11-18T20:43:44.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.426+0000 7f2aaeffd640 1 -- 172.21.15.60:0/3388356133 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2aa8012c60 con 0x7f2ab006f040 2023-11-18T20:43:44.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.427+0000 7f2aaeffd640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a9c06ae40 0x7f2a9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.427+0000 7f2aaeffd640 1 -- 172.21.15.60:0/3388356133 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2aa8089aa0 con 0x7f2ab006f040 2023-11-18T20:43:44.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.427+0000 7f2ab5944640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a9c06ae40 0x7f2a9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.428+0000 7f2ab5944640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a9c06ae40 0x7f2a9c06d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4036000 tx=0x7f2aa403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:44.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.431+0000 7f2aaeffd640 1 -- 172.21.15.60:0/3388356133 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2aa8054a70 con 0x7f2ab006f040 2023-11-18T20:43:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.609+0000 7f2ab83a9640 1 -- 172.21.15.60:0/3388356133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f2ab0069400 con 0x7f2ab006f040 2023-11-18T20:43:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.609+0000 7f2aaeffd640 1 -- 172.21.15.60:0/3388356133 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7f2aa8058090 con 0x7f2ab006f040 2023-11-18T20:43:44.612 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:44.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.611+0000 7f2ab83a9640 1 -- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a9c06ae40 msgr2=0x7f2a9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.611+0000 7f2ab83a9640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a9c06ae40 0x7f2a9c06d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4036000 tx=0x7f2aa403c040 comp rx=0 tx=0).stop 2023-11-18T20:43:44.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.611+0000 7f2ab83a9640 1 -- 172.21.15.60:0/3388356133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 msgr2=0x7f2ab013d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.611+0000 7f2ab83a9640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 0x7f2ab013d3d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800d620 tx=0x7f2aa800dae0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.615 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.612+0000 7f2ab83a9640 1 -- 172.21.15.60:0/3388356133 shutdown_connections 2023-11-18T20:43:44.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.612+0000 7f2ab83a9640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a9c06ae40 0x7f2a9c06d300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.612+0000 7f2ab83a9640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ab015c2f0 0x7f2ab015cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.612+0000 7f2ab83a9640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ab0140d40 0x7f2ab015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.612+0000 7f2ab83a9640 1 --2- 172.21.15.60:0/3388356133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ab006f040 0x7f2ab013d3d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.612+0000 7f2ab83a9640 1 -- 172.21.15.60:0/3388356133 >> 172.21.15.60:0/3388356133 conn(0x7f2ab006a6b0 msgr2=0x7f2ab013fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:44.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.612+0000 7f2ab83a9640 1 -- 172.21.15.60:0/3388356133 shutdown_connections 2023-11-18T20:43:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.612+0000 7f2ab83a9640 1 -- 172.21.15.60:0/3388356133 wait complete. 2023-11-18T20:43:44.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:44.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.731+0000 7fbf2531d640 1 Processor -- start 2023-11-18T20:43:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.731+0000 7fbf2531d640 1 -- start start 2023-11-18T20:43:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf2531d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbf20160fb0 0x7fbf20161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf2531d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf20161d20 con 0x7fbf2006ea50 2023-11-18T20:43:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf2531d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2015e760 con 0x7fbf200d0ed0 2023-11-18T20:43:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf2531d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2015d9a0 con 0x7fbf20160fb0 2023-11-18T20:43:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1effd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbf2006ea50 0x7fbf2006ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37520/0 (socket says 172.21.15.60:37520) 2023-11-18T20:43:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1effd640 1 -- 172.21.15.60:0/2177176076 learned_addr learned my addr 172.21.15.60:0/2177176076 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1e7fc640 1 --2- 172.21.15.60:0/2177176076 >> v2:172.21.15.100:3300/0 conn(0x7fbf20160fb0 0x7fbf20161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015886359 0 0) 0x7fbf20161d20 con 0x7fbf2006ea50 2023-11-18T20:43:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0c003520 con 0x7fbf2006ea50 2023-11-18T20:43:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2940929139 0 0) 0x7fbf0c003520 con 0x7fbf2006ea50 2023-11-18T20:43:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 >> v2:172.21.15.100:3300/0 conn(0x7fbf20160fb0 msgr2=0x7fbf20161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1dffb640 1 --2- 172.21.15.60:0/2177176076 >> v2:172.21.15.100:3300/0 conn(0x7fbf20160fb0 0x7fbf20161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 >> v1:172.21.15.100:6789/0 conn(0x7fbf200d0ed0 legacy=0x7fbf200d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2015dc80 con 0x7fbf2006ea50 2023-11-18T20:43:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fbf18001670 con 0x7fbf2006ea50 2023-11-18T20:43:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 >> v1:172.21.15.60:6789/0 conn(0x7fbf2006ea50 legacy=0x7fbf2006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.732+0000 7fbf1dffb640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf0c003970 0x7fbf0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1dffb640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1dffb640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf0c005180 0x7fbf0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2015d9a0 con 0x7fbf0c0047c0 2023-11-18T20:43:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c00c120 con 0x7fbf0c003970 2023-11-18T20:43:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c00c2e0 con 0x7fbf0c005180 2023-11-18T20:43:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1e7fc640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1effd640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf0c003970 0x7fbf0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1f7fe640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf0c005180 0x7fbf0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1e7fc640 1 -- 172.21.15.60:0/2177176076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf0c005180 msgr2=0x7fbf0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1e7fc640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf0c005180 0x7fbf0c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1e7fc640 1 -- 172.21.15.60:0/2177176076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf0c003970 msgr2=0x7fbf0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1e7fc640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf0c003970 0x7fbf0c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1f7fe640 1 -- 172.21.15.60:0/2177176076 reap_dead start 2023-11-18T20:43:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.733+0000 7fbf1e7fc640 1 -- 172.21.15.60:0/2177176076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c003520 con 0x7fbf0c0047c0 2023-11-18T20:43:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf1e7fc640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf0c004c40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009ac0 tx=0x7fbf0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e649bed44e2092b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0802c6d0 con 0x7fbf0c0047c0 2023-11-18T20:43:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0802c830 con 0x7fbf0c0047c0 2023-11-18T20:43:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf1dffb640 1 -- 172.21.15.60:0/2177176076 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0802cbc0 con 0x7fbf0c0047c0 2023-11-18T20:43:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf2531d640 1 -- 172.21.15.60:0/2177176076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 msgr2=0x7fbf0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf2531d640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf0c004c40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009ac0 tx=0x7fbf0c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf2531d640 1 -- 172.21.15.60:0/2177176076 shutdown_connections 2023-11-18T20:43:44.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf2531d640 1 --2- 172.21.15.60:0/2177176076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf0c004c40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf2531d640 1 -- 172.21.15.60:0/2177176076 >> 172.21.15.60:0/2177176076 conn(0x7fbf2006a6b0 msgr2=0x7fbf2006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:44.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf2531d640 1 -- 172.21.15.60:0/2177176076 shutdown_connections 2023-11-18T20:43:44.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.734+0000 7fbf2531d640 1 -- 172.21.15.60:0/2177176076 wait complete. 2023-11-18T20:43:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.736+0000 7fbf2531d640 1 Processor -- start 2023-11-18T20:43:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.736+0000 7fbf2531d640 1 -- start start 2023-11-18T20:43:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.736+0000 7fbf2531d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf200cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.736+0000 7fbf2531d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf200b1d50 0x7fbf200cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.736+0000 7fbf2531d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf200cc060 0x7fbf20158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.736+0000 7fbf2531d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200cc510 con 0x7fbf0c0047c0 2023-11-18T20:43:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.736+0000 7fbf2531d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200d0000 con 0x7fbf200cc060 2023-11-18T20:43:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.736+0000 7fbf2531d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200ce740 con 0x7fbf200b1d50 2023-11-18T20:43:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf200cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf200cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57444/0 (socket says 172.21.15.60:57444) 2023-11-18T20:43:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1effd640 1 -- 172.21.15.60:0/1820349349 learned_addr learned my addr 172.21.15.60:0/1820349349 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1f7fe640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf200cc060 0x7fbf20158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1e7fc640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf200b1d50 0x7fbf200cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1effd640 1 -- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf200b1d50 msgr2=0x7fbf200cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1effd640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf200b1d50 0x7fbf200cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1effd640 1 -- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf200cc060 msgr2=0x7fbf20158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1effd640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf200cc060 0x7fbf20158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1effd640 1 -- 172.21.15.60:0/1820349349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf200cea20 con 0x7fbf0c0047c0 2023-11-18T20:43:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf1effd640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf200cfac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf18007290 tx=0x7fbf18007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf177fe640 1 -- 172.21.15.60:0/1820349349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18001610 con 0x7fbf0c0047c0 2023-11-18T20:43:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf177fe640 1 -- 172.21.15.60:0/1820349349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1800f940 con 0x7fbf0c0047c0 2023-11-18T20:43:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf2531d640 1 -- 172.21.15.60:0/1820349349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf200ced00 con 0x7fbf0c0047c0 2023-11-18T20:43:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf177fe640 1 -- 172.21.15.60:0/1820349349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800fcf0 con 0x7fbf0c0047c0 2023-11-18T20:43:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.737+0000 7fbf2531d640 1 -- 172.21.15.60:0/1820349349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf200cf2c0 con 0x7fbf0c0047c0 2023-11-18T20:43:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.738+0000 7fbf2531d640 1 -- 172.21.15.60:0/1820349349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf20069400 con 0x7fbf0c0047c0 2023-11-18T20:43:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.739+0000 7fbf177fe640 1 -- 172.21.15.60:0/1820349349 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf1801d460 con 0x7fbf0c0047c0 2023-11-18T20:43:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.739+0000 7fbf177fe640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf0006ae70 0x7fbf0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.739+0000 7fbf177fe640 1 -- 172.21.15.60:0/1820349349 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fbf18089640 con 0x7fbf0c0047c0 2023-11-18T20:43:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.739+0000 7fbf1e7fc640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf0006ae70 0x7fbf0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.740+0000 7fbf1e7fc640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf0006ae70 0x7fbf0006d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf08036000 tx=0x7fbf08030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.743+0000 7fbf177fe640 1 -- 172.21.15.60:0/1820349349 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf18054610 con 0x7fbf0c0047c0 2023-11-18T20:43:44.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.933+0000 7fbf2531d640 1 -- 172.21.15.60:0/1820349349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbf20161830 con 0x7fbf0c0047c0 2023-11-18T20:43:44.934 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.933+0000 7fbf177fe640 1 -- 172.21.15.60:0/1820349349 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbf18057c30 con 0x7fbf0c0047c0 2023-11-18T20:43:44.934 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:44.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 -- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf0006ae70 msgr2=0x7fbf0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf0006ae70 0x7fbf0006d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf08036000 tx=0x7fbf08030000 comp rx=0 tx=0).stop 2023-11-18T20:43:44.936 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 -- 172.21.15.60:0/1820349349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 msgr2=0x7fbf200cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:44.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf200cfac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf18007290 tx=0x7fbf18007750 comp rx=0 tx=0).stop 2023-11-18T20:43:44.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 -- 172.21.15.60:0/1820349349 shutdown_connections 2023-11-18T20:43:44.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbf0006ae70 0x7fbf0006d330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf200cc060 0x7fbf20158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf200b1d50 0x7fbf200cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 --2- 172.21.15.60:0/1820349349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf0c0047c0 0x7fbf200cfac0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:44.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 -- 172.21.15.60:0/1820349349 >> 172.21.15.60:0/1820349349 conn(0x7fbf2006a6b0 msgr2=0x7fbf2015ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:44.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 -- 172.21.15.60:0/1820349349 shutdown_connections 2023-11-18T20:43:44.937 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:44.935+0000 7fbf2531d640 1 -- 172.21.15.60:0/1820349349 wait complete. 2023-11-18T20:43:44.947 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:43:44.947 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:44.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:45.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.108+0000 7fe3ae644640 1 Processor -- start 2023-11-18T20:43:45.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.108+0000 7fe3ae644640 1 -- start start 2023-11-18T20:43:45.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3ae644640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 0x7fe3a806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3ae644640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3a806f040 0x7fe3a8140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3ae644640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a815ba40 con 0x7fe3a8140d40 2023-11-18T20:43:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3ae644640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a815c6d0 con 0x7fe3a806f040 2023-11-18T20:43:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3ae644640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a813e530 con 0x7fe3a815c2f0 2023-11-18T20:43:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3a77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3a806f040 0x7fe3a8140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3acbe1640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe3a8140d40 0x7fe3a815ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37524/0 (socket says 172.21.15.60:37524) 2023-11-18T20:43:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3acbe1640 1 -- 172.21.15.60:0/1004953681 learned_addr learned my addr 172.21.15.60:0/1004953681 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3a7fff640 1 --2- 172.21.15.60:0/1004953681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 0x7fe3a806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3a77fe640 1 -- 172.21.15.60:0/1004953681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 msgr2=0x7fe3a806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3a77fe640 1 --2- 172.21.15.60:0/1004953681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 0x7fe3a806eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3a77fe640 1 -- 172.21.15.60:0/1004953681 >> v1:172.21.15.60:6789/0 conn(0x7fe3a8140d40 legacy=0x7fe3a815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.109+0000 7fe3a77fe640 1 -- 172.21.15.60:0/1004953681 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a81411d0 con 0x7fe3a806f040 2023-11-18T20:43:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.110+0000 7fe3a77fe640 1 --2- 172.21.15.60:0/1004953681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 0x7fe3a8140800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009800 tx=0x7fe39c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf8cf64320dc684 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.110+0000 7fe3a6ffd640 1 -- 172.21.15.60:0/1004953681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c031800 con 0x7fe3a806f040 2023-11-18T20:43:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.110+0000 7fe3a6ffd640 1 -- 172.21.15.60:0/1004953681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c031960 con 0x7fe3a806f040 2023-11-18T20:43:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.110+0000 7fe3a6ffd640 1 -- 172.21.15.60:0/1004953681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c031cf0 con 0x7fe3a806f040 2023-11-18T20:43:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.111+0000 7fe3ae644640 1 -- 172.21.15.60:0/1004953681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 msgr2=0x7fe3a8140800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.111+0000 7fe3ae644640 1 --2- 172.21.15.60:0/1004953681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 0x7fe3a8140800 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009800 tx=0x7fe39c02b8a0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.111+0000 7fe3ae644640 1 -- 172.21.15.60:0/1004953681 shutdown_connections 2023-11-18T20:43:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.111+0000 7fe3ae644640 1 --2- 172.21.15.60:0/1004953681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 0x7fe3a8140800 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.111+0000 7fe3ae644640 1 --2- 172.21.15.60:0/1004953681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 0x7fe3a806eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.111+0000 7fe3ae644640 1 -- 172.21.15.60:0/1004953681 >> 172.21.15.60:0/1004953681 conn(0x7fe3a806a6b0 msgr2=0x7fe3a806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.111+0000 7fe3ae644640 1 -- 172.21.15.60:0/1004953681 shutdown_connections 2023-11-18T20:43:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.111+0000 7fe3ae644640 1 -- 172.21.15.60:0/1004953681 wait complete. 2023-11-18T20:43:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.112+0000 7fe3ae644640 1 Processor -- start 2023-11-18T20:43:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.112+0000 7fe3ae644640 1 -- start start 2023-11-18T20:43:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3ae644640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 0x7fe3a81508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3ae644640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a8140d40 0x7fe3a8153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3ae644640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 0x7fe3a80b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3ae644640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8150df0 con 0x7fe3a806f040 2023-11-18T20:43:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3ae644640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8154170 con 0x7fe3a8140d40 2023-11-18T20:43:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3ae644640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8154450 con 0x7fe3a815c2f0 2023-11-18T20:43:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 0x7fe3a81508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 0x7fe3a81508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57466/0 (socket says 172.21.15.60:57466) 2023-11-18T20:43:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a7fff640 1 -- 172.21.15.60:0/3556820640 learned_addr learned my addr 172.21.15.60:0/3556820640 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a77fe640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a8140d40 0x7fe3a8153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3acbe1640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 0x7fe3a80b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a7fff640 1 -- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 msgr2=0x7fe3a80b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a7fff640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 0x7fe3a80b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a7fff640 1 -- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a8140d40 msgr2=0x7fe3a8153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a7fff640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a8140d40 0x7fe3a8153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a7fff640 1 -- 172.21.15.60:0/3556820640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c009400 con 0x7fe3a806f040 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a77fe640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a8140d40 0x7fe3a8153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3acbe1640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 0x7fe3a80b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a7fff640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 0x7fe3a81508b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe39800d5f0 tx=0x7fe39800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.113+0000 7fe3a4ff9640 1 -- 172.21.15.60:0/3556820640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3980125f0 con 0x7fe3a806f040 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.114+0000 7fe3a4ff9640 1 -- 172.21.15.60:0/3556820640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe398012750 con 0x7fe3a806f040 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.114+0000 7fe3ae644640 1 -- 172.21.15.60:0/3556820640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a806f4a0 con 0x7fe3a806f040 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.114+0000 7fe3a4ff9640 1 -- 172.21.15.60:0/3556820640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398012b00 con 0x7fe3a806f040 2023-11-18T20:43:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.114+0000 7fe3ae644640 1 -- 172.21.15.60:0/3556820640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a8154a10 con 0x7fe3a806f040 2023-11-18T20:43:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.115+0000 7fe3ae644640 1 -- 172.21.15.60:0/3556820640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a8069400 con 0x7fe3a806f040 2023-11-18T20:43:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.115+0000 7fe3a4ff9640 1 -- 172.21.15.60:0/3556820640 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe398012c60 con 0x7fe3a806f040 2023-11-18T20:43:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.115+0000 7fe3a4ff9640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe39406ae40 0x7fe39406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.115+0000 7fe3a4ff9640 1 -- 172.21.15.60:0/3556820640 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe398089c70 con 0x7fe3a806f040 2023-11-18T20:43:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.116+0000 7fe3a77fe640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe39406ae40 0x7fe39406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.116+0000 7fe3a77fe640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe39406ae40 0x7fe39406d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe39c036000 tx=0x7fe39c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:45.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.120+0000 7fe3a4ff9640 1 -- 172.21.15.60:0/3556820640 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe398054b90 con 0x7fe3a806f040 2023-11-18T20:43:45.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.297+0000 7fe3ae644640 1 -- 172.21.15.60:0/3556820640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3a8070d30 con 0x7fe3a806f040 2023-11-18T20:43:45.299 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.298+0000 7fe3a4ff9640 1 -- 172.21.15.60:0/3556820640 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18336 (secure 0 0 0) 0x7fe3980581b0 con 0x7fe3a806f040 2023-11-18T20:43:45.299 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:45.299 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:42.814654+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-11-18T20:43:40.650315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:45.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 -- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe39406ae40 msgr2=0x7fe39406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe39406ae40 0x7fe39406d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe39c036000 tx=0x7fe39c008030 comp rx=0 tx=0).stop 2023-11-18T20:43:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 -- 172.21.15.60:0/3556820640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 msgr2=0x7fe3a81508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 0x7fe3a81508b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe39800d5f0 tx=0x7fe39800dab0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 -- 172.21.15.60:0/3556820640 shutdown_connections 2023-11-18T20:43:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe39406ae40 0x7fe39406d300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3a815c2f0 0x7fe3a80b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3a8140d40 0x7fe3a8153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 --2- 172.21.15.60:0/3556820640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3a806f040 0x7fe3a81508b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 -- 172.21.15.60:0/3556820640 >> 172.21.15.60:0/3556820640 conn(0x7fe3a806a6b0 msgr2=0x7fe3a815db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:45.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 -- 172.21.15.60:0/3556820640 shutdown_connections 2023-11-18T20:43:45.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.300+0000 7fe3ae644640 1 -- 172.21.15.60:0/3556820640 wait complete. 2023-11-18T20:43:45.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:43:31.703619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:43:32.121610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-11-18T20:43:40.650315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:45.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:45.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.421+0000 7fca9dd83640 1 Processor -- start 2023-11-18T20:43:45.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.421+0000 7fca9dd83640 1 -- start start 2023-11-18T20:43:45.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.421+0000 7fca9dd83640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca9815d330 0x7fca980b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.421+0000 7fca9dd83640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fca980d2d90 0x7fca980b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.421+0000 7fca9dd83640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca980b2b10 con 0x7fca980d2910 2023-11-18T20:43:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.421+0000 7fca9dd83640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca980d1590 con 0x7fca980d2d90 2023-11-18T20:43:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.421+0000 7fca9dd83640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca980d1870 con 0x7fca9815d330 2023-11-18T20:43:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.421+0000 7fca96ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fca980d2910 0x7fca980d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37538/0 (socket says 172.21.15.60:37538) 2023-11-18T20:43:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca96ffd640 1 -- 172.21.15.60:0/3899900706 learned_addr learned my addr 172.21.15.60:0/3899900706 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca97fff640 1 --2- 172.21.15.60:0/3899900706 >> v2:172.21.15.100:3300/0 conn(0x7fca980d2d90 0x7fca980b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca977fe640 1 --2- 172.21.15.60:0/3899900706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca9815d330 0x7fca980b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca967fc640 1 -- 172.21.15.60:0/3899900706 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679838795 0 0) 0x7fca980b2b10 con 0x7fca980d2910 2023-11-18T20:43:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca967fc640 1 -- 172.21.15.60:0/3899900706 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca80003520 con 0x7fca980d2910 2023-11-18T20:43:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca967fc640 1 -- 172.21.15.60:0/3899900706 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2719103812 0 0) 0x7fca80003520 con 0x7fca980d2910 2023-11-18T20:43:45.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca977fe640 1 -- 172.21.15.60:0/3899900706 >> v2:172.21.15.100:3300/0 conn(0x7fca980d2d90 msgr2=0x7fca980b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca977fe640 1 --2- 172.21.15.60:0/3899900706 >> v2:172.21.15.100:3300/0 conn(0x7fca980d2d90 0x7fca980b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca977fe640 1 -- 172.21.15.60:0/3899900706 >> v1:172.21.15.60:6789/0 conn(0x7fca980d2910 legacy=0x7fca980d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca977fe640 1 -- 172.21.15.60:0/3899900706 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca980d1b50 con 0x7fca9815d330 2023-11-18T20:43:45.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.422+0000 7fca97fff640 1 --2- 172.21.15.60:0/3899900706 >> v2:172.21.15.100:3300/0 conn(0x7fca980d2d90 0x7fca980b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.423+0000 7fca977fe640 1 --2- 172.21.15.60:0/3899900706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca9815d330 0x7fca980b8b80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca880097a0 tx=0x7fca8802b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0cbb2e71771526f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.423+0000 7fca967fc640 1 -- 172.21.15.60:0/3899900706 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c001670 con 0x7fca9815d330 2023-11-18T20:43:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.423+0000 7fca967fc640 1 -- 172.21.15.60:0/3899900706 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8802e050 con 0x7fca9815d330 2023-11-18T20:43:45.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.423+0000 7fca967fc640 1 -- 172.21.15.60:0/3899900706 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88030730 con 0x7fca9815d330 2023-11-18T20:43:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.424+0000 7fca9dd83640 1 -- 172.21.15.60:0/3899900706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca9815d330 msgr2=0x7fca980b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.424+0000 7fca9dd83640 1 --2- 172.21.15.60:0/3899900706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca9815d330 0x7fca980b8b80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca880097a0 tx=0x7fca8802b710 comp rx=0 tx=0).stop 2023-11-18T20:43:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.424+0000 7fca9dd83640 1 -- 172.21.15.60:0/3899900706 shutdown_connections 2023-11-18T20:43:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.424+0000 7fca9dd83640 1 --2- 172.21.15.60:0/3899900706 >> v2:172.21.15.100:3300/0 conn(0x7fca980d2d90 0x7fca980b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.424+0000 7fca9dd83640 1 --2- 172.21.15.60:0/3899900706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca9815d330 0x7fca980b8b80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.424+0000 7fca9dd83640 1 -- 172.21.15.60:0/3899900706 >> 172.21.15.60:0/3899900706 conn(0x7fca98072020 msgr2=0x7fca98072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.424+0000 7fca9dd83640 1 -- 172.21.15.60:0/3899900706 shutdown_connections 2023-11-18T20:43:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.424+0000 7fca9dd83640 1 -- 172.21.15.60:0/3899900706 wait complete. 2023-11-18T20:43:45.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.425+0000 7fca9dd83640 1 Processor -- start 2023-11-18T20:43:45.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.425+0000 7fca9dd83640 1 -- start start 2023-11-18T20:43:45.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca9dd83640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca980d2910 0x7fca98154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca9dd83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca980d2d90 0x7fca980c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca9dd83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca9815d330 0x7fca980c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca9dd83640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca98155af0 con 0x7fca980d2d90 2023-11-18T20:43:45.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca9dd83640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca981f3210 con 0x7fca9815d330 2023-11-18T20:43:45.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca9dd83640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca98154e70 con 0x7fca980d2910 2023-11-18T20:43:45.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca96ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca980d2d90 0x7fca980c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca96ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca980d2d90 0x7fca980c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57474/0 (socket says 172.21.15.60:57474) 2023-11-18T20:43:45.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca96ffd640 1 -- 172.21.15.60:0/3966916933 learned_addr learned my addr 172.21.15.60:0/3966916933 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:45.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca97fff640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca9815d330 0x7fca980c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca977fe640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca980d2910 0x7fca98154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca96ffd640 1 -- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca980d2910 msgr2=0x7fca98154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca96ffd640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca980d2910 0x7fca98154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca96ffd640 1 -- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca9815d330 msgr2=0x7fca980c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca96ffd640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca9815d330 0x7fca980c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.426+0000 7fca96ffd640 1 -- 172.21.15.60:0/3966916933 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca90009a70 con 0x7fca980d2d90 2023-11-18T20:43:45.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.427+0000 7fca96ffd640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca980d2d90 0x7fca980c6ba0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00c750 tx=0x7fca8c00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:45.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.427+0000 7fca7ffff640 1 -- 172.21.15.60:0/3966916933 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c005ad0 con 0x7fca980d2d90 2023-11-18T20:43:45.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.427+0000 7fca7ffff640 1 -- 172.21.15.60:0/3966916933 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c00ce90 con 0x7fca980d2d90 2023-11-18T20:43:45.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.427+0000 7fca9dd83640 1 -- 172.21.15.60:0/3966916933 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca88009400 con 0x7fca980d2d90 2023-11-18T20:43:45.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.427+0000 7fca7ffff640 1 -- 172.21.15.60:0/3966916933 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c005e90 con 0x7fca980d2d90 2023-11-18T20:43:45.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.427+0000 7fca9dd83640 1 -- 172.21.15.60:0/3966916933 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca98155430 con 0x7fca980d2d90 2023-11-18T20:43:45.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.428+0000 7fca9dd83640 1 -- 172.21.15.60:0/3966916933 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca98155c30 con 0x7fca980d2d90 2023-11-18T20:43:45.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.428+0000 7fca7ffff640 1 -- 172.21.15.60:0/3966916933 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca8c007070 con 0x7fca980d2d90 2023-11-18T20:43:45.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.429+0000 7fca7ffff640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca8406ae40 0x7fca8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.429+0000 7fca7ffff640 1 -- 172.21.15.60:0/3966916933 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fca8c089450 con 0x7fca980d2d90 2023-11-18T20:43:45.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.429+0000 7fca977fe640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca8406ae40 0x7fca8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.429+0000 7fca977fe640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca8406ae40 0x7fca8406d300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fca88034000 tx=0x7fca88008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:45.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.433+0000 7fca7ffff640 1 -- 172.21.15.60:0/3966916933 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca98155c30 con 0x7fca980d2d90 2023-11-18T20:43:45.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.608+0000 7fca9dd83640 1 -- 172.21.15.60:0/3966916933 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca98155c30 con 0x7fca980d2d90 2023-11-18T20:43:45.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.608+0000 7fca7ffff640 1 -- 172.21.15.60:0/3966916933 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca98155c30 con 0x7fca980d2d90 2023-11-18T20:43:45.610 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 -- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca8406ae40 msgr2=0x7fca8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca8406ae40 0x7fca8406d300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fca88034000 tx=0x7fca88008030 comp rx=0 tx=0).stop 2023-11-18T20:43:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 -- 172.21.15.60:0/3966916933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca980d2d90 msgr2=0x7fca980c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca980d2d90 0x7fca980c6ba0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00c750 tx=0x7fca8c00cb50 comp rx=0 tx=0).stop 2023-11-18T20:43:45.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 -- 172.21.15.60:0/3966916933 shutdown_connections 2023-11-18T20:43:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fca8406ae40 0x7fca8406d300 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca9815d330 0x7fca980c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca980d2d90 0x7fca980c6ba0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 --2- 172.21.15.60:0/3966916933 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca980d2910 0x7fca98154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 -- 172.21.15.60:0/3966916933 >> 172.21.15.60:0/3966916933 conn(0x7fca98072020 msgr2=0x7fca98072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 -- 172.21.15.60:0/3966916933 shutdown_connections 2023-11-18T20:43:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.611+0000 7fca9dd83640 1 -- 172.21.15.60:0/3966916933 wait complete. 2023-11-18T20:43:45.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.732+0000 7faae9e61640 1 Processor -- start 2023-11-18T20:43:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.732+0000 7faae9e61640 1 -- start start 2023-11-18T20:43:45.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae9e61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faae415d1d0 0x7faae406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae9e61640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faae40aeb80 0x7faae406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae9e61640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae40adae0 con 0x7faae40ad640 2023-11-18T20:43:45.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae9e61640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae40addc0 con 0x7faae40aeb80 2023-11-18T20:43:45.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae9e61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae40ae0a0 con 0x7faae415d1d0 2023-11-18T20:43:45.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faae40aeb80 0x7faae406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae2ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faae40ad640 0x7faae4160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37542/0 (socket says 172.21.15.60:37542) 2023-11-18T20:43:45.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae2ffd640 1 -- 172.21.15.60:0/691447617 learned_addr learned my addr 172.21.15.60:0/691447617 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:45.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae37fe640 1 --2- 172.21.15.60:0/691447617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faae415d1d0 0x7faae406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae3fff640 1 -- 172.21.15.60:0/691447617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faae415d1d0 msgr2=0x7faae406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:45.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae3fff640 1 --2- 172.21.15.60:0/691447617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faae415d1d0 0x7faae406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae3fff640 1 -- 172.21.15.60:0/691447617 >> v1:172.21.15.60:6789/0 conn(0x7faae40ad640 legacy=0x7faae4160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae3fff640 1 -- 172.21.15.60:0/691447617 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae40ae380 con 0x7faae40aeb80 2023-11-18T20:43:45.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.733+0000 7faae27fc640 1 -- 172.21.15.60:0/691447617 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510829220 0 0) 0x7faae40adae0 con 0x7faae40ad640 2023-11-18T20:43:45.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.734+0000 7faae3fff640 1 --2- 172.21.15.60:0/691447617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae40aeb80 0x7faae406ba20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faadc009d90 tx=0x7faadc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d43224d6ff74b39 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:45.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.734+0000 7faae27fc640 1 -- 172.21.15.60:0/691447617 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc02cc50 con 0x7faae40aeb80 2023-11-18T20:43:45.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.734+0000 7faae27fc640 1 -- 172.21.15.60:0/691447617 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faadc02cdb0 con 0x7faae40aeb80 2023-11-18T20:43:45.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.734+0000 7faae27fc640 1 -- 172.21.15.60:0/691447617 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc034660 con 0x7faae40aeb80 2023-11-18T20:43:45.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.735+0000 7faae9e61640 1 -- 172.21.15.60:0/691447617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae40aeb80 msgr2=0x7faae406ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.735+0000 7faae9e61640 1 --2- 172.21.15.60:0/691447617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae40aeb80 0x7faae406ba20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faadc009d90 tx=0x7faadc02bea0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.735+0000 7faae9e61640 1 -- 172.21.15.60:0/691447617 shutdown_connections 2023-11-18T20:43:45.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.735+0000 7faae9e61640 1 --2- 172.21.15.60:0/691447617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae40aeb80 0x7faae406ba20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.735+0000 7faae9e61640 1 --2- 172.21.15.60:0/691447617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faae415d1d0 0x7faae406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.735+0000 7faae9e61640 1 -- 172.21.15.60:0/691447617 >> 172.21.15.60:0/691447617 conn(0x7faae4070e70 msgr2=0x7faae4071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:45.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.735+0000 7faae9e61640 1 -- 172.21.15.60:0/691447617 shutdown_connections 2023-11-18T20:43:45.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.735+0000 7faae9e61640 1 -- 172.21.15.60:0/691447617 wait complete. 2023-11-18T20:43:45.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.736+0000 7faae9e61640 1 Processor -- start 2023-11-18T20:43:45.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.736+0000 7faae9e61640 1 -- start start 2023-11-18T20:43:45.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae9e61640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae40ad640 0x7faae40cead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae9e61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faae40aeb80 0x7faae40cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae9e61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae415d1d0 0x7faae413eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae9e61640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae413ce80 con 0x7faae415d1d0 2023-11-18T20:43:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae9e61640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae415f3f0 con 0x7faae40ad640 2023-11-18T20:43:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae9e61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae41400b0 con 0x7faae40aeb80 2023-11-18T20:43:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae415d1d0 0x7faae413eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae415d1d0 0x7faae413eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57486/0 (socket says 172.21.15.60:57486) 2023-11-18T20:43:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae3fff640 1 -- 172.21.15.60:0/4011971293 learned_addr learned my addr 172.21.15.60:0/4011971293 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae2ffd640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faae40aeb80 0x7faae40cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae37fe640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae40ad640 0x7faae40cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae3fff640 1 -- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faae40aeb80 msgr2=0x7faae40cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae3fff640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faae40aeb80 0x7faae40cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae3fff640 1 -- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae40ad640 msgr2=0x7faae40cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae3fff640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae40ad640 0x7faae40cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.737+0000 7faae3fff640 1 -- 172.21.15.60:0/4011971293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc009a70 con 0x7faae415d1d0 2023-11-18T20:43:45.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.738+0000 7faae3fff640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae415d1d0 0x7faae413eca0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7faadc009d90 tx=0x7faadc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:45.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.738+0000 7faacbfff640 1 -- 172.21.15.60:0/4011971293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc040650 con 0x7faae415d1d0 2023-11-18T20:43:45.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.738+0000 7faacbfff640 1 -- 172.21.15.60:0/4011971293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faadc0407d0 con 0x7faae415d1d0 2023-11-18T20:43:45.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.738+0000 7faae9e61640 1 -- 172.21.15.60:0/4011971293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae413faf0 con 0x7faae415d1d0 2023-11-18T20:43:45.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.738+0000 7faacbfff640 1 -- 172.21.15.60:0/4011971293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc040b80 con 0x7faae415d1d0 2023-11-18T20:43:45.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.738+0000 7faae9e61640 1 -- 172.21.15.60:0/4011971293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae415f080 con 0x7faae415d1d0 2023-11-18T20:43:45.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.739+0000 7faae9e61640 1 -- 172.21.15.60:0/4011971293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae415eee0 con 0x7faae415d1d0 2023-11-18T20:43:45.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.739+0000 7faacbfff640 1 -- 172.21.15.60:0/4011971293 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faadc040ce0 con 0x7faae415d1d0 2023-11-18T20:43:45.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.739+0000 7faacbfff640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faad006ae40 0x7faad006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.740+0000 7faacbfff640 1 -- 172.21.15.60:0/4011971293 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7faadc0ac4e0 con 0x7faae415d1d0 2023-11-18T20:43:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.740+0000 7faae37fe640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faad006ae40 0x7faad006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.740+0000 7faae37fe640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faad006ae40 0x7faad006d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faad80079a0 tx=0x7faad8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.744+0000 7faacbfff640 1 -- 172.21.15.60:0/4011971293 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faae415eee0 con 0x7faae415d1d0 2023-11-18T20:43:45.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.922+0000 7faae9e61640 1 -- 172.21.15.60:0/4011971293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faae415eee0 con 0x7faae415d1d0 2023-11-18T20:43:45.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.922+0000 7faacbfff640 1 -- 172.21.15.60:0/4011971293 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7faae415eee0 con 0x7faae415d1d0 2023-11-18T20:43:45.924 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:45.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 -- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faad006ae40 msgr2=0x7faad006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faad006ae40 0x7faad006d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faad80079a0 tx=0x7faad8008040 comp rx=0 tx=0).stop 2023-11-18T20:43:45.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 -- 172.21.15.60:0/4011971293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae415d1d0 msgr2=0x7faae413eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:45.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae415d1d0 0x7faae413eca0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7faadc009d90 tx=0x7faadc004a40 comp rx=0 tx=0).stop 2023-11-18T20:43:45.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 -- 172.21.15.60:0/4011971293 shutdown_connections 2023-11-18T20:43:45.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faad006ae40 0x7faad006d300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faae415d1d0 0x7faae413eca0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faae40aeb80 0x7faae40cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 --2- 172.21.15.60:0/4011971293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faae40ad640 0x7faae40cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:45.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 -- 172.21.15.60:0/4011971293 >> 172.21.15.60:0/4011971293 conn(0x7faae4070e70 msgr2=0x7faae40b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:45.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 -- 172.21.15.60:0/4011971293 shutdown_connections 2023-11-18T20:43:45.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:45.924+0000 7faae9e61640 1 -- 172.21.15.60:0/4011971293 wait complete. 2023-11-18T20:43:45.936 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:46.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.044+0000 7fba02933640 1 Processor -- start 2023-11-18T20:43:46.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.044+0000 7fba02933640 1 -- start start 2023-11-18T20:43:46.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.045+0000 7fba02933640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb9fc06ea50 0x7fb9fc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.045+0000 7fba02933640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc161d20 con 0x7fb9fc160fb0 2023-11-18T20:43:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.045+0000 7fba02933640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc06f370 con 0x7fb9fc0d0ed0 2023-11-18T20:43:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.045+0000 7fba02933640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9fc15e760 con 0x7fb9fc06ea50 2023-11-18T20:43:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.045+0000 7fb9fb7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb9fc160fb0 0x7fb9fc161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37548/0 (socket says 172.21.15.60:37548) 2023-11-18T20:43:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.045+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/1162994786 learned_addr learned my addr 172.21.15.60:0/1162994786 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9fbfff640 1 --2- 172.21.15.60:0/1162994786 >> v2:172.21.15.100:3300/0 conn(0x7fb9fc06ea50 0x7fb9fc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147204470 0 0) 0x7fb9fc161d20 con 0x7fb9fc160fb0 2023-11-18T20:43:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9e4003520 con 0x7fb9fc160fb0 2023-11-18T20:43:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 171021409 0 0) 0x7fb9e4003520 con 0x7fb9fc160fb0 2023-11-18T20:43:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 >> v2:172.21.15.100:3300/0 conn(0x7fb9fc06ea50 msgr2=0x7fb9fc06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:46.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 --2- 172.21.15.60:0/1162994786 >> v2:172.21.15.100:3300/0 conn(0x7fb9fc06ea50 0x7fb9fc06ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 >> v1:172.21.15.100:6789/0 conn(0x7fb9fc0d0ed0 legacy=0x7fb9fc0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc15d9a0 con 0x7fb9fc160fb0 2023-11-18T20:43:46.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fb9ec001670 con 0x7fb9fc160fb0 2023-11-18T20:43:46.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 >> v1:172.21.15.60:6789/0 conn(0x7fb9fc160fb0 legacy=0x7fb9fc161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9e4003970 0x7fb9e4003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.050 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9e4004370 0x7fb9e4004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e40050e0 0x7fb9e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc15e760 con 0x7fb9e4004370 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e400c020 con 0x7fb9e40050e0 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e400c1e0 con 0x7fb9e4003970 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9e4004370 0x7fb9e4004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fba00ed0640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e40050e0 0x7fb9e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9fbfff640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9e4003970 0x7fb9e4003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/1162994786 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9e4003970 msgr2=0x7fb9e4003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9e4003970 0x7fb9e4003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/1162994786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e40050e0 msgr2=0x7fb9e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e40050e0 0x7fb9e400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.051 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/1162994786 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e4003520 con 0x7fb9e4004370 2023-11-18T20:43:46.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fba00ed0640 1 -- 172.21.15.60:0/1162994786 reap_dead start 2023-11-18T20:43:46.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.046+0000 7fb9fbfff640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9e4003970 0x7fb9e4003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:46.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.047+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9e4004370 0x7fb9e4004840 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c210 tx=0x7fb9e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac2a1fc552a795f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:46.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.047+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec005ad0 con 0x7fb9e4004370 2023-11-18T20:43:46.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.047+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec005c50 con 0x7fb9e4004370 2023-11-18T20:43:46.052 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.047+0000 7fb9faffd640 1 -- 172.21.15.60:0/1162994786 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec02c8d0 con 0x7fb9e4004370 2023-11-18T20:43:46.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.048+0000 7fba02933640 1 -- 172.21.15.60:0/1162994786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9e4004370 msgr2=0x7fb9e4004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.048+0000 7fba02933640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9e4004370 0x7fb9e4004840 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c210 tx=0x7fb9e40057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.048+0000 7fba02933640 1 -- 172.21.15.60:0/1162994786 shutdown_connections 2023-11-18T20:43:46.055 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.048+0000 7fba02933640 1 --2- 172.21.15.60:0/1162994786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9e4004370 0x7fb9e4004840 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.048+0000 7fba02933640 1 -- 172.21.15.60:0/1162994786 >> 172.21.15.60:0/1162994786 conn(0x7fb9fc06a6b0 msgr2=0x7fb9fc06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.048+0000 7fba02933640 1 -- 172.21.15.60:0/1162994786 shutdown_connections 2023-11-18T20:43:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.048+0000 7fba02933640 1 -- 172.21.15.60:0/1162994786 wait complete. 2023-11-18T20:43:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.049+0000 7fba02933640 1 Processor -- start 2023-11-18T20:43:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.049+0000 7fba02933640 1 -- start start 2023-11-18T20:43:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fba02933640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4004370 0x7fb9fc1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fba02933640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc1f2cd0 0x7fb9fc15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fba02933640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc1f3150 0x7fb9fc15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fba02933640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc06c090 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fba02933640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc06b760 con 0x7fb9e4004370 2023-11-18T20:43:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fba02933640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc06ba40 con 0x7fb9fc1f3150 2023-11-18T20:43:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fb9fb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc1f2cd0 0x7fb9fc15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fb9fb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc1f2cd0 0x7fb9fc15a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57512/0 (socket says 172.21.15.60:57512) 2023-11-18T20:43:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/3645217058 learned_addr learned my addr 172.21.15.60:0/3645217058 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fb9fbfff640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4004370 0x7fb9fc1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fba00ed0640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc1f3150 0x7fb9fc15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc1f3150 msgr2=0x7fb9fc15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc1f3150 0x7fb9fc15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4004370 msgr2=0x7fb9fc1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4004370 0x7fb9fc1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/3645217058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc06bd20 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.050+0000 7fb9fbfff640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4004370 0x7fb9fc1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.051+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc1f2cd0 0x7fb9fc15a140 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec02c470 tx=0x7fb9ec02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.051+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/3645217058 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec03c040 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.051+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/3645217058 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec02cda0 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.051+0000 7fba02933640 1 -- 172.21.15.60:0/3645217058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc156c00 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.051+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/3645217058 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec007290 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.051+0000 7fba02933640 1 -- 172.21.15.60:0/3645217058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc06c700 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.052+0000 7fba02933640 1 -- 172.21.15.60:0/3645217058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc06beb0 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.052+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/3645217058 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9ec032070 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.053+0000 7fb9f8ff9640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae40 0x7fb9e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.053+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/3645217058 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fb9ec0aab60 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.053+0000 7fb9fbfff640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae40 0x7fb9e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.053+0000 7fb9fbfff640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae40 0x7fb9e806d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc050020 tx=0x7fb9f0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:46.063 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.057+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/3645217058 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9ec075a80 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.232+0000 7fba02933640 1 -- 172.21.15.60:0/3645217058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9fc069400 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.233+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/3645217058 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb9ec0790a0 con 0x7fb9fc1f2cd0 2023-11-18T20:43:46.234 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.234+0000 7fba02933640 1 -- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae40 msgr2=0x7fb9e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.234+0000 7fba02933640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae40 0x7fb9e806d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc050020 tx=0x7fb9f0008040 comp rx=0 tx=0).stop 2023-11-18T20:43:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.234+0000 7fba02933640 1 -- 172.21.15.60:0/3645217058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc1f2cd0 msgr2=0x7fb9fc15a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.234+0000 7fba02933640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc1f2cd0 0x7fb9fc15a140 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec02c470 tx=0x7fb9ec02e070 comp rx=0 tx=0).stop 2023-11-18T20:43:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.234+0000 7fba02933640 1 -- 172.21.15.60:0/3645217058 shutdown_connections 2023-11-18T20:43:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.234+0000 7fba02933640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae40 0x7fb9e806d300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.234+0000 7fba02933640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc1f3150 0x7fb9fc15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.234+0000 7fba02933640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc1f2cd0 0x7fb9fc15a140 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.234+0000 7fba02933640 1 --2- 172.21.15.60:0/3645217058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9e4004370 0x7fb9fc1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.234+0000 7fba02933640 1 -- 172.21.15.60:0/3645217058 >> 172.21.15.60:0/3645217058 conn(0x7fb9fc06a6b0 msgr2=0x7fb9fc13fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:46.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.235+0000 7fba02933640 1 -- 172.21.15.60:0/3645217058 shutdown_connections 2023-11-18T20:43:46.237 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.235+0000 7fba02933640 1 -- 172.21.15.60:0/3645217058 wait complete. 2023-11-18T20:43:46.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-11-18T20:43:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.354+0000 7f9df37c9640 1 Processor -- start 2023-11-18T20:43:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.354+0000 7f9df37c9640 1 -- start start 2023-11-18T20:43:46.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.355+0000 7f9df37c9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9dec160fb0 0x7f9dec161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.355+0000 7f9df37c9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dec161d20 con 0x7f9dec0d0ed0 2023-11-18T20:43:46.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.355+0000 7f9df37c9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dec15e760 con 0x7f9dec06ea50 2023-11-18T20:43:46.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.355+0000 7f9df37c9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dec15d9a0 con 0x7f9dec160fb0 2023-11-18T20:43:46.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.355+0000 7f9df0d64640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9dec160fb0 0x7f9dec161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.355+0000 7f9df1d66640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9dec0d0ed0 0x7f9dec0d1320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37554/0 (socket says 172.21.15.60:37554) 2023-11-18T20:43:46.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.355+0000 7f9df1d66640 1 -- 172.21.15.60:0/3240550447 learned_addr learned my addr 172.21.15.60:0/3240550447 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:46.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.355+0000 7f9df0d64640 1 -- 172.21.15.60:0/3240550447 >> v1:172.21.15.100:6789/0 conn(0x7f9dec06ea50 legacy=0x7f9dec06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.355+0000 7f9df0d64640 1 -- 172.21.15.60:0/3240550447 >> v1:172.21.15.60:6789/0 conn(0x7f9dec0d0ed0 legacy=0x7f9dec0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.356+0000 7f9df0d64640 1 -- 172.21.15.60:0/3240550447 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec15dc80 con 0x7f9dec160fb0 2023-11-18T20:43:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.356+0000 7f9debfff640 1 -- 172.21.15.60:0/3240550447 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904212345 0 0) 0x7f9dec161d20 con 0x7f9dec0d0ed0 2023-11-18T20:43:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.356+0000 7f9df0d64640 1 --2- 172.21.15.60:0/3240550447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec160fb0 0x7f9dec161440 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9de00097a0 tx=0x7f9de002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc69367d8d89a19 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.356+0000 7f9debfff640 1 -- 172.21.15.60:0/3240550447 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0031830 con 0x7f9dec160fb0 2023-11-18T20:43:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.356+0000 7f9debfff640 1 -- 172.21.15.60:0/3240550447 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de0031990 con 0x7f9dec160fb0 2023-11-18T20:43:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.356+0000 7f9debfff640 1 -- 172.21.15.60:0/3240550447 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0031d20 con 0x7f9dec160fb0 2023-11-18T20:43:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.357+0000 7f9df37c9640 1 -- 172.21.15.60:0/3240550447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec160fb0 msgr2=0x7f9dec161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.357+0000 7f9df37c9640 1 --2- 172.21.15.60:0/3240550447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec160fb0 0x7f9dec161440 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9de00097a0 tx=0x7f9de002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.357+0000 7f9df37c9640 1 -- 172.21.15.60:0/3240550447 shutdown_connections 2023-11-18T20:43:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.357+0000 7f9df37c9640 1 --2- 172.21.15.60:0/3240550447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec160fb0 0x7f9dec161440 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.357+0000 7f9df37c9640 1 -- 172.21.15.60:0/3240550447 >> 172.21.15.60:0/3240550447 conn(0x7f9dec06a6b0 msgr2=0x7f9dec06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.357+0000 7f9df37c9640 1 -- 172.21.15.60:0/3240550447 shutdown_connections 2023-11-18T20:43:46.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.357+0000 7f9df37c9640 1 -- 172.21.15.60:0/3240550447 wait complete. 2023-11-18T20:43:46.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.358+0000 7f9df37c9640 1 Processor -- start 2023-11-18T20:43:46.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.358+0000 7f9df37c9640 1 -- start start 2023-11-18T20:43:46.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df37c9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec06ea50 0x7f9dec156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df37c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dec0d0ed0 0x7f9dec1e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df37c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dec160fb0 0x7f9dec15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df37c9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec1e7e00 con 0x7f9dec06ea50 2023-11-18T20:43:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df37c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec1e7180 con 0x7f9dec0d0ed0 2023-11-18T20:43:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df37c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec1e7460 con 0x7f9dec160fb0 2023-11-18T20:43:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df1565640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec06ea50 0x7f9dec156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df1565640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec06ea50 0x7f9dec156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57518/0 (socket says 172.21.15.60:57518) 2023-11-18T20:43:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df1565640 1 -- 172.21.15.60:0/704112608 learned_addr learned my addr 172.21.15.60:0/704112608 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df1d66640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dec160fb0 0x7f9dec15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df0d64640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dec0d0ed0 0x7f9dec1e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.359+0000 7f9df1565640 1 -- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dec160fb0 msgr2=0x7f9dec15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9df1565640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dec160fb0 0x7f9dec15a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9df1565640 1 -- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dec0d0ed0 msgr2=0x7f9dec1e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9df1565640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dec0d0ed0 0x7f9dec1e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9df1565640 1 -- 172.21.15.60:0/704112608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de0009400 con 0x7f9dec06ea50 2023-11-18T20:43:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9df1d66640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dec160fb0 0x7f9dec15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9df0d64640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dec0d0ed0 0x7f9dec1e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9df1565640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec06ea50 0x7f9dec156b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c400 tx=0x7f9de400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9de9ffb640 1 -- 172.21.15.60:0/704112608 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40052d0 con 0x7f9dec06ea50 2023-11-18T20:43:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9de9ffb640 1 -- 172.21.15.60:0/704112608 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de4005430 con 0x7f9dec06ea50 2023-11-18T20:43:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9df37c9640 1 -- 172.21.15.60:0/704112608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec1e7740 con 0x7f9dec06ea50 2023-11-18T20:43:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9de9ffb640 1 -- 172.21.15.60:0/704112608 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4003830 con 0x7f9dec06ea50 2023-11-18T20:43:46.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.360+0000 7f9df37c9640 1 -- 172.21.15.60:0/704112608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec1e6de0 con 0x7f9dec06ea50 2023-11-18T20:43:46.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.361+0000 7f9df37c9640 1 -- 172.21.15.60:0/704112608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec069400 con 0x7f9dec06ea50 2023-11-18T20:43:46.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.361+0000 7f9de9ffb640 1 -- 172.21.15.60:0/704112608 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9de4003990 con 0x7f9dec06ea50 2023-11-18T20:43:46.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.362+0000 7f9de9ffb640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9dd806ae70 0x7f9dd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.362+0000 7f9de9ffb640 1 -- 172.21.15.60:0/704112608 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f9de4088c70 con 0x7f9dec06ea50 2023-11-18T20:43:46.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.362+0000 7f9df0d64640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9dd806ae70 0x7f9dd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.363+0000 7f9df0d64640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9dd806ae70 0x7f9dd806d330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9de0036000 tx=0x7f9de0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:46.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.366+0000 7f9de9ffb640 1 -- 172.21.15.60:0/704112608 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9de4053c20 con 0x7f9dec06ea50 2023-11-18T20:43:46.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.542+0000 7f9df37c9640 1 -- 172.21.15.60:0/704112608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f9dec070d30 con 0x7f9dec06ea50 2023-11-18T20:43:46.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.542+0000 7f9de9ffb640 1 -- 172.21.15.60:0/704112608 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7f9de40170a0 con 0x7f9dec06ea50 2023-11-18T20:43:46.544 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:46.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 -- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9dd806ae70 msgr2=0x7f9dd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9dd806ae70 0x7f9dd806d330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9de0036000 tx=0x7f9de0008030 comp rx=0 tx=0).stop 2023-11-18T20:43:46.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 -- 172.21.15.60:0/704112608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec06ea50 msgr2=0x7f9dec156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec06ea50 0x7f9dec156b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c400 tx=0x7f9de400c8c0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 -- 172.21.15.60:0/704112608 shutdown_connections 2023-11-18T20:43:46.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9dd806ae70 0x7f9dd806d330 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dec160fb0 0x7f9dec15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dec0d0ed0 0x7f9dec1e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 --2- 172.21.15.60:0/704112608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dec06ea50 0x7f9dec156b90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 -- 172.21.15.60:0/704112608 >> 172.21.15.60:0/704112608 conn(0x7f9dec06a6b0 msgr2=0x7f9dec13fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:46.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 -- 172.21.15.60:0/704112608 shutdown_connections 2023-11-18T20:43:46.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.544+0000 7f9df37c9640 1 -- 172.21.15.60:0/704112608 wait complete. 2023-11-18T20:43:46.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:46.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.662+0000 7f2306821640 1 Processor -- start 2023-11-18T20:43:46.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.662+0000 7f2306821640 1 -- start start 2023-11-18T20:43:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.663+0000 7f2306821640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d2910 0x7f23000d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.663+0000 7f2306821640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f23000d3730 0x7f23000b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.663+0000 7f2306821640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23000b2b10 con 0x7f230015d330 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.663+0000 7f2306821640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23000d3b50 con 0x7f23000d3730 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.663+0000 7f2306821640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23000d1590 con 0x7f23000d2910 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.663+0000 7f22ff7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d2910 0x7f23000d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.663+0000 7f2304dbe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f23000d3730 0x7f23000b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.663+0000 7f22ff7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d2910 0x7f23000d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:35248/0 (socket says 172.21.15.60:35248) 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.663+0000 7f22ff7fe640 1 -- 172.21.15.60:0/3483748797 learned_addr learned my addr 172.21.15.60:0/3483748797 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.663+0000 7f22fffff640 1 --1- 172.21.15.60:0/3483748797 >> v1:172.21.15.100:6789/0 conn(0x7f230015d330 0x7f23000b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:33866/0 (socket says 172.21.15.60:33866) 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.664+0000 7f22ff7fe640 1 -- 172.21.15.60:0/3483748797 >> v2:172.21.15.100:3300/0 conn(0x7f23000d3730 msgr2=0x7f23000b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.664+0000 7f22ff7fe640 1 --2- 172.21.15.60:0/3483748797 >> v2:172.21.15.100:3300/0 conn(0x7f23000d3730 0x7f23000b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.664+0000 7f22ff7fe640 1 -- 172.21.15.60:0/3483748797 >> v1:172.21.15.100:6789/0 conn(0x7f230015d330 legacy=0x7f23000b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.664+0000 7f22ff7fe640 1 -- 172.21.15.60:0/3483748797 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23000d1870 con 0x7f23000d2910 2023-11-18T20:43:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.664+0000 7f22feffd640 1 -- 172.21.15.60:0/3483748797 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347500801 0 0) 0x7f23000b2b10 con 0x7f230015d330 2023-11-18T20:43:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.664+0000 7f2304dbe640 1 --2- 172.21.15.60:0/3483748797 >> v2:172.21.15.100:3300/0 conn(0x7f23000d3730 0x7f23000b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.664+0000 7f22ff7fe640 1 --2- 172.21.15.60:0/3483748797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d2910 0x7f23000d31f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009870 tx=0x7f22f002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad19dac5c46f0c56 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f22feffd640 1 -- 172.21.15.60:0/3483748797 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0031640 con 0x7f23000d2910 2023-11-18T20:43:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f22feffd640 1 -- 172.21.15.60:0/3483748797 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f00317a0 con 0x7f23000d2910 2023-11-18T20:43:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f22feffd640 1 -- 172.21.15.60:0/3483748797 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0031b30 con 0x7f23000d2910 2023-11-18T20:43:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f2306821640 1 -- 172.21.15.60:0/3483748797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d2910 msgr2=0x7f23000d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f2306821640 1 --2- 172.21.15.60:0/3483748797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d2910 0x7f23000d31f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009870 tx=0x7f22f002b8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f2306821640 1 -- 172.21.15.60:0/3483748797 shutdown_connections 2023-11-18T20:43:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f2306821640 1 --2- 172.21.15.60:0/3483748797 >> v2:172.21.15.100:3300/0 conn(0x7f23000d3730 0x7f23000b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f2306821640 1 --2- 172.21.15.60:0/3483748797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d2910 0x7f23000d31f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f2306821640 1 -- 172.21.15.60:0/3483748797 >> 172.21.15.60:0/3483748797 conn(0x7f2300072020 msgr2=0x7f2300072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f2306821640 1 -- 172.21.15.60:0/3483748797 shutdown_connections 2023-11-18T20:43:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.665+0000 7f2306821640 1 -- 172.21.15.60:0/3483748797 wait complete. 2023-11-18T20:43:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.666+0000 7f2306821640 1 Processor -- start 2023-11-18T20:43:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.666+0000 7f2306821640 1 -- start start 2023-11-18T20:43:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.667+0000 7f2306821640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23000d2910 0x7f230014e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.667+0000 7f2306821640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d3730 0x7f230015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.667+0000 7f2306821640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f230015d330 0x7f230015f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.667+0000 7f2306821640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23000c95f0 con 0x7f230015d330 2023-11-18T20:43:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.667+0000 7f2306821640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230014ec40 con 0x7f23000d2910 2023-11-18T20:43:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.667+0000 7f2306821640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23000c8970 con 0x7f23000d3730 2023-11-18T20:43:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.667+0000 7f2304dbe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f230015d330 0x7f230015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.667+0000 7f2304dbe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f230015d330 0x7f230015f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57530/0 (socket says 172.21.15.60:57530) 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.667+0000 7f2304dbe640 1 -- 172.21.15.60:0/4015074109 learned_addr learned my addr 172.21.15.60:0/4015074109 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f22fffff640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23000d2910 0x7f230014e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f22ff7fe640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d3730 0x7f230015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f2304dbe640 1 -- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d3730 msgr2=0x7f230015f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f2304dbe640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d3730 0x7f230015f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f2304dbe640 1 -- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23000d2910 msgr2=0x7f230014e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f2304dbe640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23000d2910 0x7f230014e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f2304dbe640 1 -- 172.21.15.60:0/4015074109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8009a70 con 0x7f230015d330 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f2304dbe640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f230015d330 0x7f230015f800 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f22f800efc0 tx=0x7f22f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f22fcff9640 1 -- 172.21.15.60:0/4015074109 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8009cd0 con 0x7f230015d330 2023-11-18T20:43:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f22fcff9640 1 -- 172.21.15.60:0/4015074109 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f800f040 con 0x7f230015d330 2023-11-18T20:43:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f2306821640 1 -- 172.21.15.60:0/4015074109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f00094d0 con 0x7f230015d330 2023-11-18T20:43:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f22fcff9640 1 -- 172.21.15.60:0/4015074109 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8003990 con 0x7f230015d330 2023-11-18T20:43:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.668+0000 7f2306821640 1 -- 172.21.15.60:0/4015074109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23000c8f30 con 0x7f230015d330 2023-11-18T20:43:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.669+0000 7f2306821640 1 -- 172.21.15.60:0/4015074109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2300069400 con 0x7f230015d330 2023-11-18T20:43:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.670+0000 7f22fcff9640 1 -- 172.21.15.60:0/4015074109 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22f8014480 con 0x7f230015d330 2023-11-18T20:43:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.670+0000 7f22fcff9640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22e006ae40 0x7f22e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.670+0000 7f22fcff9640 1 -- 172.21.15.60:0/4015074109 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f22f808a250 con 0x7f230015d330 2023-11-18T20:43:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.670+0000 7f22fffff640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22e006ae40 0x7f22e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.671+0000 7f22fffff640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22e006ae40 0x7f22e006d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f22e8000f60 tx=0x7f22e8007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.675+0000 7f22fcff9640 1 -- 172.21.15.60:0/4015074109 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22f8055220 con 0x7f230015d330 2023-11-18T20:43:46.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.865+0000 7f2306821640 1 -- 172.21.15.60:0/4015074109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f23000b3aa0 con 0x7f230015d330 2023-11-18T20:43:46.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.866+0000 7f22fcff9640 1 -- 172.21.15.60:0/4015074109 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f22f8058840 con 0x7f230015d330 2023-11-18T20:43:46.867 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:46.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 -- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22e006ae40 msgr2=0x7f22e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22e006ae40 0x7f22e006d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f22e8000f60 tx=0x7f22e8007d50 comp rx=0 tx=0).stop 2023-11-18T20:43:46.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 -- 172.21.15.60:0/4015074109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f230015d330 msgr2=0x7f230015f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:46.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f230015d330 0x7f230015f800 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f22f800efc0 tx=0x7f22f800c490 comp rx=0 tx=0).stop 2023-11-18T20:43:46.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 -- 172.21.15.60:0/4015074109 shutdown_connections 2023-11-18T20:43:46.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f22e006ae40 0x7f22e006d300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f230015d330 0x7f230015f800 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f23000d3730 0x7f230015f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 --2- 172.21.15.60:0/4015074109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23000d2910 0x7f230014e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:46.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 -- 172.21.15.60:0/4015074109 >> 172.21.15.60:0/4015074109 conn(0x7f2300072020 msgr2=0x7f23000cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:46.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 -- 172.21.15.60:0/4015074109 shutdown_connections 2023-11-18T20:43:46.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:46.868+0000 7f2306821640 1 -- 172.21.15.60:0/4015074109 wait complete. 2023-11-18T20:43:46.880 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-11-18T20:43:46.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:46.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:46.910 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[101199]: fuse finished with error 0 and tester_r 0 2023-11-18T20:43:46.911 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:43:53.013 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:43:53.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:53.013 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-18T20:43:53.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-18T20:43:53.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.190+0000 7fe0d458a640 1 Processor -- start 2023-11-18T20:43:53.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.190+0000 7fe0d458a640 1 -- start start 2023-11-18T20:43:53.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d458a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0cc143490 0x7fe0cc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:53.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d458a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0cc15e730 0x7fe0cc0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:53.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d458a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe0cc15a4e0 0x7fe0cc15a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:53.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d458a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0cc15b520 con 0x7fe0cc143490 2023-11-18T20:43:53.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d458a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0cc143870 con 0x7fe0cc15a4e0 2023-11-18T20:43:53.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d458a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc15e2a0 con 0x7fe0cc15e730 2023-11-18T20:43:53.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d2326640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0cc143490 0x7fe0cc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:53.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d2326640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0cc143490 0x7fe0cc0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48132/0 (socket says 172.21.15.60:48132) 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d2326640 1 -- 172.21.15.60:0/3115864182 learned_addr learned my addr 172.21.15.60:0/3115864182 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d1b25640 1 --2- 172.21.15.60:0/3115864182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0cc15e730 0x7fe0cc0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d2b27640 1 --2- 172.21.15.60:0/3115864182 >> v2:172.21.15.100:3300/0 conn(0x7fe0cc15a4e0 0x7fe0cc15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d2326640 1 -- 172.21.15.60:0/3115864182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0cc15e730 msgr2=0x7fe0cc0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d2326640 1 --2- 172.21.15.60:0/3115864182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0cc15e730 0x7fe0cc0b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d2326640 1 -- 172.21.15.60:0/3115864182 >> v2:172.21.15.100:3300/0 conn(0x7fe0cc15a4e0 msgr2=0x7fe0cc15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d2326640 1 --2- 172.21.15.60:0/3115864182 >> v2:172.21.15.100:3300/0 conn(0x7fe0cc15a4e0 0x7fe0cc15a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.191+0000 7fe0d2326640 1 -- 172.21.15.60:0/3115864182 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0cc15b240 con 0x7fe0cc143490 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.192+0000 7fe0d2326640 1 --2- 172.21.15.60:0/3115864182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc143490 0x7fe0cc0b0cf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc15eb90 tx=0x7fe0c802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ee06eac65ba9c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d1324640 1 -- 172.21.15.60:0/3115864182 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c802ee90 con 0x7fe0cc143490 2023-11-18T20:43:53.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d1324640 1 -- 172.21.15.60:0/3115864182 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c8036440 con 0x7fe0cc143490 2023-11-18T20:43:53.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d1324640 1 -- 172.21.15.60:0/3115864182 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80367d0 con 0x7fe0cc143490 2023-11-18T20:43:53.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d458a640 1 -- 172.21.15.60:0/3115864182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc143490 msgr2=0x7fe0cc0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:53.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d458a640 1 --2- 172.21.15.60:0/3115864182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc143490 0x7fe0cc0b0cf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc15eb90 tx=0x7fe0c802e0e0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d458a640 1 -- 172.21.15.60:0/3115864182 shutdown_connections 2023-11-18T20:43:53.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d458a640 1 --2- 172.21.15.60:0/3115864182 >> v2:172.21.15.100:3300/0 conn(0x7fe0cc15a4e0 0x7fe0cc15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d458a640 1 --2- 172.21.15.60:0/3115864182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0cc15e730 0x7fe0cc0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d458a640 1 --2- 172.21.15.60:0/3115864182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc143490 0x7fe0cc0b0cf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d458a640 1 -- 172.21.15.60:0/3115864182 >> 172.21.15.60:0/3115864182 conn(0x7fe0cc06a4d0 msgr2=0x7fe0cc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d458a640 1 -- 172.21.15.60:0/3115864182 shutdown_connections 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.193+0000 7fe0d458a640 1 -- 172.21.15.60:0/3115864182 wait complete. 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.194+0000 7fe0d458a640 1 Processor -- start 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.194+0000 7fe0d458a640 1 -- start start 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d458a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0cc143490 0x7fe0cc1411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d458a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0cc15a4e0 0x7fe0cc0adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d458a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc15e730 0x7fe0cc1598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d458a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc13f1a0 con 0x7fe0cc15e730 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d458a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc141700 con 0x7fe0cc15a4e0 2023-11-18T20:43:53.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d458a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc140120 con 0x7fe0cc143490 2023-11-18T20:43:53.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d2b27640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc15e730 0x7fe0cc1598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:53.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d2b27640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc15e730 0x7fe0cc1598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48138/0 (socket says 172.21.15.60:48138) 2023-11-18T20:43:53.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d2b27640 1 -- 172.21.15.60:0/3532316419 learned_addr learned my addr 172.21.15.60:0/3532316419 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:53.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d1b25640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0cc15a4e0 0x7fe0cc0adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:53.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.195+0000 7fe0d2326640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0cc143490 0x7fe0cc1411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:53.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0d2b27640 1 -- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0cc143490 msgr2=0x7fe0cc1411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:53.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0d2b27640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0cc143490 0x7fe0cc1411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0d2b27640 1 -- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0cc15a4e0 msgr2=0x7fe0cc0adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:53.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0d2b27640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0cc15a4e0 0x7fe0cc0adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0d2b27640 1 -- 172.21.15.60:0/3532316419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c800b770 con 0x7fe0cc15e730 2023-11-18T20:43:53.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0d2b27640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc15e730 0x7fe0cc1598d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4007e40 tx=0x7fe0c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:53.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0c2ffd640 1 -- 172.21.15.60:0/3532316419 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4014620 con 0x7fe0cc15e730 2023-11-18T20:43:53.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0c2ffd640 1 -- 172.21.15.60:0/3532316419 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c400ee90 con 0x7fe0cc15e730 2023-11-18T20:43:53.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0d458a640 1 -- 172.21.15.60:0/3532316419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0cc140400 con 0x7fe0cc15e730 2023-11-18T20:43:53.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0c2ffd640 1 -- 172.21.15.60:0/3532316419 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40149e0 con 0x7fe0cc15e730 2023-11-18T20:43:53.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.196+0000 7fe0d458a640 1 -- 172.21.15.60:0/3532316419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0cc1409c0 con 0x7fe0cc15e730 2023-11-18T20:43:53.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.197+0000 7fe0d458a640 1 -- 172.21.15.60:0/3532316419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0cc0b4380 con 0x7fe0cc15e730 2023-11-18T20:43:53.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.198+0000 7fe0c2ffd640 1 -- 172.21.15.60:0/3532316419 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe0c4014b40 con 0x7fe0cc15e730 2023-11-18T20:43:53.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.198+0000 7fe0c2ffd640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe0b006ae40 0x7fe0b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:53.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.198+0000 7fe0c2ffd640 1 -- 172.21.15.60:0/3532316419 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe0c408a070 con 0x7fe0cc15e730 2023-11-18T20:43:53.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.198+0000 7fe0d2326640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe0b006ae40 0x7fe0b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:53.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.199+0000 7fe0d2326640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe0b006ae40 0x7fe0b006d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8038000 tx=0x7fe0c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:53.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.202+0000 7fe0c2ffd640 1 -- 172.21.15.60:0/3532316419 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0c4054fb0 con 0x7fe0cc15e730 2023-11-18T20:43:53.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.417+0000 7fe0d458a640 1 -- 172.21.15.60:0/3532316419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe0cc069400 con 0x7fe0cc15e730 2023-11-18T20:43:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.417+0000 7fe0c2ffd640 1 -- 172.21.15.60:0/3532316419 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+82 (secure 0 0 0) 0x7fe0c40585d0 con 0x7fe0cc15e730 2023-11-18T20:43:53.419 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2023-11-18T20:43:53.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 -- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe0b006ae40 msgr2=0x7fe0b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:53.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe0b006ae40 0x7fe0b006d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8038000 tx=0x7fe0c8008040 comp rx=0 tx=0).stop 2023-11-18T20:43:53.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 -- 172.21.15.60:0/3532316419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc15e730 msgr2=0x7fe0cc1598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:53.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc15e730 0x7fe0cc1598d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4007e40 tx=0x7fe0c400e4b0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 -- 172.21.15.60:0/3532316419 shutdown_connections 2023-11-18T20:43:53.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe0b006ae40 0x7fe0b006d300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0cc15e730 0x7fe0cc1598d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0cc15a4e0 0x7fe0cc0adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 --2- 172.21.15.60:0/3532316419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0cc143490 0x7fe0cc1411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:53.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 -- 172.21.15.60:0/3532316419 >> 172.21.15.60:0/3532316419 conn(0x7fe0cc06a4d0 msgr2=0x7fe0cc071a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:53.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 -- 172.21.15.60:0/3532316419 shutdown_connections 2023-11-18T20:43:53.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:53.419+0000 7fe0d458a640 1 -- 172.21.15.60:0/3532316419 wait complete. 2023-11-18T20:43:53.431 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-18T20:43:53.431 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-18T20:43:53.431 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-11-18T20:43:53.431 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-18T20:43:53.432 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:53.432 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-18T20:43:53.432 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-18T20:43:53.432 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-18T20:43:53.432 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:43:53.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:53.433 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-11-18T20:43:53.449 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:43:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:43:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:43:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-11-18T20:43:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:53.449 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:43:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42623sec preferred_lft 42623sec 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2023-11-18T20:43:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:53.451 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:43:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:43:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-11-18T20:43:53.451 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:43:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:43:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-18T20:43:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::d896:cfff:fef3:88dc/64 scope link 2023-11-18T20:43:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:53.451 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:43:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0a:c3:0c:c2:e6:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:43:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8c3:cff:fec2:e672/64 scope link 2023-11-18T20:43:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:53.452 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:43:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:43:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::40c:deff:fe9d:c1f6/64 scope link 2023-11-18T20:43:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:43:53.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:53.452 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-18T20:43:53.518 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-11-18T20:43:53.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:53.520 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-11-18T20:43:53.575 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-11-18T20:43:53.576 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-18T20:43:53.576 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-18T20:43:53.576 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-18T20:43:53.576 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-18T20:43:53.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:53.576 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:43:53.576 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:43:53.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-18T20:43:53.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-18T20:43:53.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-18T20:43:53.577 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:43:53.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:53.682 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-11-18T20:43:53.697 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-18T20:43:53.697 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-18T20:43:53.697 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:43:53.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:53.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-11-18T20:43:53.768 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-18T20:43:53.768 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:43:53.768 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:53.823 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:43:53.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:43:53.823 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:53.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:43:53.949 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:43:54.005 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:43:54.005 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:43:54.005 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:43:54.006 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:43:54.007 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:43:54.008 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:43:54.008 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:43:54.008 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:43:54.008 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:43:54.008 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:43:54.008 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:43:54.008 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:43:54.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:54.009 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:43:54.064 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-18T20:43:54.064 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-18T20:43:54.106 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:43:54.137 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:43:54.193 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:43:54.193 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:43:54.193 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:43:54.193 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:43:54.193 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:43:54.193 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:43:54.194 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:43:54.195 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:43:54.196 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:43:54.196 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:43:54.196 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:43:54.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:54.196 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:43:54.200 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-11-18T20:43:54.199+0000 7f911aa47480 -1 init, newargv = 0x5612c9b38690 newargc=15 2023-11-18T20:43:54.200 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[102433]: starting ceph client 2023-11-18T20:43:54.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[102433]: starting fuse 2023-11-18T20:43:54.216 INFO:teuthology.orchestra.run.smithi060.stdout:41 2023-11-18T20:43:54.216 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-18T20:43:54.216 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2023-11-18T20:43:54.216 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-11-18T20:43:54.216 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:43:54.217 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-11-18T20:43:54.218 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-18T20:43:54.218 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-18T20:43:54.218 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-18T20:43:54.218 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-11-18T20:43:54.218 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-11-18T20:43:54.218 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-11-18T20:43:54.218 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-18T20:43:54.218 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:43:54.218 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-11-18T20:43:54.218 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-11-18T20:43:54.341 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.102433.asok 2023-11-18T20:43:54.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:43:54.352 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.102433.asok 2023-11-18T20:43:54.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:54.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.102433.asok status 2023-11-18T20:43:54.494 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:43:54.494 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-11-18T20:43:54.494 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe", 2023-11-18T20:43:54.494 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-7418-gb0f45a51 (b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe) reef (dev)", 2023-11-18T20:43:54.494 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-11-18T20:43:54.494 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "102433", 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5559, 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 5559 2023-11-18T20:43:54.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 414608041 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 414608041 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.5559 192.168.0.1:0/414608041", 2023-11-18T20:43:54.496 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/414608041", 2023-11-18T20:43:54.497 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-11-18T20:43:54.497 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 14, 2023-11-18T20:43:54.497 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 37, 2023-11-18T20:43:54.497 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-11-18T20:43:54.497 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-11-18T20:43:54.497 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-11-18T20:43:54.497 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:43:54.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:54.503 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-11-18T20:43:54.503 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:54.518 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-11-18T20:43:54.519 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:54.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:54.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:54.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:54.590 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-11-18T20:43:54.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:43:54.651 INFO:teuthology.orchestra.run.smithi060.stderr:setfattr: layoutfile: Permission denied 2023-11-18T20:43:54.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:54.651 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-11-18T20:43:54.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:43:54.707 INFO:teuthology.orchestra.run.smithi060.stderr:setfattr: layoutdir: Permission denied 2023-11-18T20:43:54.708 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:43:54.708 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:54.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:54.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.909+0000 7fef93fa9640 1 Processor -- start 2023-11-18T20:43:54.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.909+0000 7fef93fa9640 1 -- start start 2023-11-18T20:43:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.910+0000 7fef93fa9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef8c142270 0x7fef8c1426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.910+0000 7fef93fa9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c161d20 con 0x7fef8c06ea50 2023-11-18T20:43:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.910+0000 7fef93fa9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c15e760 con 0x7fef8c160fb0 2023-11-18T20:43:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.910+0000 7fef93fa9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8c0d1ee0 con 0x7fef8c142270 2023-11-18T20:43:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.910+0000 7fef92546640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef8c142270 0x7fef8c1426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.910+0000 7fef91d45640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fef8c06ea50 0x7fef8c06ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46388/0 (socket says 172.21.15.60:46388) 2023-11-18T20:43:54.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.910+0000 7fef91d45640 1 -- 172.21.15.60:0/2475821580 learned_addr learned my addr 172.21.15.60:0/2475821580 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:54.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.910+0000 7fef92546640 1 -- 172.21.15.60:0/2475821580 >> v1:172.21.15.100:6789/0 conn(0x7fef8c160fb0 legacy=0x7fef8c161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:54.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.910+0000 7fef92546640 1 -- 172.21.15.60:0/2475821580 >> v1:172.21.15.60:6789/0 conn(0x7fef8c06ea50 legacy=0x7fef8c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:54.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.910+0000 7fef92546640 1 -- 172.21.15.60:0/2475821580 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c15d9a0 con 0x7fef8c142270 2023-11-18T20:43:54.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.911+0000 7fef92546640 1 --2- 172.21.15.60:0/2475821580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c142270 0x7fef8c1426c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fef88009d90 tx=0x7fef8802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557465d343d7d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:54.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.911+0000 7fef90d43640 1 -- 172.21.15.60:0/2475821580 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8802cbd0 con 0x7fef8c142270 2023-11-18T20:43:54.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.911+0000 7fef90d43640 1 -- 172.21.15.60:0/2475821580 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8802cd30 con 0x7fef8c142270 2023-11-18T20:43:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.911+0000 7fef90d43640 1 -- 172.21.15.60:0/2475821580 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8802cea0 con 0x7fef8c142270 2023-11-18T20:43:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.912+0000 7fef93fa9640 1 -- 172.21.15.60:0/2475821580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c142270 msgr2=0x7fef8c1426c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.912+0000 7fef93fa9640 1 --2- 172.21.15.60:0/2475821580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c142270 0x7fef8c1426c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fef88009d90 tx=0x7fef8802be20 comp rx=0 tx=0).stop 2023-11-18T20:43:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.912+0000 7fef93fa9640 1 -- 172.21.15.60:0/2475821580 shutdown_connections 2023-11-18T20:43:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.912+0000 7fef93fa9640 1 --2- 172.21.15.60:0/2475821580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c142270 0x7fef8c1426c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.912+0000 7fef93fa9640 1 -- 172.21.15.60:0/2475821580 >> 172.21.15.60:0/2475821580 conn(0x7fef8c06a6b0 msgr2=0x7fef8c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:54.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.912+0000 7fef93fa9640 1 -- 172.21.15.60:0/2475821580 shutdown_connections 2023-11-18T20:43:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.912+0000 7fef93fa9640 1 -- 172.21.15.60:0/2475821580 wait complete. 2023-11-18T20:43:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.913+0000 7fef93fa9640 1 Processor -- start 2023-11-18T20:43:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.913+0000 7fef93fa9640 1 -- start start 2023-11-18T20:43:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef93fa9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06ea50 0x7fef8c156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef93fa9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c142270 0x7fef8c155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef93fa9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c160fb0 0x7fef8c0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef93fa9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1e81a0 con 0x7fef8c06ea50 2023-11-18T20:43:54.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef93fa9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1e7520 con 0x7fef8c142270 2023-11-18T20:43:54.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef93fa9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1e7800 con 0x7fef8c160fb0 2023-11-18T20:43:54.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91d45640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06ea50 0x7fef8c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:54.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91d45640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06ea50 0x7fef8c156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48154/0 (socket says 172.21.15.60:48154) 2023-11-18T20:43:54.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91d45640 1 -- 172.21.15.60:0/160721376 learned_addr learned my addr 172.21.15.60:0/160721376 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:54.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91d45640 1 -- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c160fb0 msgr2=0x7fef8c0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:54.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91d45640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c160fb0 0x7fef8c0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:54.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91d45640 1 -- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c142270 msgr2=0x7fef8c155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:54.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91544640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c142270 0x7fef8c155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91d45640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c142270 0x7fef8c155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91d45640 1 -- 172.21.15.60:0/160721376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef88009a70 con 0x7fef8c06ea50 2023-11-18T20:43:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91544640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c142270 0x7fef8c155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.914+0000 7fef91d45640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06ea50 0x7fef8c156390 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c480 tx=0x7fef8400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.915+0000 7fef827fc640 1 -- 172.21.15.60:0/160721376 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84005ce0 con 0x7fef8c06ea50 2023-11-18T20:43:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.915+0000 7fef827fc640 1 -- 172.21.15.60:0/160721376 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84005e40 con 0x7fef8c06ea50 2023-11-18T20:43:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.915+0000 7fef93fa9640 1 -- 172.21.15.60:0/160721376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c1e7ae0 con 0x7fef8c06ea50 2023-11-18T20:43:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.915+0000 7fef827fc640 1 -- 172.21.15.60:0/160721376 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840072f0 con 0x7fef8c06ea50 2023-11-18T20:43:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.915+0000 7fef93fa9640 1 -- 172.21.15.60:0/160721376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c1e7060 con 0x7fef8c06ea50 2023-11-18T20:43:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.916+0000 7fef93fa9640 1 -- 172.21.15.60:0/160721376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c1e82e0 con 0x7fef8c06ea50 2023-11-18T20:43:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.916+0000 7fef827fc640 1 -- 172.21.15.60:0/160721376 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef84007450 con 0x7fef8c06ea50 2023-11-18T20:43:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.916+0000 7fef827fc640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fef7006ae40 0x7fef7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.916+0000 7fef827fc640 1 -- 172.21.15.60:0/160721376 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fef84088b60 con 0x7fef8c06ea50 2023-11-18T20:43:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.917+0000 7fef91544640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fef7006ae40 0x7fef7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.917+0000 7fef91544640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fef7006ae40 0x7fef7006d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0d0ab0 tx=0x7fef7c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:54.921+0000 7fef827fc640 1 -- 172.21.15.60:0/160721376 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef8c1e82e0 con 0x7fef8c06ea50 2023-11-18T20:43:55.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.099+0000 7fef93fa9640 1 -- 172.21.15.60:0/160721376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef8c1e82e0 con 0x7fef8c06ea50 2023-11-18T20:43:55.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.100+0000 7fef827fc640 1 -- 172.21.15.60:0/160721376 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18336 (secure 0 0 0) 0x7fef8c1e82e0 con 0x7fef8c06ea50 2023-11-18T20:43:55.102 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:55.102 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:42.814654+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-11-18T20:43:40.650315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:55.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 -- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fef7006ae40 msgr2=0x7fef7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fef7006ae40 0x7fef7006d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0d0ab0 tx=0x7fef7c009040 comp rx=0 tx=0).stop 2023-11-18T20:43:55.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 -- 172.21.15.60:0/160721376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06ea50 msgr2=0x7fef8c156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06ea50 0x7fef8c156390 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c480 tx=0x7fef8400c940 comp rx=0 tx=0).stop 2023-11-18T20:43:55.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 -- 172.21.15.60:0/160721376 shutdown_connections 2023-11-18T20:43:55.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fef7006ae40 0x7fef7006d300 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c160fb0 0x7fef8c0b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c142270 0x7fef8c155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 --2- 172.21.15.60:0/160721376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06ea50 0x7fef8c156390 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 -- 172.21.15.60:0/160721376 >> 172.21.15.60:0/160721376 conn(0x7fef8c06a6b0 msgr2=0x7fef8c0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:55.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 -- 172.21.15.60:0/160721376 shutdown_connections 2023-11-18T20:43:55.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.102+0000 7fef93fa9640 1 -- 172.21.15.60:0/160721376 wait complete. 2023-11-18T20:43:55.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:43:31.703619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:43:32.121610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-11-18T20:43:40.650315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:55.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:55.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.223+0000 7f517076c640 1 Processor -- start 2023-11-18T20:43:55.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.223+0000 7f517076c640 1 -- start start 2023-11-18T20:43:55.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f517076c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f51680ad640 0x7f516806d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f517076c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f516806bbf0 0x7f516806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f517076c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51680adae0 con 0x7f516815d1d0 2023-11-18T20:43:55.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f517076c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51680addc0 con 0x7f51680ad640 2023-11-18T20:43:55.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f517076c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51680ae0a0 con 0x7f516806bbf0 2023-11-18T20:43:55.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f516ed09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f516806bbf0 0x7f516806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f516dd07640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f51680ad640 0x7f516806d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f516e508640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f516815d1d0 0x7f516806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48532/0 (socket says 172.21.15.60:48532) 2023-11-18T20:43:55.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f516e508640 1 -- 172.21.15.60:0/1546360161 learned_addr learned my addr 172.21.15.60:0/1546360161 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:55.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f516dd07640 1 -- 172.21.15.60:0/1546360161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f516806bbf0 msgr2=0x7f516806d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:55.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f516dd07640 1 --2- 172.21.15.60:0/1546360161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f516806bbf0 0x7f516806d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f516dd07640 1 -- 172.21.15.60:0/1546360161 >> v1:172.21.15.100:6789/0 conn(0x7f516815d1d0 legacy=0x7f516806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.224+0000 7f516dd07640 1 -- 172.21.15.60:0/1546360161 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51680ae380 con 0x7f51680ad640 2023-11-18T20:43:55.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.225+0000 7f516ed09640 1 --2- 172.21.15.60:0/1546360161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f516806bbf0 0x7f516806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:55.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.225+0000 7f516dd07640 1 --2- 172.21.15.60:0/1546360161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51680ad640 0x7f516806d0b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51580097a0 tx=0x7f515802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a89c22eca3c261 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f516d506640 1 -- 172.21.15.60:0/1546360161 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158031830 con 0x7f51680ad640 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f516d506640 1 -- 172.21.15.60:0/1546360161 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5158031990 con 0x7f51680ad640 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f516d506640 1 -- 172.21.15.60:0/1546360161 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158031d20 con 0x7f51680ad640 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f517076c640 1 -- 172.21.15.60:0/1546360161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51680ad640 msgr2=0x7f516806d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f517076c640 1 --2- 172.21.15.60:0/1546360161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51680ad640 0x7f516806d0b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51580097a0 tx=0x7f515802b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f517076c640 1 -- 172.21.15.60:0/1546360161 shutdown_connections 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f517076c640 1 --2- 172.21.15.60:0/1546360161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51680ad640 0x7f516806d0b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f517076c640 1 --2- 172.21.15.60:0/1546360161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f516806bbf0 0x7f516806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f517076c640 1 -- 172.21.15.60:0/1546360161 >> 172.21.15.60:0/1546360161 conn(0x7f5168070e70 msgr2=0x7f5168071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f517076c640 1 -- 172.21.15.60:0/1546360161 shutdown_connections 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.226+0000 7f517076c640 1 -- 172.21.15.60:0/1546360161 wait complete. 2023-11-18T20:43:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.227+0000 7f517076c640 1 Processor -- start 2023-11-18T20:43:55.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.227+0000 7f517076c640 1 -- start start 2023-11-18T20:43:55.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f517076c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516806bbf0 0x7f51681568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f517076c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51680ad640 0x7f51680cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f517076c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f516815d1d0 0x7f51680ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f517076c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51680cedb0 con 0x7f516815d1d0 2023-11-18T20:43:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f517076c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5168156430 con 0x7f516806bbf0 2023-11-18T20:43:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f517076c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51680cefe0 con 0x7f51680ad640 2023-11-18T20:43:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f516ed09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f516815d1d0 0x7f51680ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f516ed09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f516815d1d0 0x7f51680ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48170/0 (socket says 172.21.15.60:48170) 2023-11-18T20:43:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f516ed09640 1 -- 172.21.15.60:0/366699611 learned_addr learned my addr 172.21.15.60:0/366699611 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f516e508640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516806bbf0 0x7f51681568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f516dd07640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51680ad640 0x7f51680cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f516ed09640 1 -- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51680ad640 msgr2=0x7f51680cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f516ed09640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51680ad640 0x7f51680cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f516ed09640 1 -- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516806bbf0 msgr2=0x7f51681568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f516ed09640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516806bbf0 0x7f51681568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.228+0000 7f516ed09640 1 -- 172.21.15.60:0/366699611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160009a70 con 0x7f516815d1d0 2023-11-18T20:43:55.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.229+0000 7f516ed09640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f516815d1d0 0x7f51680ccdb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f516000efc0 tx=0x7f516000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:55.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.229+0000 7f515effd640 1 -- 172.21.15.60:0/366699611 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160014480 con 0x7f516815d1d0 2023-11-18T20:43:55.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.229+0000 7f515effd640 1 -- 172.21.15.60:0/366699611 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516000f040 con 0x7f516815d1d0 2023-11-18T20:43:55.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.229+0000 7f517076c640 1 -- 172.21.15.60:0/366699611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5158009400 con 0x7f516815d1d0 2023-11-18T20:43:55.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.229+0000 7f515effd640 1 -- 172.21.15.60:0/366699611 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160014840 con 0x7f516815d1d0 2023-11-18T20:43:55.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.229+0000 7f517076c640 1 -- 172.21.15.60:0/366699611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51681560c0 con 0x7f516815d1d0 2023-11-18T20:43:55.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.230+0000 7f517076c640 1 -- 172.21.15.60:0/366699611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51680cf500 con 0x7f516815d1d0 2023-11-18T20:43:55.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.230+0000 7f515effd640 1 -- 172.21.15.60:0/366699611 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51600149a0 con 0x7f516815d1d0 2023-11-18T20:43:55.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.231+0000 7f515effd640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f514c06ae40 0x7f514c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.231+0000 7f515effd640 1 -- 172.21.15.60:0/366699611 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f516008a770 con 0x7f516815d1d0 2023-11-18T20:43:55.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.231+0000 7f516e508640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f514c06ae40 0x7f514c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.235 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.231+0000 7f516e508640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f514c06ae40 0x7f514c06d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5164005880 tx=0x7f5164009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.235+0000 7f515effd640 1 -- 172.21.15.60:0/366699611 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5160055690 con 0x7f516815d1d0 2023-11-18T20:43:55.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.410+0000 7f517076c640 1 -- 172.21.15.60:0/366699611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f516804edd0 con 0x7f516815d1d0 2023-11-18T20:43:55.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.411+0000 7f515effd640 1 -- 172.21.15.60:0/366699611 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5160018020 con 0x7f516815d1d0 2023-11-18T20:43:55.412 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.412+0000 7f517076c640 1 -- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f514c06ae40 msgr2=0x7f514c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.412+0000 7f517076c640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f514c06ae40 0x7f514c06d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5164005880 tx=0x7f5164009040 comp rx=0 tx=0).stop 2023-11-18T20:43:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.412+0000 7f517076c640 1 -- 172.21.15.60:0/366699611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f516815d1d0 msgr2=0x7f51680ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.412+0000 7f517076c640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f516815d1d0 0x7f51680ccdb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f516000efc0 tx=0x7f516000c490 comp rx=0 tx=0).stop 2023-11-18T20:43:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.412+0000 7f517076c640 1 -- 172.21.15.60:0/366699611 shutdown_connections 2023-11-18T20:43:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.412+0000 7f517076c640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f514c06ae40 0x7f514c06d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.412+0000 7f517076c640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f516815d1d0 0x7f51680ccdb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.412+0000 7f517076c640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51680ad640 0x7f51680cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.412+0000 7f517076c640 1 --2- 172.21.15.60:0/366699611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f516806bbf0 0x7f51681568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.412+0000 7f517076c640 1 -- 172.21.15.60:0/366699611 >> 172.21.15.60:0/366699611 conn(0x7f5168070e70 msgr2=0x7f516806d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.413+0000 7f517076c640 1 -- 172.21.15.60:0/366699611 shutdown_connections 2023-11-18T20:43:55.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.413+0000 7f517076c640 1 -- 172.21.15.60:0/366699611 wait complete. 2023-11-18T20:43:55.424 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.532+0000 7f73a95c9640 1 Processor -- start 2023-11-18T20:43:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.532+0000 7f73a95c9640 1 -- start start 2023-11-18T20:43:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a95c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73a4144980 0x7f73a415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a95c9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f73a40ceb70 0x7f73a40cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a95c9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a4071770 con 0x7f73a40b4370 2023-11-18T20:43:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a95c9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a40ccdc0 con 0x7f73a40ceb70 2023-11-18T20:43:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a95c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40cd0a0 con 0x7f73a4144980 2023-11-18T20:43:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f73a40ceb70 0x7f73a40cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f73a40ceb70 0x7f73a40cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48180/0 (socket says 172.21.15.60:48180) 2023-11-18T20:43:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a27fc640 1 -- 172.21.15.60:0/2621092168 learned_addr learned my addr 172.21.15.60:0/2621092168 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a2ffd640 1 --2- 172.21.15.60:0/2621092168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73a4144980 0x7f73a415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a27fc640 1 -- 172.21.15.60:0/2621092168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73a4144980 msgr2=0x7f73a415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:55.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a27fc640 1 --2- 172.21.15.60:0/2621092168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73a4144980 0x7f73a415c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a27fc640 1 -- 172.21.15.60:0/2621092168 >> v1:172.21.15.100:6789/0 conn(0x7f73a40b4370 legacy=0x7f73a4071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.533+0000 7f73a27fc640 1 -- 172.21.15.60:0/2621092168 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a40cd380 con 0x7f73a40ceb70 2023-11-18T20:43:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.534+0000 7f73a27fc640 1 --2- 172.21.15.60:0/2621092168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 0x7f73a40cc450 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7398009870 tx=0x7f739802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06b7c977fd7530a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.534+0000 7f73a1ffb640 1 -- 172.21.15.60:0/2621092168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398030840 con 0x7f73a40ceb70 2023-11-18T20:43:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.534+0000 7f73a1ffb640 1 -- 172.21.15.60:0/2621092168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73980309a0 con 0x7f73a40ceb70 2023-11-18T20:43:55.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.534+0000 7f73a1ffb640 1 -- 172.21.15.60:0/2621092168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398030d30 con 0x7f73a40ceb70 2023-11-18T20:43:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.535+0000 7f73a95c9640 1 -- 172.21.15.60:0/2621092168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 msgr2=0x7f73a40cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.535+0000 7f73a95c9640 1 --2- 172.21.15.60:0/2621092168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 0x7f73a40cc450 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7398009870 tx=0x7f739802b980 comp rx=0 tx=0).stop 2023-11-18T20:43:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.535+0000 7f73a95c9640 1 -- 172.21.15.60:0/2621092168 shutdown_connections 2023-11-18T20:43:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.535+0000 7f73a95c9640 1 --2- 172.21.15.60:0/2621092168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 0x7f73a40cc450 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.535+0000 7f73a95c9640 1 --2- 172.21.15.60:0/2621092168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73a4144980 0x7f73a415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.535+0000 7f73a95c9640 1 -- 172.21.15.60:0/2621092168 >> 172.21.15.60:0/2621092168 conn(0x7f73a406a4d0 msgr2=0x7f73a406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.535+0000 7f73a95c9640 1 -- 172.21.15.60:0/2621092168 shutdown_connections 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.535+0000 7f73a95c9640 1 -- 172.21.15.60:0/2621092168 wait complete. 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.536+0000 7f73a95c9640 1 Processor -- start 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.536+0000 7f73a95c9640 1 -- start start 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a95c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a40b4370 0x7f73a41f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a95c9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 0x7f73a415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a95c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73a4144980 0x7f73a41e4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a95c9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a415c5e0 con 0x7f73a40ceb70 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a95c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a415ba60 con 0x7f73a40b4370 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a95c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a415bd40 con 0x7f73a4144980 2023-11-18T20:43:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 0x7f73a415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 0x7f73a415b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48184/0 (socket says 172.21.15.60:48184) 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a27fc640 1 -- 172.21.15.60:0/2191208473 learned_addr learned my addr 172.21.15.60:0/2191208473 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a2ffd640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a40b4370 0x7f73a41f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a37fe640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73a4144980 0x7f73a41e4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a27fc640 1 -- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73a4144980 msgr2=0x7f73a41e4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a27fc640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73a4144980 0x7f73a41e4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a27fc640 1 -- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a40b4370 msgr2=0x7f73a41f0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a27fc640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a40b4370 0x7f73a41f0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a27fc640 1 -- 172.21.15.60:0/2191208473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73980094d0 con 0x7f73a40ceb70 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.537+0000 7f73a27fc640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 0x7f73a415b010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7398000c00 tx=0x7f7398004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.538+0000 7f738b7fe640 1 -- 172.21.15.60:0/2191208473 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398039c90 con 0x7f73a40ceb70 2023-11-18T20:43:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.538+0000 7f738b7fe640 1 -- 172.21.15.60:0/2191208473 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7398039df0 con 0x7f73a40ceb70 2023-11-18T20:43:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.538+0000 7f73a95c9640 1 -- 172.21.15.60:0/2191208473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a415c020 con 0x7f73a40ceb70 2023-11-18T20:43:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.538+0000 7f738b7fe640 1 -- 172.21.15.60:0/2191208473 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398043640 con 0x7f73a40ceb70 2023-11-18T20:43:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.538+0000 7f73a95c9640 1 -- 172.21.15.60:0/2191208473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a41f0f50 con 0x7f73a40ceb70 2023-11-18T20:43:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.539+0000 7f73a95c9640 1 -- 172.21.15.60:0/2191208473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a40b1ed0 con 0x7f73a40ceb70 2023-11-18T20:43:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.539+0000 7f738b7fe640 1 -- 172.21.15.60:0/2191208473 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73980437a0 con 0x7f73a40ceb70 2023-11-18T20:43:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.540+0000 7f738b7fe640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f739006ae40 0x7f739006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.540+0000 7f738b7fe640 1 -- 172.21.15.60:0/2191208473 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f739803e070 con 0x7f73a40ceb70 2023-11-18T20:43:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.540+0000 7f73a2ffd640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f739006ae40 0x7f739006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.540+0000 7f73a2ffd640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f739006ae40 0x7f739006d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f739c009aa0 tx=0x7f739c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:55.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.544+0000 7f738b7fe640 1 -- 172.21.15.60:0/2191208473 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7398077650 con 0x7f73a40ceb70 2023-11-18T20:43:55.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.718+0000 7f73a95c9640 1 -- 172.21.15.60:0/2191208473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73a4069400 con 0x7f73a40ceb70 2023-11-18T20:43:55.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.719+0000 7f738b7fe640 1 -- 172.21.15.60:0/2191208473 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f739807ac70 con 0x7f73a40ceb70 2023-11-18T20:43:55.721 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:55.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 -- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f739006ae40 msgr2=0x7f739006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f739006ae40 0x7f739006d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f739c009aa0 tx=0x7f739c009190 comp rx=0 tx=0).stop 2023-11-18T20:43:55.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 -- 172.21.15.60:0/2191208473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 msgr2=0x7f73a415b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 0x7f73a415b010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7398000c00 tx=0x7f7398004730 comp rx=0 tx=0).stop 2023-11-18T20:43:55.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 -- 172.21.15.60:0/2191208473 shutdown_connections 2023-11-18T20:43:55.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f739006ae40 0x7f739006d300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73a4144980 0x7f73a41e4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73a40ceb70 0x7f73a415b010 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 --2- 172.21.15.60:0/2191208473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73a40b4370 0x7f73a41f0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 -- 172.21.15.60:0/2191208473 >> 172.21.15.60:0/2191208473 conn(0x7f73a406a4d0 msgr2=0x7f73a4071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:55.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 -- 172.21.15.60:0/2191208473 shutdown_connections 2023-11-18T20:43:55.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.721+0000 7f73a95c9640 1 -- 172.21.15.60:0/2191208473 wait complete. 2023-11-18T20:43:55.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:55.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.841+0000 7f43e2890640 1 Processor -- start 2023-11-18T20:43:55.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.841+0000 7f43e2890640 1 -- start start 2023-11-18T20:43:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.842+0000 7f43e2890640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43dc15d330 0x7f43dc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.842+0000 7f43e2890640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43dc0d2910 0x7f43dc0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.842+0000 7f43e2890640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43dc0b2b10 con 0x7f43dc0d3730 2023-11-18T20:43:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.842+0000 7f43e2890640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43dc0d3b50 con 0x7f43dc15d330 2023-11-18T20:43:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.842+0000 7f43e2890640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0d1590 con 0x7f43dc0d2910 2023-11-18T20:43:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.842+0000 7f43dbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43dc15d330 0x7f43dc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.842+0000 7f43dbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43dc15d330 0x7f43dc0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48194/0 (socket says 172.21.15.60:48194) 2023-11-18T20:43:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.842+0000 7f43dbfff640 1 -- 172.21.15.60:0/127394903 learned_addr learned my addr 172.21.15.60:0/127394903 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:55.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.843+0000 7f43db7fe640 1 --2- 172.21.15.60:0/127394903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43dc0d2910 0x7f43dc0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.843+0000 7f43dbfff640 1 -- 172.21.15.60:0/127394903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43dc0d2910 msgr2=0x7f43dc0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:55.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.843+0000 7f43dbfff640 1 --2- 172.21.15.60:0/127394903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43dc0d2910 0x7f43dc0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.843+0000 7f43dbfff640 1 -- 172.21.15.60:0/127394903 >> v1:172.21.15.100:6789/0 conn(0x7f43dc0d3730 legacy=0x7f43dc0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.843+0000 7f43dbfff640 1 -- 172.21.15.60:0/127394903 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0d1870 con 0x7f43dc15d330 2023-11-18T20:43:55.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.843+0000 7f43dbfff640 1 --2- 172.21.15.60:0/127394903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc15d330 0x7f43dc0b8b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009870 tx=0x7f43cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d3b65ec585bcea server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:55.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43daffd640 1 -- 172.21.15.60:0/127394903 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc031640 con 0x7f43dc15d330 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43daffd640 1 -- 172.21.15.60:0/127394903 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc0317a0 con 0x7f43dc15d330 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43daffd640 1 -- 172.21.15.60:0/127394903 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc031b30 con 0x7f43dc15d330 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43e2890640 1 -- 172.21.15.60:0/127394903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc15d330 msgr2=0x7f43dc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43e2890640 1 --2- 172.21.15.60:0/127394903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc15d330 0x7f43dc0b8b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009870 tx=0x7f43cc02b8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43e2890640 1 -- 172.21.15.60:0/127394903 shutdown_connections 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43e2890640 1 --2- 172.21.15.60:0/127394903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43dc0d2910 0x7f43dc0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43e2890640 1 --2- 172.21.15.60:0/127394903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc15d330 0x7f43dc0b8b80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43e2890640 1 -- 172.21.15.60:0/127394903 >> 172.21.15.60:0/127394903 conn(0x7f43dc072020 msgr2=0x7f43dc072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43e2890640 1 -- 172.21.15.60:0/127394903 shutdown_connections 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.844+0000 7f43e2890640 1 -- 172.21.15.60:0/127394903 wait complete. 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.845+0000 7f43e2890640 1 Processor -- start 2023-11-18T20:43:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.846+0000 7f43e2890640 1 -- start start 2023-11-18T20:43:55.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.846+0000 7f43e2890640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc0d2910 0x7f43dc157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.846+0000 7f43e2890640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc0d3730 0x7f43dc153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.846+0000 7f43e2890640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43dc15d330 0x7f43dc15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.846+0000 7f43e2890640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc15d710 con 0x7f43dc0d3730 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.846+0000 7f43e2890640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc15d9f0 con 0x7f43dc0d2910 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.846+0000 7f43e2890640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc15dcd0 con 0x7f43dc15d330 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.846+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc0d3730 0x7f43dc153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.846+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc0d3730 0x7f43dc153320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48202/0 (socket says 172.21.15.60:48202) 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.846+0000 7f43db7fe640 1 -- 172.21.15.60:0/367792626 learned_addr learned my addr 172.21.15.60:0/367792626 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43dbfff640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc0d2910 0x7f43dc157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43e0e2d640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43dc15d330 0x7f43dc15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43db7fe640 1 -- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43dc15d330 msgr2=0x7f43dc15a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43db7fe640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43dc15d330 0x7f43dc15a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43db7fe640 1 -- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc0d2910 msgr2=0x7f43dc157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43db7fe640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc0d2910 0x7f43dc157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43db7fe640 1 -- 172.21.15.60:0/367792626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc0094d0 con 0x7f43dc0d3730 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43db7fe640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc0d3730 0x7f43dc153320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f43d000d5f0 tx=0x7f43d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43d8ff9640 1 -- 172.21.15.60:0/367792626 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00125f0 con 0x7f43dc0d3730 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43d8ff9640 1 -- 172.21.15.60:0/367792626 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d0012750 con 0x7f43dc0d3730 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43e2890640 1 -- 172.21.15.60:0/367792626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc15dfb0 con 0x7f43dc0d3730 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43d8ff9640 1 -- 172.21.15.60:0/367792626 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d0012b00 con 0x7f43dc0d3730 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.847+0000 7f43e2890640 1 -- 172.21.15.60:0/367792626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc153860 con 0x7f43dc0d3730 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.848+0000 7f43e2890640 1 -- 172.21.15.60:0/367792626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc0b7750 con 0x7f43dc0d3730 2023-11-18T20:43:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.848+0000 7f43d8ff9640 1 -- 172.21.15.60:0/367792626 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f43d0012c60 con 0x7f43dc0d3730 2023-11-18T20:43:55.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.849+0000 7f43d8ff9640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43c806ae70 0x7f43c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:55.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.849+0000 7f43d8ff9640 1 -- 172.21.15.60:0/367792626 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f43d0089bb0 con 0x7f43dc0d3730 2023-11-18T20:43:55.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.849+0000 7f43dbfff640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43c806ae70 0x7f43c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:55.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.850+0000 7f43dbfff640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43c806ae70 0x7f43c806d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f43cc036000 tx=0x7f43cc03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:55.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:55.853+0000 7f43d8ff9640 1 -- 172.21.15.60:0/367792626 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43d0054b60 con 0x7f43dc0d3730 2023-11-18T20:43:56.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.031+0000 7f43e2890640 1 -- 172.21.15.60:0/367792626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43dc069400 con 0x7f43dc0d3730 2023-11-18T20:43:56.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.031+0000 7f43d8ff9640 1 -- 172.21.15.60:0/367792626 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43d0058180 con 0x7f43dc0d3730 2023-11-18T20:43:56.032 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:56.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 -- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43c806ae70 msgr2=0x7f43c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43c806ae70 0x7f43c806d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f43cc036000 tx=0x7f43cc03c040 comp rx=0 tx=0).stop 2023-11-18T20:43:56.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 -- 172.21.15.60:0/367792626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc0d3730 msgr2=0x7f43dc153320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc0d3730 0x7f43dc153320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f43d000d5f0 tx=0x7f43d000dab0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 -- 172.21.15.60:0/367792626 shutdown_connections 2023-11-18T20:43:56.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f43c806ae70 0x7f43c806d330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43dc15d330 0x7f43dc15a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43dc0d3730 0x7f43dc153320 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 --2- 172.21.15.60:0/367792626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc0d2910 0x7f43dc157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 -- 172.21.15.60:0/367792626 >> 172.21.15.60:0/367792626 conn(0x7f43dc072020 msgr2=0x7f43dc0b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:56.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 -- 172.21.15.60:0/367792626 shutdown_connections 2023-11-18T20:43:56.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.033+0000 7f43e2890640 1 -- 172.21.15.60:0/367792626 wait complete. 2023-11-18T20:43:56.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-11-18T20:43:56.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.154+0000 7efee8a1f640 1 Processor -- start 2023-11-18T20:43:56.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.154+0000 7efee8a1f640 1 -- start start 2023-11-18T20:43:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.155+0000 7efee8a1f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efee4160fb0 0x7efee4161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.155+0000 7efee8a1f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee4161d20 con 0x7efee406ea50 2023-11-18T20:43:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.155+0000 7efee8a1f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee415e760 con 0x7efee40d0ed0 2023-11-18T20:43:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.155+0000 7efee8a1f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee415d9a0 con 0x7efee4160fb0 2023-11-18T20:43:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.155+0000 7efee259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efee406ea50 0x7efee406ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46404/0 (socket says 172.21.15.60:46404) 2023-11-18T20:43:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.155+0000 7efee259c640 1 -- 172.21.15.60:0/1936681990 learned_addr learned my addr 172.21.15.60:0/1936681990 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.155+0000 7efee1d9b640 1 --2- 172.21.15.60:0/1936681990 >> v2:172.21.15.100:3300/0 conn(0x7efee4160fb0 0x7efee4161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.155+0000 7efee159a640 1 -- 172.21.15.60:0/1936681990 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077725725 0 0) 0x7efee4161d20 con 0x7efee406ea50 2023-11-18T20:43:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.156+0000 7efee159a640 1 -- 172.21.15.60:0/1936681990 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efecc003520 con 0x7efee406ea50 2023-11-18T20:43:56.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.156+0000 7efee1d9b640 1 -- 172.21.15.60:0/1936681990 >> v1:172.21.15.100:6789/0 conn(0x7efee40d0ed0 legacy=0x7efee40d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.156+0000 7efee1d9b640 1 -- 172.21.15.60:0/1936681990 >> v1:172.21.15.60:6789/0 conn(0x7efee406ea50 legacy=0x7efee406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.158 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.156+0000 7efee1d9b640 1 -- 172.21.15.60:0/1936681990 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee415dc80 con 0x7efee4160fb0 2023-11-18T20:43:56.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee1d9b640 1 --2- 172.21.15.60:0/1936681990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee4160fb0 0x7efee4161440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efed80097a0 tx=0x7efed802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8464f0226c11232c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:56.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee159a640 1 -- 172.21.15.60:0/1936681990 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8031830 con 0x7efee4160fb0 2023-11-18T20:43:56.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee159a640 1 -- 172.21.15.60:0/1936681990 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed8031990 con 0x7efee4160fb0 2023-11-18T20:43:56.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee159a640 1 -- 172.21.15.60:0/1936681990 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8031d20 con 0x7efee4160fb0 2023-11-18T20:43:56.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee8a1f640 1 -- 172.21.15.60:0/1936681990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee4160fb0 msgr2=0x7efee4161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee8a1f640 1 --2- 172.21.15.60:0/1936681990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee4160fb0 0x7efee4161440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efed80097a0 tx=0x7efed802b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee8a1f640 1 -- 172.21.15.60:0/1936681990 shutdown_connections 2023-11-18T20:43:56.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee8a1f640 1 --2- 172.21.15.60:0/1936681990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee4160fb0 0x7efee4161440 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee8a1f640 1 -- 172.21.15.60:0/1936681990 >> 172.21.15.60:0/1936681990 conn(0x7efee406a6b0 msgr2=0x7efee406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:56.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee8a1f640 1 -- 172.21.15.60:0/1936681990 shutdown_connections 2023-11-18T20:43:56.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.157+0000 7efee8a1f640 1 -- 172.21.15.60:0/1936681990 wait complete. 2023-11-18T20:43:56.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.159+0000 7efee8a1f640 1 Processor -- start 2023-11-18T20:43:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.159+0000 7efee8a1f640 1 -- start start 2023-11-18T20:43:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.159+0000 7efee8a1f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee406ea50 0x7efee40cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.159+0000 7efee8a1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efee40d0ed0 0x7efee41f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.159+0000 7efee8a1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee4160fb0 0x7efee41f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.159+0000 7efee8a1f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee40cc3c0 con 0x7efee4160fb0 2023-11-18T20:43:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.159+0000 7efee8a1f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee40cb840 con 0x7efee406ea50 2023-11-18T20:43:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.159+0000 7efee8a1f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee40cbb20 con 0x7efee40d0ed0 2023-11-18T20:43:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee4160fb0 0x7efee41f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee4160fb0 0x7efee41f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48216/0 (socket says 172.21.15.60:48216) 2023-11-18T20:43:56.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee2d9d640 1 -- 172.21.15.60:0/1619268958 learned_addr learned my addr 172.21.15.60:0/1619268958 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee259c640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee406ea50 0x7efee40cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee1d9b640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efee40d0ed0 0x7efee41f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee2d9d640 1 -- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efee40d0ed0 msgr2=0x7efee41f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee2d9d640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efee40d0ed0 0x7efee41f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee2d9d640 1 -- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee406ea50 msgr2=0x7efee40cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee2d9d640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee406ea50 0x7efee40cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee2d9d640 1 -- 172.21.15.60:0/1619268958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed8009400 con 0x7efee4160fb0 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee2d9d640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee4160fb0 0x7efee41f66d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efed400c730 tx=0x7efed400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efed2ffd640 1 -- 172.21.15.60:0/1619268958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4003b70 con 0x7efee4160fb0 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efed2ffd640 1 -- 172.21.15.60:0/1619268958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed4003cf0 con 0x7efee4160fb0 2023-11-18T20:43:56.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efed2ffd640 1 -- 172.21.15.60:0/1619268958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4003ea0 con 0x7efee4160fb0 2023-11-18T20:43:56.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee8a1f640 1 -- 172.21.15.60:0/1619268958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee40cbe00 con 0x7efee4160fb0 2023-11-18T20:43:56.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.160+0000 7efee8a1f640 1 -- 172.21.15.60:0/1619268958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee41f6f50 con 0x7efee4160fb0 2023-11-18T20:43:56.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.161+0000 7efee8a1f640 1 -- 172.21.15.60:0/1619268958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee4069400 con 0x7efee4160fb0 2023-11-18T20:43:56.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.162+0000 7efed2ffd640 1 -- 172.21.15.60:0/1619268958 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efed4005520 con 0x7efee4160fb0 2023-11-18T20:43:56.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.162+0000 7efed2ffd640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efec406ae40 0x7efec406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.162+0000 7efed2ffd640 1 -- 172.21.15.60:0/1619268958 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7efed4089600 con 0x7efee4160fb0 2023-11-18T20:43:56.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.162+0000 7efee259c640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efec406ae40 0x7efec406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.163+0000 7efee259c640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efec406ae40 0x7efec406d300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efedc0011e0 tx=0x7efedc007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:56.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.167+0000 7efed2ffd640 1 -- 172.21.15.60:0/1619268958 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efed40545d0 con 0x7efee4160fb0 2023-11-18T20:43:56.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.342+0000 7efee8a1f640 1 -- 172.21.15.60:0/1619268958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7efee4070d30 con 0x7efee4160fb0 2023-11-18T20:43:56.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.342+0000 7efed2ffd640 1 -- 172.21.15.60:0/1619268958 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7efed4057bf0 con 0x7efee4160fb0 2023-11-18T20:43:56.344 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:56.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 -- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efec406ae40 msgr2=0x7efec406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efec406ae40 0x7efec406d300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efedc0011e0 tx=0x7efedc007d30 comp rx=0 tx=0).stop 2023-11-18T20:43:56.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 -- 172.21.15.60:0/1619268958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee4160fb0 msgr2=0x7efee41f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee4160fb0 0x7efee41f66d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efed400c730 tx=0x7efed400cbf0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 -- 172.21.15.60:0/1619268958 shutdown_connections 2023-11-18T20:43:56.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7efec406ae40 0x7efec406d300 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efee4160fb0 0x7efee41f66d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efee40d0ed0 0x7efee41f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 --2- 172.21.15.60:0/1619268958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efee406ea50 0x7efee40cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 -- 172.21.15.60:0/1619268958 >> 172.21.15.60:0/1619268958 conn(0x7efee406a6b0 msgr2=0x7efee415e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:56.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 -- 172.21.15.60:0/1619268958 shutdown_connections 2023-11-18T20:43:56.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.344+0000 7efee8a1f640 1 -- 172.21.15.60:0/1619268958 wait complete. 2023-11-18T20:43:56.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.462+0000 7f8d46ba5640 1 Processor -- start 2023-11-18T20:43:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.462+0000 7f8d46ba5640 1 -- start start 2023-11-18T20:43:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.463+0000 7f8d46ba5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d4015d330 0x7f8d400b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.463+0000 7f8d46ba5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d400b2b10 con 0x7f8d400d3900 2023-11-18T20:43:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.463+0000 7f8d46ba5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d400d1590 con 0x7f8d400d2910 2023-11-18T20:43:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.463+0000 7f8d46ba5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d400d1870 con 0x7f8d4015d330 2023-11-18T20:43:56.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.463+0000 7f8d45142640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8d400d3900 0x7f8d400b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46408/0 (socket says 172.21.15.60:46408) 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.463+0000 7f8d45142640 1 -- 172.21.15.60:0/264835962 learned_addr learned my addr 172.21.15.60:0/264835962 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.463+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381094720 0 0) 0x7f8d400b2b10 con 0x7f8d400d3900 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.463+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d28003520 con 0x7f8d400d3900 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.463+0000 7f8d44941640 1 --2- 172.21.15.60:0/264835962 >> v2:172.21.15.100:3300/0 conn(0x7f8d4015d330 0x7f8d400b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1970155989 0 0) 0x7f8d28003520 con 0x7f8d400d3900 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 >> v2:172.21.15.100:3300/0 conn(0x7f8d4015d330 msgr2=0x7f8d400b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 --2- 172.21.15.60:0/264835962 >> v2:172.21.15.100:3300/0 conn(0x7f8d4015d330 0x7f8d400b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 >> v1:172.21.15.100:6789/0 conn(0x7f8d400d2910 legacy=0x7f8d400d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d400d1b50 con 0x7f8d400d3900 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f8d30001670 con 0x7f8d400d3900 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 >> v1:172.21.15.60:6789/0 conn(0x7f8d400d3900 legacy=0x7f8d400b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d28003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d280047c0 0x7f8d28004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28005180 0x7f8d2800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400d1870 con 0x7f8d28003970 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2800c120 con 0x7f8d28005180 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2800c2e0 con 0x7f8d280047c0 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d44941640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d28003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3ffff640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d280047c0 0x7f8d28004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d45142640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28005180 0x7f8d2800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d44941640 1 -- 172.21.15.60:0/264835962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d280047c0 msgr2=0x7f8d28004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d44941640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d280047c0 0x7f8d28004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d44941640 1 -- 172.21.15.60:0/264835962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28005180 msgr2=0x7f8d2800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d44941640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28005180 0x7f8d2800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d44941640 1 -- 172.21.15.60:0/264835962 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d28003520 con 0x7f8d28003970 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d45142640 1 -- 172.21.15.60:0/264835962 reap_dead start 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.464+0000 7f8d3ffff640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d280047c0 0x7f8d28004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.465+0000 7f8d44941640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d28003d90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009fc0 tx=0x7f8d280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bce3f9fd42f0b7d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.465+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3802cc20 con 0x7f8d28003970 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.465+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3800f040 con 0x7f8d28003970 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.465+0000 7f8d3f7fe640 1 -- 172.21.15.60:0/264835962 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38034440 con 0x7f8d28003970 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.466+0000 7f8d46ba5640 1 -- 172.21.15.60:0/264835962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 msgr2=0x7f8d28003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.466+0000 7f8d46ba5640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d28003d90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009fc0 tx=0x7f8d280057f0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.466+0000 7f8d46ba5640 1 -- 172.21.15.60:0/264835962 shutdown_connections 2023-11-18T20:43:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.466+0000 7f8d46ba5640 1 --2- 172.21.15.60:0/264835962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d28003d90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.466+0000 7f8d46ba5640 1 -- 172.21.15.60:0/264835962 >> 172.21.15.60:0/264835962 conn(0x7f8d40072020 msgr2=0x7f8d40072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:56.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.466+0000 7f8d46ba5640 1 -- 172.21.15.60:0/264835962 shutdown_connections 2023-11-18T20:43:56.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.466+0000 7f8d46ba5640 1 -- 172.21.15.60:0/264835962 wait complete. 2023-11-18T20:43:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.467+0000 7f8d46ba5640 1 Processor -- start 2023-11-18T20:43:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.467+0000 7f8d46ba5640 1 -- start start 2023-11-18T20:43:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d46ba5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d400cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d46ba5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400c7520 0x7f8d40152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d46ba5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d4015fb30 0x7f8d4015db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d46ba5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400c82e0 con 0x7f8d28003970 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d46ba5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40152fc0 con 0x7f8d400c7520 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d46ba5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40160480 con 0x7f8d4015fb30 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d44941640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d400cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d44941640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d400cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48242/0 (socket says 172.21.15.60:48242) 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d44941640 1 -- 172.21.15.60:0/1803431772 learned_addr learned my addr 172.21.15.60:0/1803431772 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d45142640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d4015fb30 0x7f8d4015db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d3ffff640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400c7520 0x7f8d40152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d44941640 1 -- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d4015fb30 msgr2=0x7f8d4015db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d44941640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d4015fb30 0x7f8d4015db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d44941640 1 -- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400c7520 msgr2=0x7f8d40152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d44941640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400c7520 0x7f8d40152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d44941640 1 -- 172.21.15.60:0/1803431772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d40160760 con 0x7f8d28003970 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d3ffff640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400c7520 0x7f8d40152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d44941640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d400cb380 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e230 tx=0x7f8d38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.468+0000 7f8d3d7fa640 1 -- 172.21.15.60:0/1803431772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38034c50 con 0x7f8d28003970 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.469+0000 7f8d3d7fa640 1 -- 172.21.15.60:0/1803431772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d38034db0 con 0x7f8d28003970 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.469+0000 7f8d46ba5640 1 -- 172.21.15.60:0/1803431772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d400c7f00 con 0x7f8d28003970 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.469+0000 7f8d3d7fa640 1 -- 172.21.15.60:0/1803431772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38003bb0 con 0x7f8d28003970 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.469+0000 7f8d46ba5640 1 -- 172.21.15.60:0/1803431772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d40160190 con 0x7f8d28003970 2023-11-18T20:43:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.470+0000 7f8d46ba5640 1 -- 172.21.15.60:0/1803431772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d40069400 con 0x7f8d28003970 2023-11-18T20:43:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.470+0000 7f8d3d7fa640 1 -- 172.21.15.60:0/1803431772 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d38040450 con 0x7f8d28003970 2023-11-18T20:43:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.470+0000 7f8d3d7fa640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d2c06ae40 0x7f8d2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.471+0000 7f8d3d7fa640 1 -- 172.21.15.60:0/1803431772 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f8d380abf00 con 0x7f8d28003970 2023-11-18T20:43:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.471+0000 7f8d3ffff640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d2c06ae40 0x7f8d2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.471+0000 7f8d3ffff640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d2c06ae40 0x7f8d2c06d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8d34000c00 tx=0x7f8d34009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.475+0000 7f8d3d7fa640 1 -- 172.21.15.60:0/1803431772 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d38076ed0 con 0x7f8d28003970 2023-11-18T20:43:56.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.667+0000 7f8d46ba5640 1 -- 172.21.15.60:0/1803431772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8d400d0930 con 0x7f8d28003970 2023-11-18T20:43:56.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.668+0000 7f8d3d7fa640 1 -- 172.21.15.60:0/1803431772 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8d3807a4f0 con 0x7f8d28003970 2023-11-18T20:43:56.669 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:56.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.669+0000 7f8d46ba5640 1 -- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d2c06ae40 msgr2=0x7f8d2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.669+0000 7f8d46ba5640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d2c06ae40 0x7f8d2c06d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8d34000c00 tx=0x7f8d34009040 comp rx=0 tx=0).stop 2023-11-18T20:43:56.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.669+0000 7f8d46ba5640 1 -- 172.21.15.60:0/1803431772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 msgr2=0x7f8d400cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.669+0000 7f8d46ba5640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d400cb380 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e230 tx=0x7f8d38009750 comp rx=0 tx=0).stop 2023-11-18T20:43:56.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.670+0000 7f8d46ba5640 1 -- 172.21.15.60:0/1803431772 shutdown_connections 2023-11-18T20:43:56.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.670+0000 7f8d46ba5640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f8d2c06ae40 0x7f8d2c06d300 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.670+0000 7f8d46ba5640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d4015fb30 0x7f8d4015db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.670+0000 7f8d46ba5640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d400c7520 0x7f8d40152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.670+0000 7f8d46ba5640 1 --2- 172.21.15.60:0/1803431772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d28003970 0x7f8d400cb380 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.670+0000 7f8d46ba5640 1 -- 172.21.15.60:0/1803431772 >> 172.21.15.60:0/1803431772 conn(0x7f8d40072020 msgr2=0x7f8d400b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:56.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.670+0000 7f8d46ba5640 1 -- 172.21.15.60:0/1803431772 shutdown_connections 2023-11-18T20:43:56.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.670+0000 7f8d46ba5640 1 -- 172.21.15.60:0/1803431772 wait complete. 2023-11-18T20:43:56.682 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:43:56.682 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:43:56.698 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:43:56.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.845+0000 7f5663fff640 1 Processor -- start 2023-11-18T20:43:56.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.845+0000 7f5663fff640 1 -- start start 2023-11-18T20:43:56.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f5663fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f56640b1710 0x7f56640b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f5663fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640cf6b0 0x7f56640cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f5663fff640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56640b20e0 con 0x7f56640b3410 2023-11-18T20:43:56.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f5663fff640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566406d090 con 0x7f56640b1710 2023-11-18T20:43:56.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f5663fff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56640b2610 con 0x7f56640cf6b0 2023-11-18T20:43:56.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f565a7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f56640b3410 0x7f56640cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46412/0 (socket says 172.21.15.60:46412) 2023-11-18T20:43:56.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f565a7fc640 1 -- 172.21.15.60:0/2258014113 learned_addr learned my addr 172.21.15.60:0/2258014113 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:56.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f56637fe640 1 --2- 172.21.15.60:0/2258014113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640cf6b0 0x7f56640cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f5662ffd640 1 --2- 172.21.15.60:0/2258014113 >> v2:172.21.15.100:3300/0 conn(0x7f56640b1710 0x7f56640b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f56627fc640 1 -- 172.21.15.60:0/2258014113 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862391711 0 0) 0x7f56640b20e0 con 0x7f56640b3410 2023-11-18T20:43:56.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f56627fc640 1 -- 172.21.15.60:0/2258014113 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5650003520 con 0x7f56640b3410 2023-11-18T20:43:56.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f56637fe640 1 -- 172.21.15.60:0/2258014113 >> v2:172.21.15.100:3300/0 conn(0x7f56640b1710 msgr2=0x7f56640b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:56.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f56627fc640 1 -- 172.21.15.60:0/2258014113 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1021354376 0 0) 0x7f5650003520 con 0x7f56640b3410 2023-11-18T20:43:56.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f56637fe640 1 --2- 172.21.15.60:0/2258014113 >> v2:172.21.15.100:3300/0 conn(0x7f56640b1710 0x7f56640b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f56637fe640 1 -- 172.21.15.60:0/2258014113 >> v1:172.21.15.60:6789/0 conn(0x7f56640b3410 legacy=0x7f56640cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.846+0000 7f56637fe640 1 -- 172.21.15.60:0/2258014113 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640cedd0 con 0x7f56640cf6b0 2023-11-18T20:43:56.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.847+0000 7f56637fe640 1 --2- 172.21.15.60:0/2258014113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640cf6b0 0x7f56640cfac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f565c009d90 tx=0x7f565c02bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7319ce6b7a186be2 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:56.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.847+0000 7f56627fc640 1 -- 172.21.15.60:0/2258014113 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c001670 con 0x7f56640cf6b0 2023-11-18T20:43:56.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.848+0000 7f56627fc640 1 -- 172.21.15.60:0/2258014113 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c02c780 con 0x7f56640cf6b0 2023-11-18T20:43:56.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.848+0000 7f56627fc640 1 -- 172.21.15.60:0/2258014113 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c02caf0 con 0x7f56640cf6b0 2023-11-18T20:43:56.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.848+0000 7f5663fff640 1 -- 172.21.15.60:0/2258014113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640cf6b0 msgr2=0x7f56640cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.848+0000 7f5663fff640 1 --2- 172.21.15.60:0/2258014113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640cf6b0 0x7f56640cfac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f565c009d90 tx=0x7f565c02bc50 comp rx=0 tx=0).stop 2023-11-18T20:43:56.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.848+0000 7f5663fff640 1 -- 172.21.15.60:0/2258014113 shutdown_connections 2023-11-18T20:43:56.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.848+0000 7f5663fff640 1 --2- 172.21.15.60:0/2258014113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640cf6b0 0x7f56640cfac0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.848+0000 7f5663fff640 1 --2- 172.21.15.60:0/2258014113 >> v2:172.21.15.100:3300/0 conn(0x7f56640b1710 0x7f56640b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.848+0000 7f5663fff640 1 -- 172.21.15.60:0/2258014113 >> 172.21.15.60:0/2258014113 conn(0x7f56640b28f0 msgr2=0x7f56640b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:56.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.848+0000 7f5663fff640 1 -- 172.21.15.60:0/2258014113 shutdown_connections 2023-11-18T20:43:56.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.848+0000 7f5663fff640 1 -- 172.21.15.60:0/2258014113 wait complete. 2023-11-18T20:43:56.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.849+0000 7f5663fff640 1 Processor -- start 2023-11-18T20:43:56.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.849+0000 7f5663fff640 1 -- start start 2023-11-18T20:43:56.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.850+0000 7f5663fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640b1710 0x7f5664141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.850+0000 7f5663fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56640b3410 0x7f56640c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.850+0000 7f5663fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cf6b0 0x7f56640ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.850+0000 7f5663fff640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640cb170 con 0x7f56640b3410 2023-11-18T20:43:56.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.850+0000 7f5663fff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640af3a0 con 0x7f56640cf6b0 2023-11-18T20:43:56.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.850+0000 7f5663fff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56640cae90 con 0x7f56640b1710 2023-11-18T20:43:56.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.850+0000 7f565a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56640b3410 0x7f56640c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.850+0000 7f565a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56640b3410 0x7f56640c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48252/0 (socket says 172.21.15.60:48252) 2023-11-18T20:43:56.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.850+0000 7f565a7fc640 1 -- 172.21.15.60:0/86366070 learned_addr learned my addr 172.21.15.60:0/86366070 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:56.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f56637fe640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cf6b0 0x7f56640ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f5662ffd640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640b1710 0x7f5664141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f565a7fc640 1 -- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640b1710 msgr2=0x7f5664141c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f565a7fc640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640b1710 0x7f5664141c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f565a7fc640 1 -- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cf6b0 msgr2=0x7f56640ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:56.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f565a7fc640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cf6b0 0x7f56640ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f565a7fc640 1 -- 172.21.15.60:0/86366070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c009a70 con 0x7f56640b3410 2023-11-18T20:43:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f5662ffd640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640b1710 0x7f5664141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f56637fe640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cf6b0 0x7f56640ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f565a7fc640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56640b3410 0x7f56640c80e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f564c00c780 tx=0x7f564c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f565bfff640 1 -- 172.21.15.60:0/86366070 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c005ad0 con 0x7f56640b3410 2023-11-18T20:43:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f565bfff640 1 -- 172.21.15.60:0/86366070 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c005c30 con 0x7f56640b3410 2023-11-18T20:43:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f565bfff640 1 -- 172.21.15.60:0/86366070 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c007070 con 0x7f56640b3410 2023-11-18T20:43:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f5663fff640 1 -- 172.21.15.60:0/86366070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56640aed50 con 0x7f56640b3410 2023-11-18T20:43:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.851+0000 7f5663fff640 1 -- 172.21.15.60:0/86366070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56640c8620 con 0x7f56640b3410 2023-11-18T20:43:56.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.852+0000 7f5663fff640 1 -- 172.21.15.60:0/86366070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56640af530 con 0x7f56640b3410 2023-11-18T20:43:56.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.853+0000 7f565bfff640 1 -- 172.21.15.60:0/86366070 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f564c0071d0 con 0x7f56640b3410 2023-11-18T20:43:56.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.853+0000 7f565bfff640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f564406ae40 0x7f564406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:56.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.853+0000 7f565bfff640 1 -- 172.21.15.60:0/86366070 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f564c0893c0 con 0x7f56640b3410 2023-11-18T20:43:56.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.853+0000 7f5662ffd640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f564406ae40 0x7f564406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:56.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.854+0000 7f5662ffd640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f564406ae40 0x7f564406d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5654009a20 tx=0x7f5654008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:56.858 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:56.857+0000 7f565bfff640 1 -- 172.21.15.60:0/86366070 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f564c054370 con 0x7f56640b3410 2023-11-18T20:43:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.034+0000 7f5663fff640 1 -- 172.21.15.60:0/86366070 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5664069320 con 0x7f56640b3410 2023-11-18T20:43:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.035+0000 7f565bfff640 1 -- 172.21.15.60:0/86366070 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18336 (secure 0 0 0) 0x7f564c057990 con 0x7f56640b3410 2023-11-18T20:43:57.037 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:43:57.037 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:42.814654+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-11-18T20:43:40.650315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:43:57.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 -- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f564406ae40 msgr2=0x7f564406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f564406ae40 0x7f564406d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5654009a20 tx=0x7f5654008040 comp rx=0 tx=0).stop 2023-11-18T20:43:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 -- 172.21.15.60:0/86366070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56640b3410 msgr2=0x7f56640c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56640b3410 0x7f56640c80e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f564c00c780 tx=0x7f564c00cc40 comp rx=0 tx=0).stop 2023-11-18T20:43:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 -- 172.21.15.60:0/86366070 shutdown_connections 2023-11-18T20:43:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f564406ae40 0x7f564406d300 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56640cf6b0 0x7f56640ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56640b3410 0x7f56640c80e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 --2- 172.21.15.60:0/86366070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56640b1710 0x7f5664141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 -- 172.21.15.60:0/86366070 >> 172.21.15.60:0/86366070 conn(0x7f56640b28f0 msgr2=0x7f56640d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:57.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 -- 172.21.15.60:0/86366070 shutdown_connections 2023-11-18T20:43:57.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.037+0000 7f5663fff640 1 -- 172.21.15.60:0/86366070 wait complete. 2023-11-18T20:43:57.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:43:31.703619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:43:32.121610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-11-18T20:43:40.650315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:43:57.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:43:57.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.158+0000 7fb132333640 1 Processor -- start 2023-11-18T20:43:57.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.158+0000 7fb132333640 1 -- start start 2023-11-18T20:43:57.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.159+0000 7fb132333640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb12c143490 0x7fb12c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.159+0000 7fb132333640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15abf0 0x7fb12c0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.159+0000 7fb132333640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12c15e2a0 con 0x7fb12c15e730 2023-11-18T20:43:57.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.159+0000 7fb132333640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12c15b520 con 0x7fb12c143490 2023-11-18T20:43:57.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.159+0000 7fb132333640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c143870 con 0x7fb12c15abf0 2023-11-18T20:43:57.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.159+0000 7fb1308d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15abf0 0x7fb12c0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.159+0000 7fb12bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb12c143490 0x7fb12c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.159+0000 7fb1308d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15abf0 0x7fb12c0b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:49078/0 (socket says 172.21.15.60:49078) 2023-11-18T20:43:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.159+0000 7fb1308d0640 1 -- 172.21.15.60:0/1496317748 learned_addr learned my addr 172.21.15.60:0/1496317748 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.160+0000 7fb1308d0640 1 -- 172.21.15.60:0/1496317748 >> v2:172.21.15.100:3300/0 conn(0x7fb12c143490 msgr2=0x7fb12c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.160+0000 7fb1308d0640 1 --2- 172.21.15.60:0/1496317748 >> v2:172.21.15.100:3300/0 conn(0x7fb12c143490 0x7fb12c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.160+0000 7fb1308d0640 1 -- 172.21.15.60:0/1496317748 >> v1:172.21.15.100:6789/0 conn(0x7fb12c15e730 legacy=0x7fb12c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.160+0000 7fb1308d0640 1 -- 172.21.15.60:0/1496317748 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c15b240 con 0x7fb12c15abf0 2023-11-18T20:43:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.160+0000 7fb12bfff640 1 --2- 172.21.15.60:0/1496317748 >> v2:172.21.15.100:3300/0 conn(0x7fb12c143490 0x7fb12c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:43:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb1308d0640 1 --2- 172.21.15.60:0/1496317748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15abf0 0x7fb12c0b1230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb124009e50 tx=0x7fb12402bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c4849468a9b70f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb12affd640 1 -- 172.21.15.60:0/1496317748 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12402caf0 con 0x7fb12c15abf0 2023-11-18T20:43:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb12affd640 1 -- 172.21.15.60:0/1496317748 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12402cc50 con 0x7fb12c15abf0 2023-11-18T20:43:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb12affd640 1 -- 172.21.15.60:0/1496317748 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240343f0 con 0x7fb12c15abf0 2023-11-18T20:43:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb132333640 1 -- 172.21.15.60:0/1496317748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15abf0 msgr2=0x7fb12c0b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb132333640 1 --2- 172.21.15.60:0/1496317748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15abf0 0x7fb12c0b1230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb124009e50 tx=0x7fb12402bf50 comp rx=0 tx=0).stop 2023-11-18T20:43:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb132333640 1 -- 172.21.15.60:0/1496317748 shutdown_connections 2023-11-18T20:43:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb132333640 1 --2- 172.21.15.60:0/1496317748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15abf0 0x7fb12c0b1230 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb132333640 1 --2- 172.21.15.60:0/1496317748 >> v2:172.21.15.100:3300/0 conn(0x7fb12c143490 0x7fb12c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb132333640 1 -- 172.21.15.60:0/1496317748 >> 172.21.15.60:0/1496317748 conn(0x7fb12c06a4d0 msgr2=0x7fb12c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb132333640 1 -- 172.21.15.60:0/1496317748 shutdown_connections 2023-11-18T20:43:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.161+0000 7fb132333640 1 -- 172.21.15.60:0/1496317748 wait complete. 2023-11-18T20:43:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.162+0000 7fb132333640 1 Processor -- start 2023-11-18T20:43:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.163+0000 7fb132333640 1 -- start start 2023-11-18T20:43:57.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.163+0000 7fb132333640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb12c143490 0x7fb12c1f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.163+0000 7fb132333640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c15abf0 0x7fb12c0aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.163+0000 7fb132333640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15e730 0x7fb12c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.163+0000 7fb132333640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c0c4170 con 0x7fb12c143490 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.163+0000 7fb132333640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c0abc60 con 0x7fb12c15abf0 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.163+0000 7fb132333640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c1f4180 con 0x7fb12c15e730 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb12c143490 0x7fb12c1f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb12c143490 0x7fb12c1f3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48262/0 (socket says 172.21.15.60:48262) 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb12bfff640 1 -- 172.21.15.60:0/1754216191 learned_addr learned my addr 172.21.15.60:0/1754216191 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb12b7fe640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c15abf0 0x7fb12c0aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb1308d0640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15e730 0x7fb12c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb12bfff640 1 -- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15e730 msgr2=0x7fb12c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb12bfff640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15e730 0x7fb12c15c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb12bfff640 1 -- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c15abf0 msgr2=0x7fb12c0aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb12bfff640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c15abf0 0x7fb12c0aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb12bfff640 1 -- 172.21.15.60:0/1754216191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120009400 con 0x7fb12c143490 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb12bfff640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb12c143490 0x7fb12c1f3c40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb12000ebe0 tx=0x7fb12000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb128ff9640 1 -- 172.21.15.60:0/1754216191 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120009760 con 0x7fb12c143490 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb128ff9640 1 -- 172.21.15.60:0/1754216191 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12000cc30 con 0x7fb12c143490 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb128ff9640 1 -- 172.21.15.60:0/1754216191 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12000eee0 con 0x7fb12c143490 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb132333640 1 -- 172.21.15.60:0/1754216191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb124009b30 con 0x7fb12c143490 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.164+0000 7fb132333640 1 -- 172.21.15.60:0/1754216191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12c0ab330 con 0x7fb12c143490 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.165+0000 7fb132333640 1 -- 172.21.15.60:0/1754216191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12c069400 con 0x7fb12c143490 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.166+0000 7fb128ff9640 1 -- 172.21.15.60:0/1754216191 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb120013470 con 0x7fb12c143490 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.166+0000 7fb128ff9640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb11806ae40 0x7fb11806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.166+0000 7fb128ff9640 1 -- 172.21.15.60:0/1754216191 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fb120089b40 con 0x7fb12c143490 2023-11-18T20:43:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.167+0000 7fb12b7fe640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb11806ae40 0x7fb11806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.167+0000 7fb12b7fe640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb11806ae40 0x7fb11806d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb12c15d7c0 tx=0x7fb11c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:57.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.171+0000 7fb128ff9640 1 -- 172.21.15.60:0/1754216191 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1200549f0 con 0x7fb12c143490 2023-11-18T20:43:57.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.344+0000 7fb132333640 1 -- 172.21.15.60:0/1754216191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb12c0b2820 con 0x7fb12c143490 2023-11-18T20:43:57.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.345+0000 7fb128ff9640 1 -- 172.21.15.60:0/1754216191 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb120058010 con 0x7fb12c143490 2023-11-18T20:43:57.346 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:43:57.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.346+0000 7fb132333640 1 -- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb11806ae40 msgr2=0x7fb11806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.346+0000 7fb132333640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb11806ae40 0x7fb11806d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb12c15d7c0 tx=0x7fb11c009040 comp rx=0 tx=0).stop 2023-11-18T20:43:57.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.346+0000 7fb132333640 1 -- 172.21.15.60:0/1754216191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb12c143490 msgr2=0x7fb12c1f3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.346+0000 7fb132333640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb12c143490 0x7fb12c1f3c40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb12000ebe0 tx=0x7fb12000c520 comp rx=0 tx=0).stop 2023-11-18T20:43:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.346+0000 7fb132333640 1 -- 172.21.15.60:0/1754216191 shutdown_connections 2023-11-18T20:43:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.346+0000 7fb132333640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb11806ae40 0x7fb11806d300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.346+0000 7fb132333640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb12c15e730 0x7fb12c15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.346+0000 7fb132333640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb12c15abf0 0x7fb12c0aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.346+0000 7fb132333640 1 --2- 172.21.15.60:0/1754216191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb12c143490 0x7fb12c1f3c40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.346+0000 7fb132333640 1 -- 172.21.15.60:0/1754216191 >> 172.21.15.60:0/1754216191 conn(0x7fb12c06a4d0 msgr2=0x7fb12c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:57.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.347+0000 7fb132333640 1 -- 172.21.15.60:0/1754216191 shutdown_connections 2023-11-18T20:43:57.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.347+0000 7fb132333640 1 -- 172.21.15.60:0/1754216191 wait complete. 2023-11-18T20:43:57.359 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:43:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.467+0000 7f9f5f141640 1 Processor -- start 2023-11-18T20:43:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.467+0000 7f9f5f141640 1 -- start start 2023-11-18T20:43:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.468+0000 7f9f5f141640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f580b1400 0x7f9f5815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.468+0000 7f9f5f141640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9f5815e730 0x7f9f5815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.468+0000 7f9f5f141640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5815e2a0 con 0x7f9f58143490 2023-11-18T20:43:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.468+0000 7f9f5f141640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5815b520 con 0x7f9f5815e730 2023-11-18T20:43:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.468+0000 7f9f5f141640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58143870 con 0x7f9f580b1400 2023-11-18T20:43:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.468+0000 7f9f5d6de640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9f5815e730 0x7f9f5815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.468+0000 7f9f57fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f580b1400 0x7f9f5815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.468+0000 7f9f5cedd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9f58143490 0x7f9f580b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48592/0 (socket says 172.21.15.60:48592) 2023-11-18T20:43:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.468+0000 7f9f5cedd640 1 -- 172.21.15.60:0/79381458 learned_addr learned my addr 172.21.15.60:0/79381458 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:57.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.469+0000 7f9f57fff640 1 -- 172.21.15.60:0/79381458 >> v2:172.21.15.100:3300/0 conn(0x7f9f5815e730 msgr2=0x7f9f5815aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.469+0000 7f9f57fff640 1 --2- 172.21.15.60:0/79381458 >> v2:172.21.15.100:3300/0 conn(0x7f9f5815e730 0x7f9f5815aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.469+0000 7f9f57fff640 1 -- 172.21.15.60:0/79381458 >> v1:172.21.15.100:6789/0 conn(0x7f9f58143490 legacy=0x7f9f580b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.469+0000 7f9f57fff640 1 -- 172.21.15.60:0/79381458 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5815b240 con 0x7f9f580b1400 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.469+0000 7f9f57fff640 1 --2- 172.21.15.60:0/79381458 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f580b1400 0x7f9f5815a4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0097a0 tx=0x7f9f4c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca0975413dc6b5b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f577fe640 1 -- 172.21.15.60:0/79381458 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c031830 con 0x7f9f580b1400 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f577fe640 1 -- 172.21.15.60:0/79381458 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c031990 con 0x7f9f580b1400 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f577fe640 1 -- 172.21.15.60:0/79381458 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c031d20 con 0x7f9f580b1400 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f5f141640 1 -- 172.21.15.60:0/79381458 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f580b1400 msgr2=0x7f9f5815a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f5f141640 1 --2- 172.21.15.60:0/79381458 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f580b1400 0x7f9f5815a4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0097a0 tx=0x7f9f4c02b8d0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f5f141640 1 -- 172.21.15.60:0/79381458 shutdown_connections 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f5f141640 1 --2- 172.21.15.60:0/79381458 >> v2:172.21.15.100:3300/0 conn(0x7f9f5815e730 0x7f9f5815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f5f141640 1 --2- 172.21.15.60:0/79381458 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f580b1400 0x7f9f5815a4e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f5f141640 1 -- 172.21.15.60:0/79381458 >> 172.21.15.60:0/79381458 conn(0x7f9f5806a4d0 msgr2=0x7f9f5806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f5f141640 1 -- 172.21.15.60:0/79381458 shutdown_connections 2023-11-18T20:43:57.473 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.470+0000 7f9f5f141640 1 -- 172.21.15.60:0/79381458 wait complete. 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.471+0000 7f9f5f141640 1 Processor -- start 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.471+0000 7f9f5f141640 1 -- start start 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.472+0000 7f9f5f141640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f580b1400 0x7f9f58159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.472+0000 7f9f5f141640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58143490 0x7f9f5806c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.472+0000 7f9f5f141640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f5815e730 0x7f9f5806c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.472+0000 7f9f5f141640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5815b990 con 0x7f9f580b1400 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.472+0000 7f9f5f141640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5815b660 con 0x7f9f58143490 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.472+0000 7f9f5f141640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5813d450 con 0x7f9f5815e730 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.472+0000 7f9f5cedd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f580b1400 0x7f9f58159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.472+0000 7f9f5cedd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f580b1400 0x7f9f58159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48268/0 (socket says 172.21.15.60:48268) 2023-11-18T20:43:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.472+0000 7f9f5cedd640 1 -- 172.21.15.60:0/3419541891 learned_addr learned my addr 172.21.15.60:0/3419541891 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f57fff640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58143490 0x7f9f5806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f5d6de640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f5815e730 0x7f9f5806c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f5cedd640 1 -- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f5815e730 msgr2=0x7f9f5806c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f5cedd640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f5815e730 0x7f9f5806c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f5cedd640 1 -- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58143490 msgr2=0x7f9f5806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f5cedd640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58143490 0x7f9f5806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f5cedd640 1 -- 172.21.15.60:0/3419541891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c009400 con 0x7f9f580b1400 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f5cedd640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f580b1400 0x7f9f58159910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000cba0 tx=0x7f9f5000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f557fa640 1 -- 172.21.15.60:0/3419541891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000ad40 con 0x7f9f580b1400 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f557fa640 1 -- 172.21.15.60:0/3419541891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f50012400 con 0x7f9f580b1400 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f5f141640 1 -- 172.21.15.60:0/3419541891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5813d730 con 0x7f9f580b1400 2023-11-18T20:43:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f557fa640 1 -- 172.21.15.60:0/3419541891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000aea0 con 0x7f9f580b1400 2023-11-18T20:43:57.477 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.473+0000 7f9f5f141640 1 -- 172.21.15.60:0/3419541891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f58159e50 con 0x7f9f580b1400 2023-11-18T20:43:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.474+0000 7f9f5f141640 1 -- 172.21.15.60:0/3419541891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f58069400 con 0x7f9f580b1400 2023-11-18T20:43:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.475+0000 7f9f557fa640 1 -- 172.21.15.60:0/3419541891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f50007070 con 0x7f9f580b1400 2023-11-18T20:43:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.475+0000 7f9f557fa640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f4406ae40 0x7f9f4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.475+0000 7f9f557fa640 1 -- 172.21.15.60:0/3419541891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f9f50089650 con 0x7f9f580b1400 2023-11-18T20:43:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.475+0000 7f9f57fff640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f4406ae40 0x7f9f4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.476+0000 7f9f57fff640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f4406ae40 0x7f9f4406d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c036000 tx=0x7f9f4c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:57.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.479+0000 7f9f557fa640 1 -- 172.21.15.60:0/3419541891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f50054620 con 0x7f9f580b1400 2023-11-18T20:43:57.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.653+0000 7f9f5f141640 1 -- 172.21.15.60:0/3419541891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f580ced40 con 0x7f9f580b1400 2023-11-18T20:43:57.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.654+0000 7f9f557fa640 1 -- 172.21.15.60:0/3419541891 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f50057c40 con 0x7f9f580b1400 2023-11-18T20:43:57.655 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:57.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.655+0000 7f9f5f141640 1 -- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f4406ae40 msgr2=0x7f9f4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.655+0000 7f9f5f141640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f4406ae40 0x7f9f4406d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c036000 tx=0x7f9f4c03c040 comp rx=0 tx=0).stop 2023-11-18T20:43:57.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.655+0000 7f9f5f141640 1 -- 172.21.15.60:0/3419541891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f580b1400 msgr2=0x7f9f58159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.655+0000 7f9f5f141640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f580b1400 0x7f9f58159910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000cba0 tx=0x7f9f5000a430 comp rx=0 tx=0).stop 2023-11-18T20:43:57.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.656+0000 7f9f5f141640 1 -- 172.21.15.60:0/3419541891 shutdown_connections 2023-11-18T20:43:57.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.656+0000 7f9f5f141640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9f4406ae40 0x7f9f4406d300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.656+0000 7f9f5f141640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f5815e730 0x7f9f5806c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.656+0000 7f9f5f141640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58143490 0x7f9f5806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.656+0000 7f9f5f141640 1 --2- 172.21.15.60:0/3419541891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f580b1400 0x7f9f58159910 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.656+0000 7f9f5f141640 1 -- 172.21.15.60:0/3419541891 >> 172.21.15.60:0/3419541891 conn(0x7f9f5806a4d0 msgr2=0x7f9f580cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:57.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.656+0000 7f9f5f141640 1 -- 172.21.15.60:0/3419541891 shutdown_connections 2023-11-18T20:43:57.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.656+0000 7f9f5f141640 1 -- 172.21.15.60:0/3419541891 wait complete. 2023-11-18T20:43:57.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:43:57.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.776+0000 7f5824809640 1 Processor -- start 2023-11-18T20:43:57.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.776+0000 7f5824809640 1 -- start start 2023-11-18T20:43:57.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.777+0000 7f5824809640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58200d2b00 0x7f58201682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.777+0000 7f5824809640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58201614e0 con 0x7f5820160f70 2023-11-18T20:43:57.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.777+0000 7f5824809640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58201617c0 con 0x7f582015d2d0 2023-11-18T20:43:57.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.777+0000 7f5824809640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5820161aa0 con 0x7f58200d2b00 2023-11-18T20:43:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.777+0000 7f581dd9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5820160f70 0x7f58200cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46428/0 (socket says 172.21.15.60:46428) 2023-11-18T20:43:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.777+0000 7f581dd9b640 1 -- 172.21.15.60:0/2038466009 learned_addr learned my addr 172.21.15.60:0/2038466009 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.777+0000 7f581ed9d640 1 --2- 172.21.15.60:0/2038466009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58200d2b00 0x7f58201682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.777+0000 7f581d59a640 1 -- 172.21.15.60:0/2038466009 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066970264 0 0) 0x7f58201614e0 con 0x7f5820160f70 2023-11-18T20:43:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.778+0000 7f581d59a640 1 -- 172.21.15.60:0/2038466009 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f580c003520 con 0x7f5820160f70 2023-11-18T20:43:57.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.778+0000 7f581d59a640 1 -- 172.21.15.60:0/2038466009 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622753352 0 0) 0x7f58201617c0 con 0x7f582015d2d0 2023-11-18T20:43:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.778+0000 7f581d59a640 1 -- 172.21.15.60:0/2038466009 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58201614e0 con 0x7f582015d2d0 2023-11-18T20:43:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.778+0000 7f581ed9d640 1 -- 172.21.15.60:0/2038466009 >> v1:172.21.15.100:6789/0 conn(0x7f582015d2d0 legacy=0x7f58200b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.778+0000 7f581d59a640 1 -- 172.21.15.60:0/2038466009 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2491745383 0 0) 0x7f580c003520 con 0x7f5820160f70 2023-11-18T20:43:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.778+0000 7f581ed9d640 1 -- 172.21.15.60:0/2038466009 >> v1:172.21.15.60:6789/0 conn(0x7f5820160f70 legacy=0x7f58200cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.778+0000 7f581ed9d640 1 -- 172.21.15.60:0/2038466009 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5820161d80 con 0x7f58200d2b00 2023-11-18T20:43:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f581ed9d640 1 --2- 172.21.15.60:0/2038466009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58200d2b00 0x7f58201682a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f58180097d0 tx=0x7f581802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5e9df0dc2e5945 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f581d59a640 1 -- 172.21.15.60:0/2038466009 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808001670 con 0x7f58200d2b00 2023-11-18T20:43:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f581d59a640 1 -- 172.21.15.60:0/2038466009 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581802e050 con 0x7f58200d2b00 2023-11-18T20:43:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f581d59a640 1 -- 172.21.15.60:0/2038466009 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818030730 con 0x7f58200d2b00 2023-11-18T20:43:57.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f5824809640 1 -- 172.21.15.60:0/2038466009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58200d2b00 msgr2=0x7f58201682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f5824809640 1 --2- 172.21.15.60:0/2038466009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58200d2b00 0x7f58201682a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f58180097d0 tx=0x7f581802b800 comp rx=0 tx=0).stop 2023-11-18T20:43:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f5824809640 1 -- 172.21.15.60:0/2038466009 shutdown_connections 2023-11-18T20:43:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f5824809640 1 --2- 172.21.15.60:0/2038466009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58200d2b00 0x7f58201682a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f5824809640 1 -- 172.21.15.60:0/2038466009 >> 172.21.15.60:0/2038466009 conn(0x7f5820072230 msgr2=0x7f58200cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f5824809640 1 -- 172.21.15.60:0/2038466009 shutdown_connections 2023-11-18T20:43:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.779+0000 7f5824809640 1 -- 172.21.15.60:0/2038466009 wait complete. 2023-11-18T20:43:57.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.781+0000 7f5824809640 1 Processor -- start 2023-11-18T20:43:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.781+0000 7f5824809640 1 -- start start 2023-11-18T20:43:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.781+0000 7f5824809640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58200d2b00 0x7f58200b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.781+0000 7f5824809640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f582015d2d0 0x7f582014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f5824809640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5820160f70 0x7f58201f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f5824809640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582015e150 con 0x7f5820160f70 2023-11-18T20:43:57.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f5824809640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58200b5c20 con 0x7f58200d2b00 2023-11-18T20:43:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f5824809640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582014fc10 con 0x7f582015d2d0 2023-11-18T20:43:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5820160f70 0x7f58201f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5820160f70 0x7f58201f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48270/0 (socket says 172.21.15.60:48270) 2023-11-18T20:43:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581ed9d640 1 -- 172.21.15.60:0/434672080 learned_addr learned my addr 172.21.15.60:0/434672080 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581e59c640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58200d2b00 0x7f58200b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581dd9b640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f582015d2d0 0x7f582014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581ed9d640 1 -- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f582015d2d0 msgr2=0x7f582014f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581ed9d640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f582015d2d0 0x7f582014f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581ed9d640 1 -- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58200d2b00 msgr2=0x7f58200b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581ed9d640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58200d2b00 0x7f58200b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581ed9d640 1 -- 172.21.15.60:0/434672080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5818009400 con 0x7f5820160f70 2023-11-18T20:43:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f581ed9d640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5820160f70 0x7f58201f6590 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5818009720 tx=0x7f58180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f5816ffd640 1 -- 172.21.15.60:0/434672080 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818004230 con 0x7f5820160f70 2023-11-18T20:43:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f5816ffd640 1 -- 172.21.15.60:0/434672080 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5818004390 con 0x7f5820160f70 2023-11-18T20:43:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f5824809640 1 -- 172.21.15.60:0/434672080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58200cd330 con 0x7f5820160f70 2023-11-18T20:43:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f5816ffd640 1 -- 172.21.15.60:0/434672080 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818003850 con 0x7f5820160f70 2023-11-18T20:43:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.782+0000 7f5824809640 1 -- 172.21.15.60:0/434672080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58200ccd70 con 0x7f5820160f70 2023-11-18T20:43:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.783+0000 7f5824809640 1 -- 172.21.15.60:0/434672080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582015e2c0 con 0x7f5820160f70 2023-11-18T20:43:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.784+0000 7f5816ffd640 1 -- 172.21.15.60:0/434672080 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5818039040 con 0x7f5820160f70 2023-11-18T20:43:57.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.784+0000 7f5816ffd640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f580006ae70 0x7f580006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:57.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.784+0000 7f5816ffd640 1 -- 172.21.15.60:0/434672080 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f58180aaae0 con 0x7f5820160f70 2023-11-18T20:43:57.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.784+0000 7f581e59c640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f580006ae70 0x7f580006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:57.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.785+0000 7f581e59c640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f580006ae70 0x7f580006d330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5810001560 tx=0x7f5810009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:57.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.789+0000 7f5816ffd640 1 -- 172.21.15.60:0/434672080 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f582015e2c0 con 0x7f5820160f70 2023-11-18T20:43:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.967+0000 7f5824809640 1 -- 172.21.15.60:0/434672080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f582015e290 con 0x7f5820160f70 2023-11-18T20:43:57.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.967+0000 7f5816ffd640 1 -- 172.21.15.60:0/434672080 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5818037020 con 0x7f5820160f70 2023-11-18T20:43:57.969 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 -- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f580006ae70 msgr2=0x7f580006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f580006ae70 0x7f580006d330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5810001560 tx=0x7f5810009040 comp rx=0 tx=0).stop 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 -- 172.21.15.60:0/434672080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5820160f70 msgr2=0x7f58201f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5820160f70 0x7f58201f6590 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5818009720 tx=0x7f58180046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 -- 172.21.15.60:0/434672080 shutdown_connections 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f580006ae70 0x7f580006d330 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5820160f70 0x7f58201f6590 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f582015d2d0 0x7f582014f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 --2- 172.21.15.60:0/434672080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58200d2b00 0x7f58200b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 -- 172.21.15.60:0/434672080 >> 172.21.15.60:0/434672080 conn(0x7f5820072230 msgr2=0x7f58200b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 -- 172.21.15.60:0/434672080 shutdown_connections 2023-11-18T20:43:57.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:57.969+0000 7f5824809640 1 -- 172.21.15.60:0/434672080 wait complete. 2023-11-18T20:43:57.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-11-18T20:43:58.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.089+0000 7f7ba9a07640 1 Processor -- start 2023-11-18T20:43:58.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.089+0000 7f7ba9a07640 1 -- start start 2023-11-18T20:43:58.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.090+0000 7f7ba9a07640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7ba40d2b00 0x7f7ba41682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.090+0000 7f7ba9a07640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba41614e0 con 0x7f7ba4160f70 2023-11-18T20:43:58.092 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.090+0000 7f7ba9a07640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba41617c0 con 0x7f7ba415d2d0 2023-11-18T20:43:58.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.090+0000 7f7ba9a07640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba4161aa0 con 0x7f7ba40d2b00 2023-11-18T20:43:58.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.090+0000 7f7ba37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7ba40d2b00 0x7f7ba41682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.090+0000 7f7ba27fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7ba4160f70 0x7f7ba40cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46430/0 (socket says 172.21.15.60:46430) 2023-11-18T20:43:58.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.090+0000 7f7ba27fc640 1 -- 172.21.15.60:0/1852566750 learned_addr learned my addr 172.21.15.60:0/1852566750 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:58.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.091+0000 7f7ba37fe640 1 -- 172.21.15.60:0/1852566750 >> v1:172.21.15.100:6789/0 conn(0x7f7ba415d2d0 legacy=0x7f7ba40b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:58.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.091+0000 7f7ba37fe640 1 -- 172.21.15.60:0/1852566750 >> v1:172.21.15.60:6789/0 conn(0x7f7ba4160f70 legacy=0x7f7ba40cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.091+0000 7f7ba37fe640 1 -- 172.21.15.60:0/1852566750 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba4161d80 con 0x7f7ba40d2b00 2023-11-18T20:43:58.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.091+0000 7f7ba1ffb640 1 -- 172.21.15.60:0/1852566750 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641020268 0 0) 0x7f7ba41614e0 con 0x7f7ba4160f70 2023-11-18T20:43:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.091+0000 7f7ba37fe640 1 --2- 172.21.15.60:0/1852566750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba40d2b00 0x7f7ba41682a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009d90 tx=0x7f7b9402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1cb2e519d9938b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.092+0000 7f7ba1ffb640 1 -- 172.21.15.60:0/1852566750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9402cc60 con 0x7f7ba40d2b00 2023-11-18T20:43:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.092+0000 7f7ba1ffb640 1 -- 172.21.15.60:0/1852566750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9402cdc0 con 0x7f7ba40d2b00 2023-11-18T20:43:58.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.092+0000 7f7ba1ffb640 1 -- 172.21.15.60:0/1852566750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94034660 con 0x7f7ba40d2b00 2023-11-18T20:43:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.092+0000 7f7ba9a07640 1 -- 172.21.15.60:0/1852566750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba40d2b00 msgr2=0x7f7ba41682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.092+0000 7f7ba9a07640 1 --2- 172.21.15.60:0/1852566750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba40d2b00 0x7f7ba41682a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009d90 tx=0x7f7b9402beb0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.092+0000 7f7ba9a07640 1 -- 172.21.15.60:0/1852566750 shutdown_connections 2023-11-18T20:43:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.092+0000 7f7ba9a07640 1 --2- 172.21.15.60:0/1852566750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba40d2b00 0x7f7ba41682a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.092+0000 7f7ba9a07640 1 -- 172.21.15.60:0/1852566750 >> 172.21.15.60:0/1852566750 conn(0x7f7ba4072230 msgr2=0x7f7ba40cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.092+0000 7f7ba9a07640 1 -- 172.21.15.60:0/1852566750 shutdown_connections 2023-11-18T20:43:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.092+0000 7f7ba9a07640 1 -- 172.21.15.60:0/1852566750 wait complete. 2023-11-18T20:43:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.093+0000 7f7ba9a07640 1 Processor -- start 2023-11-18T20:43:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.094+0000 7f7ba9a07640 1 -- start start 2023-11-18T20:43:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.094+0000 7f7ba9a07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba40d2b00 0x7f7ba40cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.094+0000 7f7ba9a07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba415d2d0 0x7f7ba4151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.094+0000 7f7ba9a07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba4160f70 0x7f7ba41593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.094+0000 7f7ba9a07640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba41f6480 con 0x7f7ba415d2d0 2023-11-18T20:43:58.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.094+0000 7f7ba9a07640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba41f66d0 con 0x7f7ba4160f70 2023-11-18T20:43:58.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.094+0000 7f7ba9a07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba41f69b0 con 0x7f7ba40d2b00 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.094+0000 7f7ba27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba415d2d0 0x7f7ba4151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.094+0000 7f7ba27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba415d2d0 0x7f7ba4151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48280/0 (socket says 172.21.15.60:48280) 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.094+0000 7f7ba27fc640 1 -- 172.21.15.60:0/951509347 learned_addr learned my addr 172.21.15.60:0/951509347 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.095+0000 7f7ba37fe640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba4160f70 0x7f7ba41593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.095+0000 7f7ba2ffd640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba40d2b00 0x7f7ba40cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.095+0000 7f7ba27fc640 1 -- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba40d2b00 msgr2=0x7f7ba40cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.095+0000 7f7ba27fc640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba40d2b00 0x7f7ba40cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.095+0000 7f7ba27fc640 1 -- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba4160f70 msgr2=0x7f7ba41593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.095+0000 7f7ba27fc640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba4160f70 0x7f7ba41593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.095+0000 7f7ba27fc640 1 -- 172.21.15.60:0/951509347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b94009a70 con 0x7f7ba415d2d0 2023-11-18T20:43:58.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.095+0000 7f7ba37fe640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba4160f70 0x7f7ba41593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:43:58.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.095+0000 7f7ba2ffd640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba40d2b00 0x7f7ba40cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:58.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.095+0000 7f7ba27fc640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba415d2d0 0x7f7ba4151ff0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c430 tx=0x7f7b8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:58.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.097+0000 7f7b9bfff640 1 -- 172.21.15.60:0/951509347 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0074e0 con 0x7f7ba415d2d0 2023-11-18T20:43:58.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.097+0000 7f7b9bfff640 1 -- 172.21.15.60:0/951509347 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c007660 con 0x7f7ba415d2d0 2023-11-18T20:43:58.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.097+0000 7f7b9bfff640 1 -- 172.21.15.60:0/951509347 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c004a20 con 0x7f7ba415d2d0 2023-11-18T20:43:58.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.097+0000 7f7ba9a07640 1 -- 172.21.15.60:0/951509347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba41f6c90 con 0x7f7ba415d2d0 2023-11-18T20:43:58.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.097+0000 7f7ba9a07640 1 -- 172.21.15.60:0/951509347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba41f7250 con 0x7f7ba415d2d0 2023-11-18T20:43:58.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.098+0000 7f7ba9a07640 1 -- 172.21.15.60:0/951509347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba4069400 con 0x7f7ba415d2d0 2023-11-18T20:43:58.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.098+0000 7f7b9bfff640 1 -- 172.21.15.60:0/951509347 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b8c004b80 con 0x7f7ba415d2d0 2023-11-18T20:43:58.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.099+0000 7f7b9bfff640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b8406ae40 0x7f7b8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.099+0000 7f7b9bfff640 1 -- 172.21.15.60:0/951509347 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f7b8c0895b0 con 0x7f7ba415d2d0 2023-11-18T20:43:58.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.099+0000 7f7ba2ffd640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b8406ae40 0x7f7b8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.099+0000 7f7ba2ffd640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b8406ae40 0x7f7b8406d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c003eb0 tx=0x7f7b9c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:58.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.103+0000 7f7b9bfff640 1 -- 172.21.15.60:0/951509347 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b8c0544d0 con 0x7f7ba415d2d0 2023-11-18T20:43:58.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.284+0000 7f7ba9a07640 1 -- 172.21.15.60:0/951509347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f7ba40d0aa0 con 0x7f7ba415d2d0 2023-11-18T20:43:58.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.285+0000 7f7b9bfff640 1 -- 172.21.15.60:0/951509347 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7f7b8c057af0 con 0x7f7ba415d2d0 2023-11-18T20:43:58.286 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:43:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 -- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b8406ae40 msgr2=0x7f7b8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b8406ae40 0x7f7b8406d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c003eb0 tx=0x7f7b9c009040 comp rx=0 tx=0).stop 2023-11-18T20:43:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 -- 172.21.15.60:0/951509347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba415d2d0 msgr2=0x7f7ba4151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba415d2d0 0x7f7ba4151ff0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c430 tx=0x7f7b8c00c8f0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 -- 172.21.15.60:0/951509347 shutdown_connections 2023-11-18T20:43:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7b8406ae40 0x7f7b8406d300 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba4160f70 0x7f7ba41593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba415d2d0 0x7f7ba4151ff0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 --2- 172.21.15.60:0/951509347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba40d2b00 0x7f7ba40cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 -- 172.21.15.60:0/951509347 >> 172.21.15.60:0/951509347 conn(0x7f7ba4072230 msgr2=0x7f7ba4155c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:58.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 -- 172.21.15.60:0/951509347 shutdown_connections 2023-11-18T20:43:58.289 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.287+0000 7f7ba9a07640 1 -- 172.21.15.60:0/951509347 wait complete. 2023-11-18T20:43:58.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:43:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.405+0000 7f6938890640 1 Processor -- start 2023-11-18T20:43:58.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.405+0000 7f6938890640 1 -- start start 2023-11-18T20:43:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.406+0000 7f6938890640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f693415b130 0x7f69340d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.406+0000 7f6938890640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f693415f760 0x7f69340cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.406+0000 7f6938890640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69340cf560 con 0x7f69340cf150 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.406+0000 7f6938890640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693415fd20 con 0x7f693415f760 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.406+0000 7f6938890640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69340d0730 con 0x7f693415b130 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.406+0000 7f6931d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f693415f760 0x7f69340cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.406+0000 7f6932d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f69340cf150 0x7f693406ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:46446/0 (socket says 172.21.15.60:46446) 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.406+0000 7f6932d9d640 1 -- 172.21.15.60:0/3409613176 learned_addr learned my addr 172.21.15.60:0/3409613176 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.407+0000 7f693259c640 1 --2- 172.21.15.60:0/3409613176 >> v2:172.21.15.100:3300/0 conn(0x7f693415b130 0x7f69340d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.407+0000 7f6931d9b640 1 -- 172.21.15.60:0/3409613176 >> v2:172.21.15.100:3300/0 conn(0x7f693415b130 msgr2=0x7f69340d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.407+0000 7f6931d9b640 1 --2- 172.21.15.60:0/3409613176 >> v2:172.21.15.100:3300/0 conn(0x7f693415b130 0x7f69340d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.407+0000 7f6931d9b640 1 -- 172.21.15.60:0/3409613176 >> v1:172.21.15.60:6789/0 conn(0x7f69340cf150 legacy=0x7f693406ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.407+0000 7f6931d9b640 1 -- 172.21.15.60:0/3409613176 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340b3f10 con 0x7f693415f760 2023-11-18T20:43:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.407+0000 7f693159a640 1 -- 172.21.15.60:0/3409613176 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615385922 0 0) 0x7f69340cf560 con 0x7f69340cf150 2023-11-18T20:43:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.407+0000 7f693259c640 1 --2- 172.21.15.60:0/3409613176 >> v2:172.21.15.100:3300/0 conn(0x7f693415b130 0x7f69340d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:43:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.407+0000 7f6931d9b640 1 --2- 172.21.15.60:0/3409613176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415f760 0x7f69340cec10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f693406ede0 tx=0x7f691c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9d5594dc4970e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f693159a640 1 -- 172.21.15.60:0/3409613176 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c02e070 con 0x7f693415f760 2023-11-18T20:43:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f693159a640 1 -- 172.21.15.60:0/3409613176 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c0315a0 con 0x7f693415f760 2023-11-18T20:43:58.410 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f693159a640 1 -- 172.21.15.60:0/3409613176 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c031930 con 0x7f693415f760 2023-11-18T20:43:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f6938890640 1 -- 172.21.15.60:0/3409613176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415f760 msgr2=0x7f69340cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f6938890640 1 --2- 172.21.15.60:0/3409613176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415f760 0x7f69340cec10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f693406ede0 tx=0x7f691c02b840 comp rx=0 tx=0).stop 2023-11-18T20:43:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f6938890640 1 -- 172.21.15.60:0/3409613176 shutdown_connections 2023-11-18T20:43:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f6938890640 1 --2- 172.21.15.60:0/3409613176 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415f760 0x7f69340cec10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f6938890640 1 --2- 172.21.15.60:0/3409613176 >> v2:172.21.15.100:3300/0 conn(0x7f693415b130 0x7f69340d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f6938890640 1 -- 172.21.15.60:0/3409613176 >> 172.21.15.60:0/3409613176 conn(0x7f69340b2c10 msgr2=0x7f69340b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f6938890640 1 -- 172.21.15.60:0/3409613176 shutdown_connections 2023-11-18T20:43:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.408+0000 7f6938890640 1 -- 172.21.15.60:0/3409613176 wait complete. 2023-11-18T20:43:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.409+0000 7f6938890640 1 Processor -- start 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.410+0000 7f6938890640 1 -- start start 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.410+0000 7f6938890640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69340cf150 0x7f69340b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.410+0000 7f6938890640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415b130 0x7f693406cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.410+0000 7f6938890640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f693415f760 0x7f69340c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.410+0000 7f6938890640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693406c7e0 con 0x7f693415b130 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.410+0000 7f6938890640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693406d140 con 0x7f69340cf150 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.410+0000 7f6938890640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6934156850 con 0x7f693415f760 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.410+0000 7f6931d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415b130 0x7f693406cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.410+0000 7f6931d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415b130 0x7f693406cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48294/0 (socket says 172.21.15.60:48294) 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.410+0000 7f6931d9b640 1 -- 172.21.15.60:0/263438871 learned_addr learned my addr 172.21.15.60:0/263438871 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f6932d9d640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f693415f760 0x7f69340c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f693259c640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69340cf150 0x7f69340b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f6931d9b640 1 -- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f693415f760 msgr2=0x7f69340c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f6931d9b640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f693415f760 0x7f69340c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f6931d9b640 1 -- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69340cf150 msgr2=0x7f69340b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f6931d9b640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69340cf150 0x7f69340b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f6931d9b640 1 -- 172.21.15.60:0/263438871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c009400 con 0x7f693415b130 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f6931d9b640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415b130 0x7f693406cc00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f691c009720 tx=0x7f691c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f692affd640 1 -- 172.21.15.60:0/263438871 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c004230 con 0x7f693415b130 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f692affd640 1 -- 172.21.15.60:0/263438871 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c0043b0 con 0x7f693415b130 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f6938890640 1 -- 172.21.15.60:0/263438871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6934156570 con 0x7f693415b130 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f692affd640 1 -- 172.21.15.60:0/263438871 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c004520 con 0x7f693415b130 2023-11-18T20:43:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.411+0000 7f6938890640 1 -- 172.21.15.60:0/263438871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693406c400 con 0x7f693415b130 2023-11-18T20:43:58.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.412+0000 7f6938890640 1 -- 172.21.15.60:0/263438871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6934067fc0 con 0x7f693415b130 2023-11-18T20:43:58.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.413+0000 7f692affd640 1 -- 172.21.15.60:0/263438871 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f691c032880 con 0x7f693415b130 2023-11-18T20:43:58.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.413+0000 7f692affd640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f691406ae40 0x7f691406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:43:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.413+0000 7f692affd640 1 -- 172.21.15.60:0/263438871 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f691c0ab870 con 0x7f693415b130 2023-11-18T20:43:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.413+0000 7f693259c640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f691406ae40 0x7f691406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:43:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.414+0000 7f693259c640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f691406ae40 0x7f691406d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f69240075b0 tx=0x7f6924007540 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:43:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.417+0000 7f692affd640 1 -- 172.21.15.60:0/263438871 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f691c076820 con 0x7f693415b130 2023-11-18T20:43:58.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.606+0000 7f6938890640 1 -- 172.21.15.60:0/263438871 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6934067420 con 0x7f693415b130 2023-11-18T20:43:58.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.606+0000 7f692affd640 1 -- 172.21.15.60:0/263438871 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f691c079e40 con 0x7f693415b130 2023-11-18T20:43:58.608 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:43:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 -- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f691406ae40 msgr2=0x7f691406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f691406ae40 0x7f691406d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f69240075b0 tx=0x7f6924007540 comp rx=0 tx=0).stop 2023-11-18T20:43:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 -- 172.21.15.60:0/263438871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415b130 msgr2=0x7f693406cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:43:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415b130 0x7f693406cc00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f691c009720 tx=0x7f691c0046c0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 -- 172.21.15.60:0/263438871 shutdown_connections 2023-11-18T20:43:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f691406ae40 0x7f691406d300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f693415f760 0x7f69340c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f693415b130 0x7f693406cc00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 --2- 172.21.15.60:0/263438871 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69340cf150 0x7f69340b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:43:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 -- 172.21.15.60:0/263438871 >> 172.21.15.60:0/263438871 conn(0x7f69340b2c10 msgr2=0x7f69340b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:43:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 -- 172.21.15.60:0/263438871 shutdown_connections 2023-11-18T20:43:58.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:43:58.608+0000 7f6938890640 1 -- 172.21.15.60:0/263438871 wait complete. 2023-11-18T20:43:58.620 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-11-18T20:43:58.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:43:58.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-18T20:43:58.649 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[102433]: fuse finished with error 0 and tester_r 0 2023-11-18T20:43:58.651 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:44:04.753 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:44:04.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:04.754 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-18T20:44:04.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2023-11-18T20:44:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.937+0000 7fa35e87c640 1 Processor -- start 2023-11-18T20:44:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.938+0000 7fa35e87c640 1 -- start start 2023-11-18T20:44:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.938+0000 7fa35e87c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa35815e300 0x7fa35815b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.938+0000 7fa35e87c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa35815e6e0 0x7fa358071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.938+0000 7fa35e87c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35815aea0 con 0x7fa3580b0bf0 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.938+0000 7fa35e87c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa358072090 con 0x7fa35815e6e0 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.938+0000 7fa35e87c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa358072370 con 0x7fa35815e300 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.939+0000 7fa3577fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa35815e6e0 0x7fa358071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.939+0000 7fa3577fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa35815e6e0 0x7fa358071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54970/0 (socket says 172.21.15.60:54970) 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.939+0000 7fa3577fe640 1 -- 172.21.15.60:0/4146911207 learned_addr learned my addr 172.21.15.60:0/4146911207 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.939+0000 7fa357fff640 1 --2- 172.21.15.60:0/4146911207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa35815e300 0x7fa35815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.939+0000 7fa3577fe640 1 -- 172.21.15.60:0/4146911207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa35815e300 msgr2=0x7fa35815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.939+0000 7fa3577fe640 1 --2- 172.21.15.60:0/4146911207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa35815e300 0x7fa35815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.939+0000 7fa3577fe640 1 -- 172.21.15.60:0/4146911207 >> v1:172.21.15.100:6789/0 conn(0x7fa3580b0bf0 legacy=0x7fa3580ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.939+0000 7fa3577fe640 1 -- 172.21.15.60:0/4146911207 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35815ab30 con 0x7fa35815e6e0 2023-11-18T20:44:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.939+0000 7fa357fff640 1 --2- 172.21.15.60:0/4146911207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa35815e300 0x7fa35815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:04.942 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa3577fe640 1 --2- 172.21.15.60:0/4146911207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e6e0 0x7fa358071840 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009870 tx=0x7fa34c02db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58442bb46645625 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:04.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa356ffd640 1 -- 172.21.15.60:0/4146911207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c02f8c0 con 0x7fa35815e6e0 2023-11-18T20:44:04.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa356ffd640 1 -- 172.21.15.60:0/4146911207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c02fa20 con 0x7fa35815e6e0 2023-11-18T20:44:04.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa356ffd640 1 -- 172.21.15.60:0/4146911207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c02fdb0 con 0x7fa35815e6e0 2023-11-18T20:44:04.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa35e87c640 1 -- 172.21.15.60:0/4146911207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e6e0 msgr2=0x7fa358071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:04.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa35e87c640 1 --2- 172.21.15.60:0/4146911207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e6e0 0x7fa358071840 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009870 tx=0x7fa34c02db00 comp rx=0 tx=0).stop 2023-11-18T20:44:04.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa35e87c640 1 -- 172.21.15.60:0/4146911207 shutdown_connections 2023-11-18T20:44:04.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa35e87c640 1 --2- 172.21.15.60:0/4146911207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e6e0 0x7fa358071840 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:04.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa35e87c640 1 --2- 172.21.15.60:0/4146911207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa35815e300 0x7fa35815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:04.945 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa35e87c640 1 -- 172.21.15.60:0/4146911207 >> 172.21.15.60:0/4146911207 conn(0x7fa35806d9c0 msgr2=0x7fa35806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa35e87c640 1 -- 172.21.15.60:0/4146911207 shutdown_connections 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.940+0000 7fa35e87c640 1 -- 172.21.15.60:0/4146911207 wait complete. 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.942+0000 7fa35e87c640 1 Processor -- start 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.942+0000 7fa35e87c640 1 -- start start 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.942+0000 7fa35e87c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3580b0bf0 0x7fa3580ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa35e87c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e300 0x7fa3581594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa35e87c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35815e6e0 0x7fa358159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa35e87c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3580ac8e0 con 0x7fa35815e300 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa35e87c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa358141c40 con 0x7fa35815e6e0 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa35e87c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581f3480 con 0x7fa3580b0bf0 2023-11-18T20:44:04.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e300 0x7fa3581594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:04.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e300 0x7fa3581594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54984/0 (socket says 172.21.15.60:54984) 2023-11-18T20:44:04.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa3577fe640 1 -- 172.21.15.60:0/3462098555 learned_addr learned my addr 172.21.15.60:0/3462098555 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:04.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa357fff640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3580b0bf0 0x7fa3580ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:04.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa35ce19640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35815e6e0 0x7fa358159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:04.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa3577fe640 1 -- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3580b0bf0 msgr2=0x7fa3580ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:04.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa3577fe640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3580b0bf0 0x7fa3580ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:04.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa3577fe640 1 -- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35815e6e0 msgr2=0x7fa358159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:04.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa3577fe640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35815e6e0 0x7fa358159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:04.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa3577fe640 1 -- 172.21.15.60:0/3462098555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34c0094d0 con 0x7fa35815e300 2023-11-18T20:44:04.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa357fff640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3580b0bf0 0x7fa3580ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:04.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa35ce19640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35815e6e0 0x7fa358159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:04.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa3577fe640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e300 0x7fa3581594d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa34c02db40 tx=0x7fa34c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:04.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa354ff9640 1 -- 172.21.15.60:0/3462098555 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c037c60 con 0x7fa35815e300 2023-11-18T20:44:04.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa354ff9640 1 -- 172.21.15.60:0/3462098555 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c02fe90 con 0x7fa35815e300 2023-11-18T20:44:04.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa35e87c640 1 -- 172.21.15.60:0/3462098555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3581f3760 con 0x7fa35815e300 2023-11-18T20:44:04.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa354ff9640 1 -- 172.21.15.60:0/3462098555 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c02f860 con 0x7fa35815e300 2023-11-18T20:44:04.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.943+0000 7fa35e87c640 1 -- 172.21.15.60:0/3462098555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3581f3130 con 0x7fa35815e300 2023-11-18T20:44:04.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.945+0000 7fa35e87c640 1 -- 172.21.15.60:0/3462098555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3581f3c80 con 0x7fa35815e300 2023-11-18T20:44:04.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.945+0000 7fa354ff9640 1 -- 172.21.15.60:0/3462098555 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa34c042470 con 0x7fa35815e300 2023-11-18T20:44:04.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.945+0000 7fa354ff9640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa34406ae40 0x7fa34406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:04.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.945+0000 7fa354ff9640 1 -- 172.21.15.60:0/3462098555 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fa34c033070 con 0x7fa35815e300 2023-11-18T20:44:04.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.946+0000 7fa357fff640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa34406ae40 0x7fa34406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:04.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.946+0000 7fa357fff640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa34406ae40 0x7fa34406d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa34800a990 tx=0x7fa348008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:04.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:04.950+0000 7fa354ff9640 1 -- 172.21.15.60:0/3462098555 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3581f3c80 con 0x7fa35815e300 2023-11-18T20:44:05.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.191+0000 7fa35e87c640 1 -- 172.21.15.60:0/3462098555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7fa3581f3c80 con 0x7fa35815e300 2023-11-18T20:44:05.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.193+0000 7fa354ff9640 1 -- 172.21.15.60:0/3462098555 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v15) v1 ==== 211+0+168 (secure 0 0 0) 0x7fa3581f3c80 con 0x7fa35815e300 2023-11-18T20:44:05.194 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-11-18T20:44:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA== 2023-11-18T20:44:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow rwp" 2023-11-18T20:44:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-11-18T20:44:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2023-11-18T20:44:05.195 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-11-18T20:44:05.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 -- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa34406ae40 msgr2=0x7fa34406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa34406ae40 0x7fa34406d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa34800a990 tx=0x7fa348008040 comp rx=0 tx=0).stop 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 -- 172.21.15.60:0/3462098555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e300 msgr2=0x7fa3581594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e300 0x7fa3581594d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa34c02db40 tx=0x7fa34c0046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 -- 172.21.15.60:0/3462098555 shutdown_connections 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa34406ae40 0x7fa34406d300 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35815e6e0 0x7fa358159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35815e300 0x7fa3581594d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 --2- 172.21.15.60:0/3462098555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3580b0bf0 0x7fa3580ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 -- 172.21.15.60:0/3462098555 >> 172.21.15.60:0/3462098555 conn(0x7fa35806d9c0 msgr2=0x7fa3580ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 -- 172.21.15.60:0/3462098555 shutdown_connections 2023-11-18T20:44:05.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.195+0000 7fa35e87c640 1 -- 172.21.15.60:0/3462098555 wait complete. 2023-11-18T20:44:05.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-18T20:44:05.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.312+0000 7f9631275640 1 Processor -- start 2023-11-18T20:44:05.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.312+0000 7f9631275640 1 -- start start 2023-11-18T20:44:05.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f9631275640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f962c150940 0x7f962c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:05.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f9631275640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 0x7f962c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:05.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f9631275640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962c0b49f0 con 0x7f962c150560 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f9631275640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962c0b3560 con 0x7f962c150940 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f9631275640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0b3de0 con 0x7f962c15dec0 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f962a7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f962c150940 0x7f962c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f962a7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f962c150940 0x7f962c15d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54990/0 (socket says 172.21.15.60:54990) 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f962affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f962c150560 0x7f962c0b2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47794/0 (socket says 172.21.15.60:47794) 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f962a7fc640 1 -- 172.21.15.60:0/3338801256 learned_addr learned my addr 172.21.15.60:0/3338801256 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f962b7fe640 1 --2- 172.21.15.60:0/3338801256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 0x7f962c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f962a7fc640 1 -- 172.21.15.60:0/3338801256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 msgr2=0x7f962c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f962a7fc640 1 --2- 172.21.15.60:0/3338801256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 0x7f962c15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f962a7fc640 1 -- 172.21.15.60:0/3338801256 >> v1:172.21.15.60:6789/0 conn(0x7f962c150560 legacy=0x7f962c0b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f962a7fc640 1 -- 172.21.15.60:0/3338801256 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c0b40c0 con 0x7f962c150940 2023-11-18T20:44:05.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.313+0000 7f962b7fe640 1 --2- 172.21.15.60:0/3338801256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 0x7f962c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:05.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.314+0000 7f962a7fc640 1 --2- 172.21.15.60:0/3338801256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c150940 0x7f962c15d200 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f962000bb10 tx=0x7f962002e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28180fd3670e7a10 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:05.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.314+0000 7f9629ffb640 1 -- 172.21.15.60:0/3338801256 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620036440 con 0x7f962c150940 2023-11-18T20:44:05.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.314+0000 7f9629ffb640 1 -- 172.21.15.60:0/3338801256 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96200365a0 con 0x7f962c150940 2023-11-18T20:44:05.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.314+0000 7f9629ffb640 1 -- 172.21.15.60:0/3338801256 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620036930 con 0x7f962c150940 2023-11-18T20:44:05.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.314+0000 7f9631275640 1 -- 172.21.15.60:0/3338801256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c150940 msgr2=0x7f962c15d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:05.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.314+0000 7f9631275640 1 --2- 172.21.15.60:0/3338801256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c150940 0x7f962c15d200 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f962000bb10 tx=0x7f962002e150 comp rx=0 tx=0).stop 2023-11-18T20:44:05.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.315+0000 7f9631275640 1 -- 172.21.15.60:0/3338801256 shutdown_connections 2023-11-18T20:44:05.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.315+0000 7f9631275640 1 --2- 172.21.15.60:0/3338801256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 0x7f962c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.315+0000 7f9631275640 1 --2- 172.21.15.60:0/3338801256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c150940 0x7f962c15d200 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.315+0000 7f9631275640 1 -- 172.21.15.60:0/3338801256 >> 172.21.15.60:0/3338801256 conn(0x7f962c06a4d0 msgr2=0x7f962c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:05.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.315+0000 7f9631275640 1 -- 172.21.15.60:0/3338801256 shutdown_connections 2023-11-18T20:44:05.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.315+0000 7f9631275640 1 -- 172.21.15.60:0/3338801256 wait complete. 2023-11-18T20:44:05.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.316+0000 7f9631275640 1 Processor -- start 2023-11-18T20:44:05.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.316+0000 7f9631275640 1 -- start start 2023-11-18T20:44:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f9631275640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f962c150560 0x7f962c13f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f9631275640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 0x7f962c0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f9631275640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c151f70 0x7f962c0cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f9631275640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c150ed0 con 0x7f962c151f70 2023-11-18T20:44:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f9631275640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c1511b0 con 0x7f962c150560 2023-11-18T20:44:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f9631275640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c151490 con 0x7f962c15dec0 2023-11-18T20:44:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c151f70 0x7f962c0cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c151f70 0x7f962c0cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54998/0 (socket says 172.21.15.60:54998) 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962b7fe640 1 -- 172.21.15.60:0/1016625361 learned_addr learned my addr 172.21.15.60:0/1016625361 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962a7fc640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 0x7f962c0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962affd640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f962c150560 0x7f962c13f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962b7fe640 1 -- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 msgr2=0x7f962c0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962b7fe640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 0x7f962c0ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962b7fe640 1 -- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f962c150560 msgr2=0x7f962c13f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962b7fe640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f962c150560 0x7f962c13f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962b7fe640 1 -- 172.21.15.60:0/1016625361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962000b770 con 0x7f962c151f70 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.317+0000 7f962b7fe640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c151f70 0x7f962c0cd4d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9624007e70 tx=0x7f962400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.318+0000 7f96137fe640 1 -- 172.21.15.60:0/1016625361 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624003b90 con 0x7f962c151f70 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.318+0000 7f96137fe640 1 -- 172.21.15.60:0/1016625361 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962400f040 con 0x7f962c151f70 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.318+0000 7f9631275640 1 -- 172.21.15.60:0/1016625361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962c151770 con 0x7f962c151f70 2023-11-18T20:44:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.318+0000 7f96137fe640 1 -- 172.21.15.60:0/1016625361 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962400de80 con 0x7f962c151f70 2023-11-18T20:44:05.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.318+0000 7f9631275640 1 -- 172.21.15.60:0/1016625361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c152ad0 con 0x7f962c151f70 2023-11-18T20:44:05.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.319+0000 7f9631275640 1 -- 172.21.15.60:0/1016625361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c151c90 con 0x7f962c151f70 2023-11-18T20:44:05.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.319+0000 7f96137fe640 1 -- 172.21.15.60:0/1016625361 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9624004030 con 0x7f962c151f70 2023-11-18T20:44:05.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.319+0000 7f96137fe640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f961806ae70 0x7f961806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:05.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.320+0000 7f96137fe640 1 -- 172.21.15.60:0/1016625361 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f96240899a0 con 0x7f962c151f70 2023-11-18T20:44:05.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.320+0000 7f962affd640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f961806ae70 0x7f961806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:05.323 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.320+0000 7f962affd640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f961806ae70 0x7f961806d330 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f961c000f60 tx=0x7f961c000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.324+0000 7f96137fe640 1 -- 172.21.15.60:0/1016625361 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f962c151c90 con 0x7f962c151f70 2023-11-18T20:44:05.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.535+0000 7f9631275640 1 -- 172.21.15.60:0/1016625361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f962c151c90 con 0x7f962c151f70 2023-11-18T20:44:05.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.536+0000 7f96137fe640 1 -- 172.21.15.60:0/1016625361 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+82 (secure 0 0 0) 0x7f962c151c90 con 0x7f962c151f70 2023-11-18T20:44:05.537 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2023-11-18T20:44:05.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.537+0000 7f9631275640 1 -- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f961806ae70 msgr2=0x7f961806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:05.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.537+0000 7f9631275640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f961806ae70 0x7f961806d330 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f961c000f60 tx=0x7f961c000ed0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.537+0000 7f9631275640 1 -- 172.21.15.60:0/1016625361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c151f70 msgr2=0x7f962c0cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:05.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.537+0000 7f9631275640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c151f70 0x7f962c0cd4d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9624007e70 tx=0x7f962400d4a0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.538+0000 7f9631275640 1 -- 172.21.15.60:0/1016625361 shutdown_connections 2023-11-18T20:44:05.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.538+0000 7f9631275640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f961806ae70 0x7f961806d330 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.538+0000 7f9631275640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f962c151f70 0x7f962c0cd4d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.538+0000 7f9631275640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f962c15dec0 0x7f962c0ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.538+0000 7f9631275640 1 --2- 172.21.15.60:0/1016625361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f962c150560 0x7f962c13f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:05.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.538+0000 7f9631275640 1 -- 172.21.15.60:0/1016625361 >> 172.21.15.60:0/1016625361 conn(0x7f962c06a4d0 msgr2=0x7f962c14f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:05.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.538+0000 7f9631275640 1 -- 172.21.15.60:0/1016625361 shutdown_connections 2023-11-18T20:44:05.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:05.538+0000 7f9631275640 1 -- 172.21.15.60:0/1016625361 wait complete. 2023-11-18T20:44:05.550 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-18T20:44:05.550 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-18T20:44:05.550 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-11-18T20:44:05.550 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-18T20:44:05.550 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:05.550 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-18T20:44:05.550 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-18T20:44:05.550 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-18T20:44:05.550 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:05.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:05.551 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-11-18T20:44:05.566 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:44:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:44:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:44:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-11-18T20:44:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42611sec preferred_lft 42611sec 2023-11-18T20:44:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::d896:cfff:fef3:88dc/64 scope link 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0a:c3:0c:c2:e6:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8c3:cff:fec2:e672/64 scope link 2023-11-18T20:44:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:05.569 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:44:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::40c:deff:fe9d:c1f6/64 scope link 2023-11-18T20:44:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:05.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:05.569 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-18T20:44:05.635 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-11-18T20:44:05.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:05.637 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-11-18T20:44:05.693 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-11-18T20:44:05.693 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-18T20:44:05.693 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-18T20:44:05.693 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-18T20:44:05.693 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-18T20:44:05.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:05.693 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:44:05.693 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:44:05.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-18T20:44:05.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-18T20:44:05.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-18T20:44:05.694 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:44:05.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:05.800 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-11-18T20:44:05.815 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-18T20:44:05.815 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-18T20:44:05.815 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:05.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:05.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-11-18T20:44:05.886 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-18T20:44:05.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:44:05.887 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:05.943 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:05.944 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:44:05.944 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:05.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:44:06.070 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:44:06.127 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:44:06.127 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:44:06.127 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:44:06.127 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:44:06.127 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:44:06.127 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:44:06.127 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:44:06.127 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:44:06.127 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:44:06.127 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:44:06.128 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:06.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:06.130 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:44:06.182 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-18T20:44:06.182 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-18T20:44:06.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:44:06.255 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:44:06.312 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:44:06.312 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:44:06.312 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:44:06.312 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:44:06.312 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:44:06.312 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:44:06.312 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:44:06.312 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:44:06.312 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:44:06.312 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:44:06.313 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:06.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:06.315 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:44:06.320 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-11-18T20:44:06.319+0000 7fa9e4c0d480 -1 init, newargv = 0x55e0688de970 newargc=15 2023-11-18T20:44:06.320 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[103499]: starting ceph client 2023-11-18T20:44:06.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[103499]: starting fuse 2023-11-18T20:44:06.336 INFO:teuthology.orchestra.run.smithi060.stdout:41 2023-11-18T20:44:06.336 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-18T20:44:06.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2023-11-18T20:44:06.336 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-11-18T20:44:06.336 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:06.337 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-11-18T20:44:06.338 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-18T20:44:06.338 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-18T20:44:06.338 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-18T20:44:06.338 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-11-18T20:44:06.338 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-11-18T20:44:06.338 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-11-18T20:44:06.338 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-18T20:44:06.338 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:06.338 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-11-18T20:44:06.338 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-11-18T20:44:06.460 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.103499.asok 2023-11-18T20:44:06.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:44:06.471 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.103499.asok 2023-11-18T20:44:06.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:06.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.103499.asok status 2023-11-18T20:44:06.613 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:44:06.613 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-11-18T20:44:06.613 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe", 2023-11-18T20:44:06.613 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-7418-gb0f45a51 (b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe) reef (dev)", 2023-11-18T20:44:06.613 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "103499", 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5652, 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-11-18T20:44:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 5652 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 884160297 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 884160297 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.5652 192.168.0.1:0/884160297", 2023-11-18T20:44:06.615 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/884160297", 2023-11-18T20:44:06.616 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-11-18T20:44:06.616 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 14, 2023-11-18T20:44:06.616 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 37, 2023-11-18T20:44:06.616 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-11-18T20:44:06.616 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-11-18T20:44:06.616 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-11-18T20:44:06.616 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:44:06.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:06.622 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-11-18T20:44:06.622 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:06.637 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-11-18T20:44:06.637 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:06.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:06.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:06.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:06.708 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-11-18T20:44:06.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:06.766 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-11-18T20:44:06.824 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:06.825 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:06.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:07.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.025+0000 7f81ef9ce640 1 Processor -- start 2023-11-18T20:44:07.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.026+0000 7f81ef9ce640 1 -- start start 2023-11-18T20:44:07.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.026+0000 7f81ef9ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80d2910 0x7f81e80d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.026+0000 7f81ef9ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f81e80d3730 0x7f81e80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.026+0000 7f81ef9ce640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e80b2b10 con 0x7f81e815d330 2023-11-18T20:44:07.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.026+0000 7f81ef9ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e80d3b50 con 0x7f81e80d3730 2023-11-18T20:44:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.026+0000 7f81ef9ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80d1590 con 0x7f81e80d2910 2023-11-18T20:44:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.027+0000 7f81ecf69640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80d2910 0x7f81e80d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.027+0000 7f81ecf69640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80d2910 0x7f81e80d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:40954/0 (socket says 172.21.15.60:40954) 2023-11-18T20:44:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.027+0000 7f81ecf69640 1 -- 172.21.15.60:0/4039903074 learned_addr learned my addr 172.21.15.60:0/4039903074 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.027+0000 7f81edf6b640 1 --2- 172.21.15.60:0/4039903074 >> v2:172.21.15.100:3300/0 conn(0x7f81e80d3730 0x7f81e80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.027+0000 7f81ecf69640 1 -- 172.21.15.60:0/4039903074 >> v2:172.21.15.100:3300/0 conn(0x7f81e80d3730 msgr2=0x7f81e80b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.027+0000 7f81ecf69640 1 --2- 172.21.15.60:0/4039903074 >> v2:172.21.15.100:3300/0 conn(0x7f81e80d3730 0x7f81e80b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.027+0000 7f81ecf69640 1 -- 172.21.15.60:0/4039903074 >> v1:172.21.15.100:6789/0 conn(0x7f81e815d330 legacy=0x7f81e80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.027+0000 7f81ecf69640 1 -- 172.21.15.60:0/4039903074 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e80d1870 con 0x7f81e80d2910 2023-11-18T20:44:07.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.028+0000 7f81ecf69640 1 --2- 172.21.15.60:0/4039903074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80d2910 0x7f81e80d31f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009870 tx=0x7f81d802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d609ad46527a95eb server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.028+0000 7f81dffff640 1 -- 172.21.15.60:0/4039903074 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8031640 con 0x7f81e80d2910 2023-11-18T20:44:07.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.028+0000 7f81dffff640 1 -- 172.21.15.60:0/4039903074 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d80317a0 con 0x7f81e80d2910 2023-11-18T20:44:07.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.028+0000 7f81dffff640 1 -- 172.21.15.60:0/4039903074 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8031b30 con 0x7f81e80d2910 2023-11-18T20:44:07.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.029+0000 7f81ef9ce640 1 -- 172.21.15.60:0/4039903074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80d2910 msgr2=0x7f81e80d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.029+0000 7f81ef9ce640 1 --2- 172.21.15.60:0/4039903074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80d2910 0x7f81e80d31f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009870 tx=0x7f81d802b8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.029+0000 7f81ef9ce640 1 -- 172.21.15.60:0/4039903074 shutdown_connections 2023-11-18T20:44:07.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.029+0000 7f81ef9ce640 1 --2- 172.21.15.60:0/4039903074 >> v2:172.21.15.100:3300/0 conn(0x7f81e80d3730 0x7f81e80b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.029+0000 7f81ef9ce640 1 --2- 172.21.15.60:0/4039903074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80d2910 0x7f81e80d31f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.029+0000 7f81ef9ce640 1 -- 172.21.15.60:0/4039903074 >> 172.21.15.60:0/4039903074 conn(0x7f81e8072020 msgr2=0x7f81e8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:07.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.029+0000 7f81ef9ce640 1 -- 172.21.15.60:0/4039903074 shutdown_connections 2023-11-18T20:44:07.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.029+0000 7f81ef9ce640 1 -- 172.21.15.60:0/4039903074 wait complete. 2023-11-18T20:44:07.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.030+0000 7f81ef9ce640 1 Processor -- start 2023-11-18T20:44:07.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.030+0000 7f81ef9ce640 1 -- start start 2023-11-18T20:44:07.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ef9ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80d2910 0x7f81e8150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ef9ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e80d3730 0x7f81e80b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ef9ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e815d330 0x7f81e80c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ef9ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e815dd40 con 0x7f81e80d2910 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ef9ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e815d710 con 0x7f81e80d3730 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ef9ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81590d0 con 0x7f81e815d330 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ed76a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80d2910 0x7f81e8150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ed76a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80d2910 0x7f81e8150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55002/0 (socket says 172.21.15.60:55002) 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ed76a640 1 -- 172.21.15.60:0/1764442545 learned_addr learned my addr 172.21.15.60:0/1764442545 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ecf69640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e80d3730 0x7f81e80b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81edf6b640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e815d330 0x7f81e80c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ed76a640 1 -- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e815d330 msgr2=0x7f81e80c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ed76a640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e815d330 0x7f81e80c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ed76a640 1 -- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e80d3730 msgr2=0x7f81e80b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ed76a640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e80d3730 0x7f81e80b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ed76a640 1 -- 172.21.15.60:0/1764442545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d80094d0 con 0x7f81e80d2910 2023-11-18T20:44:07.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ed76a640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80d2910 0x7f81e8150eb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f81e400c450 tx=0x7f81e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ddffb640 1 -- 172.21.15.60:0/1764442545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e40052d0 con 0x7f81e80d2910 2023-11-18T20:44:07.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ddffb640 1 -- 172.21.15.60:0/1764442545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e4005430 con 0x7f81e80d2910 2023-11-18T20:44:07.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ef9ce640 1 -- 172.21.15.60:0/1764442545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e81593b0 con 0x7f81e80d2910 2023-11-18T20:44:07.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.031+0000 7f81ddffb640 1 -- 172.21.15.60:0/1764442545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4003830 con 0x7f81e80d2910 2023-11-18T20:44:07.035 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.032+0000 7f81ef9ce640 1 -- 172.21.15.60:0/1764442545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e80c8130 con 0x7f81e80d2910 2023-11-18T20:44:07.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.033+0000 7f81ef9ce640 1 -- 172.21.15.60:0/1764442545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e8069400 con 0x7f81e80d2910 2023-11-18T20:44:07.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.033+0000 7f81ddffb640 1 -- 172.21.15.60:0/1764442545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81e4003990 con 0x7f81e80d2910 2023-11-18T20:44:07.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.033+0000 7f81ddffb640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f81cc06ae40 0x7f81cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.034+0000 7f81ddffb640 1 -- 172.21.15.60:0/1764442545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f81e4088c50 con 0x7f81e80d2910 2023-11-18T20:44:07.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.034+0000 7f81ecf69640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f81cc06ae40 0x7f81cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.034+0000 7f81ecf69640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f81cc06ae40 0x7f81cc06d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f81d8036000 tx=0x7f81d803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.038+0000 7f81ddffb640 1 -- 172.21.15.60:0/1764442545 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f81e408e020 con 0x7f81e80d2910 2023-11-18T20:44:07.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.213+0000 7f81ef9ce640 1 -- 172.21.15.60:0/1764442545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81e80b8e90 con 0x7f81e80d2910 2023-11-18T20:44:07.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.214+0000 7f81ddffb640 1 -- 172.21.15.60:0/1764442545 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18336 (secure 0 0 0) 0x7f81e4053cb0 con 0x7f81e80d2910 2023-11-18T20:44:07.216 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:07.216 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:42.814654+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-11-18T20:43:40.650315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:07.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 -- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f81cc06ae40 msgr2=0x7f81cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f81cc06ae40 0x7f81cc06d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f81d8036000 tx=0x7f81d803c040 comp rx=0 tx=0).stop 2023-11-18T20:44:07.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 -- 172.21.15.60:0/1764442545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80d2910 msgr2=0x7f81e8150eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80d2910 0x7f81e8150eb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f81e400c450 tx=0x7f81e400c910 comp rx=0 tx=0).stop 2023-11-18T20:44:07.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 -- 172.21.15.60:0/1764442545 shutdown_connections 2023-11-18T20:44:07.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f81cc06ae40 0x7f81cc06d300 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e815d330 0x7f81e80c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e80d3730 0x7f81e80b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 --2- 172.21.15.60:0/1764442545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80d2910 0x7f81e8150eb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 -- 172.21.15.60:0/1764442545 >> 172.21.15.60:0/1764442545 conn(0x7f81e8072020 msgr2=0x7f81e80b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:07.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 -- 172.21.15.60:0/1764442545 shutdown_connections 2023-11-18T20:44:07.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.216+0000 7f81ef9ce640 1 -- 172.21.15.60:0/1764442545 wait complete. 2023-11-18T20:44:07.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:43:31.703619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:43:32.121610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-11-18T20:43:40.650315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:44:07.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:44:07.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.337+0000 7fd356bb0640 1 Processor -- start 2023-11-18T20:44:07.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.337+0000 7fd356bb0640 1 -- start start 2023-11-18T20:44:07.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.338+0000 7fd356bb0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3500b1400 0x7fd35015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.338+0000 7fd356bb0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd35015e730 0x7fd35015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.338+0000 7fd356bb0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35015e2a0 con 0x7fd350143490 2023-11-18T20:44:07.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.338+0000 7fd356bb0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35015b520 con 0x7fd35015e730 2023-11-18T20:44:07.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.338+0000 7fd356bb0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd350143870 con 0x7fd3500b1400 2023-11-18T20:44:07.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.338+0000 7fd34ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3500b1400 0x7fd35015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.338+0000 7fd35514d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd35015e730 0x7fd35015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.338+0000 7fd35494c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd350143490 0x7fd3500b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:48278/0 (socket says 172.21.15.60:48278) 2023-11-18T20:44:07.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.338+0000 7fd35494c640 1 -- 172.21.15.60:0/2797921597 learned_addr learned my addr 172.21.15.60:0/2797921597 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:07.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.338+0000 7fd34ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3500b1400 0x7fd35015a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:40966/0 (socket says 172.21.15.60:40966) 2023-11-18T20:44:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.339+0000 7fd34f7fe640 1 -- 172.21.15.60:0/2797921597 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181210449 0 0) 0x7fd35015e2a0 con 0x7fd350143490 2023-11-18T20:44:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.339+0000 7fd34f7fe640 1 -- 172.21.15.60:0/2797921597 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd338003520 con 0x7fd350143490 2023-11-18T20:44:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.339+0000 7fd34ffff640 1 -- 172.21.15.60:0/2797921597 >> v2:172.21.15.100:3300/0 conn(0x7fd35015e730 msgr2=0x7fd35015aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.339+0000 7fd34ffff640 1 --2- 172.21.15.60:0/2797921597 >> v2:172.21.15.100:3300/0 conn(0x7fd35015e730 0x7fd35015aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.339+0000 7fd34ffff640 1 -- 172.21.15.60:0/2797921597 >> v1:172.21.15.100:6789/0 conn(0x7fd350143490 legacy=0x7fd3500b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.339+0000 7fd34ffff640 1 -- 172.21.15.60:0/2797921597 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35015b240 con 0x7fd3500b1400 2023-11-18T20:44:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.339+0000 7fd35514d640 1 --2- 172.21.15.60:0/2797921597 >> v2:172.21.15.100:3300/0 conn(0x7fd35015e730 0x7fd35015aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.339+0000 7fd34ffff640 1 --2- 172.21.15.60:0/2797921597 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3500b1400 0x7fd35015a4e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd3440097a0 tx=0x7fd34402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b82ca25b93fe8b0e server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd34f7fe640 1 -- 172.21.15.60:0/2797921597 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34402d040 con 0x7fd3500b1400 2023-11-18T20:44:07.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd34f7fe640 1 -- 172.21.15.60:0/2797921597 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd344031810 con 0x7fd3500b1400 2023-11-18T20:44:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd34f7fe640 1 -- 172.21.15.60:0/2797921597 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344031ba0 con 0x7fd3500b1400 2023-11-18T20:44:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd356bb0640 1 -- 172.21.15.60:0/2797921597 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3500b1400 msgr2=0x7fd35015a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd356bb0640 1 --2- 172.21.15.60:0/2797921597 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3500b1400 0x7fd35015a4e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd3440097a0 tx=0x7fd34402b8b0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd356bb0640 1 -- 172.21.15.60:0/2797921597 shutdown_connections 2023-11-18T20:44:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd356bb0640 1 --2- 172.21.15.60:0/2797921597 >> v2:172.21.15.100:3300/0 conn(0x7fd35015e730 0x7fd35015aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd356bb0640 1 --2- 172.21.15.60:0/2797921597 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3500b1400 0x7fd35015a4e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd356bb0640 1 -- 172.21.15.60:0/2797921597 >> 172.21.15.60:0/2797921597 conn(0x7fd35006a4d0 msgr2=0x7fd35006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd356bb0640 1 -- 172.21.15.60:0/2797921597 shutdown_connections 2023-11-18T20:44:07.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.340+0000 7fd356bb0640 1 -- 172.21.15.60:0/2797921597 wait complete. 2023-11-18T20:44:07.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.341+0000 7fd356bb0640 1 Processor -- start 2023-11-18T20:44:07.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.341+0000 7fd356bb0640 1 -- start start 2023-11-18T20:44:07.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.342+0000 7fd356bb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3500b1400 0x7fd3500ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.342+0000 7fd356bb0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd350143490 0x7fd3501e4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.342+0000 7fd356bb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35015e730 0x7fd3500b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.342+0000 7fd356bb0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500ac780 con 0x7fd35015e730 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.342+0000 7fd356bb0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3501e4840 con 0x7fd3500b1400 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.342+0000 7fd356bb0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500abb00 con 0x7fd350143490 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.342+0000 7fd35514d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35015e730 0x7fd3500b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.342+0000 7fd35514d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35015e730 0x7fd3500b01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55018/0 (socket says 172.21.15.60:55018) 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.342+0000 7fd35514d640 1 -- 172.21.15.60:0/3196292452 learned_addr learned my addr 172.21.15.60:0/3196292452 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd34ffff640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd350143490 0x7fd3501e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd35494c640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3500b1400 0x7fd3500ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd35514d640 1 -- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd350143490 msgr2=0x7fd3501e4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd35514d640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd350143490 0x7fd3501e4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd35514d640 1 -- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3500b1400 msgr2=0x7fd3500ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd35514d640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3500b1400 0x7fd3500ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd35514d640 1 -- 172.21.15.60:0/3196292452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348009a70 con 0x7fd35015e730 2023-11-18T20:44:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd35514d640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35015e730 0x7fd3500b01e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd34800efc0 tx=0x7fd34800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd34d7fa640 1 -- 172.21.15.60:0/3196292452 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348009cd0 con 0x7fd35015e730 2023-11-18T20:44:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd34d7fa640 1 -- 172.21.15.60:0/3196292452 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34800f040 con 0x7fd35015e730 2023-11-18T20:44:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd356bb0640 1 -- 172.21.15.60:0/3196292452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd344009400 con 0x7fd35015e730 2023-11-18T20:44:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd34d7fa640 1 -- 172.21.15.60:0/3196292452 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348003990 con 0x7fd35015e730 2023-11-18T20:44:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.343+0000 7fd356bb0640 1 -- 172.21.15.60:0/3196292452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3500ac0c0 con 0x7fd35015e730 2023-11-18T20:44:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.344+0000 7fd356bb0640 1 -- 172.21.15.60:0/3196292452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3500ac8c0 con 0x7fd35015e730 2023-11-18T20:44:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.345+0000 7fd34d7fa640 1 -- 172.21.15.60:0/3196292452 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd348014480 con 0x7fd35015e730 2023-11-18T20:44:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.345+0000 7fd34d7fa640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd33c06ae40 0x7fd33c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.345+0000 7fd34d7fa640 1 -- 172.21.15.60:0/3196292452 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd34808a1f0 con 0x7fd35015e730 2023-11-18T20:44:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.345+0000 7fd35494c640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd33c06ae40 0x7fd33c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.346+0000 7fd35494c640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd33c06ae40 0x7fd33c06d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd3400011e0 tx=0x7fd340007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.349+0000 7fd34d7fa640 1 -- 172.21.15.60:0/3196292452 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd348055250 con 0x7fd35015e730 2023-11-18T20:44:07.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.524+0000 7fd356bb0640 1 -- 172.21.15.60:0/3196292452 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd350143c20 con 0x7fd35015e730 2023-11-18T20:44:07.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.524+0000 7fd34d7fa640 1 -- 172.21.15.60:0/3196292452 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd348058870 con 0x7fd35015e730 2023-11-18T20:44:07.525 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 -- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd33c06ae40 msgr2=0x7fd33c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd33c06ae40 0x7fd33c06d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd3400011e0 tx=0x7fd340007d30 comp rx=0 tx=0).stop 2023-11-18T20:44:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 -- 172.21.15.60:0/3196292452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35015e730 msgr2=0x7fd3500b01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35015e730 0x7fd3500b01e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd34800efc0 tx=0x7fd34800c490 comp rx=0 tx=0).stop 2023-11-18T20:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 -- 172.21.15.60:0/3196292452 shutdown_connections 2023-11-18T20:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd33c06ae40 0x7fd33c06d300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35015e730 0x7fd3500b01e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd350143490 0x7fd3501e4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 --2- 172.21.15.60:0/3196292452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3500b1400 0x7fd3500ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 -- 172.21.15.60:0/3196292452 >> 172.21.15.60:0/3196292452 conn(0x7fd35006a4d0 msgr2=0x7fd35006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 -- 172.21.15.60:0/3196292452 shutdown_connections 2023-11-18T20:44:07.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.526+0000 7fd356bb0640 1 -- 172.21.15.60:0/3196292452 wait complete. 2023-11-18T20:44:07.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:44:07.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.646+0000 7f76e1ff5640 1 Processor -- start 2023-11-18T20:44:07.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.646+0000 7f76e1ff5640 1 -- start start 2023-11-18T20:44:07.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.646+0000 7f76e1ff5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f76dc13cf70 0x7f76dc071c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.646+0000 7f76e1ff5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc15e560 0x7f76dc0cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.646+0000 7f76e1ff5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76dc15a570 con 0x7f76dc0721c0 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.646+0000 7f76e1ff5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc072670 con 0x7f76dc13cf70 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.646+0000 7f76e1ff5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc15b570 con 0x7f76dc15e560 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76daffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f76dc0721c0 0x7f76dc15de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47806/0 (socket says 172.21.15.60:47806) 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76daffd640 1 -- 172.21.15.60:0/1724101678 learned_addr learned my addr 172.21.15.60:0/1724101678 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76dbfff640 1 --2- 172.21.15.60:0/1724101678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc15e560 0x7f76dc0cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76db7fe640 1 --2- 172.21.15.60:0/1724101678 >> v2:172.21.15.100:3300/0 conn(0x7f76dc13cf70 0x7f76dc071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76da7fc640 1 -- 172.21.15.60:0/1724101678 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675737084 0 0) 0x7f76c4001110 con 0x7f76dc0721c0 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76da7fc640 1 -- 172.21.15.60:0/1724101678 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c8003520 con 0x7f76dc0721c0 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76dbfff640 1 -- 172.21.15.60:0/1724101678 >> v2:172.21.15.100:3300/0 conn(0x7f76dc13cf70 msgr2=0x7f76dc071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:07.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76dbfff640 1 --2- 172.21.15.60:0/1724101678 >> v2:172.21.15.100:3300/0 conn(0x7f76dc13cf70 0x7f76dc071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76da7fc640 1 -- 172.21.15.60:0/1724101678 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1304857815 0 0) 0x7f76c8003520 con 0x7f76dc0721c0 2023-11-18T20:44:07.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76dbfff640 1 -- 172.21.15.60:0/1724101678 >> v1:172.21.15.60:6789/0 conn(0x7f76dc0721c0 legacy=0x7f76dc15de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.647+0000 7f76dbfff640 1 -- 172.21.15.60:0/1724101678 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc15a7b0 con 0x7f76dc15e560 2023-11-18T20:44:07.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.648+0000 7f76dbfff640 1 --2- 172.21.15.60:0/1724101678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc15e560 0x7f76dc0cd9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009d90 tx=0x7f76d402bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8230dd56a3658f97 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.648+0000 7f76da7fc640 1 -- 172.21.15.60:0/1724101678 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4001800 con 0x7f76dc15e560 2023-11-18T20:44:07.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.648+0000 7f76da7fc640 1 -- 172.21.15.60:0/1724101678 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d402c8a0 con 0x7f76dc15e560 2023-11-18T20:44:07.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.648+0000 7f76da7fc640 1 -- 172.21.15.60:0/1724101678 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d402cc30 con 0x7f76dc15e560 2023-11-18T20:44:07.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.649+0000 7f76e1ff5640 1 -- 172.21.15.60:0/1724101678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc15e560 msgr2=0x7f76dc0cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.649+0000 7f76e1ff5640 1 --2- 172.21.15.60:0/1724101678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc15e560 0x7f76dc0cd9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009d90 tx=0x7f76d402bd70 comp rx=0 tx=0).stop 2023-11-18T20:44:07.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.649+0000 7f76e1ff5640 1 -- 172.21.15.60:0/1724101678 shutdown_connections 2023-11-18T20:44:07.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.649+0000 7f76e1ff5640 1 --2- 172.21.15.60:0/1724101678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc15e560 0x7f76dc0cd9f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.649+0000 7f76e1ff5640 1 --2- 172.21.15.60:0/1724101678 >> v2:172.21.15.100:3300/0 conn(0x7f76dc13cf70 0x7f76dc071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.649+0000 7f76e1ff5640 1 -- 172.21.15.60:0/1724101678 >> 172.21.15.60:0/1724101678 conn(0x7f76dc06c520 msgr2=0x7f76dc06c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:07.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.649+0000 7f76e1ff5640 1 -- 172.21.15.60:0/1724101678 shutdown_connections 2023-11-18T20:44:07.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.649+0000 7f76e1ff5640 1 -- 172.21.15.60:0/1724101678 wait complete. 2023-11-18T20:44:07.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.650+0000 7f76e1ff5640 1 Processor -- start 2023-11-18T20:44:07.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.650+0000 7f76e1ff5640 1 -- start start 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76e1ff5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76dc0721c0 0x7f76dc1f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76e1ff5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc13cf70 0x7f76dc166970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76e1ff5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76dc15e560 0x7f76dc15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76e1ff5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc1536d0 con 0x7f76dc15e560 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76e1ff5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc153b60 con 0x7f76dc0721c0 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76e1ff5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc1f75a0 con 0x7f76dc13cf70 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76dbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76dc15e560 0x7f76dc15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76dbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76dc15e560 0x7f76dc15ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55030/0 (socket says 172.21.15.60:55030) 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76dbfff640 1 -- 172.21.15.60:0/2987099904 learned_addr learned my addr 172.21.15.60:0/2987099904 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76db7fe640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76dc0721c0 0x7f76dc1f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76daffd640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc13cf70 0x7f76dc166970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76dbfff640 1 -- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc13cf70 msgr2=0x7f76dc166970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76dbfff640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc13cf70 0x7f76dc166970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76dbfff640 1 -- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76dc0721c0 msgr2=0x7f76dc1f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76dbfff640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76dc0721c0 0x7f76dc1f7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76dbfff640 1 -- 172.21.15.60:0/2987099904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4009a70 con 0x7f76dc15e560 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76db7fe640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76dc0721c0 0x7f76dc1f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.651+0000 7f76daffd640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc13cf70 0x7f76dc166970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.652+0000 7f76dbfff640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76dc15e560 0x7f76dc15ee40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009eb0 tx=0x7f76d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.652+0000 7f76d3fff640 1 -- 172.21.15.60:0/2987099904 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4034e50 con 0x7f76dc15e560 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.652+0000 7f76d3fff640 1 -- 172.21.15.60:0/2987099904 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d402ce90 con 0x7f76dc15e560 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.652+0000 7f76e1ff5640 1 -- 172.21.15.60:0/2987099904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc152ac0 con 0x7f76dc15e560 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.652+0000 7f76d3fff640 1 -- 172.21.15.60:0/2987099904 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4040690 con 0x7f76dc15e560 2023-11-18T20:44:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.652+0000 7f76e1ff5640 1 -- 172.21.15.60:0/2987099904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc153080 con 0x7f76dc15e560 2023-11-18T20:44:07.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.653+0000 7f76e1ff5640 1 -- 172.21.15.60:0/2987099904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc153880 con 0x7f76dc15e560 2023-11-18T20:44:07.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.653+0000 7f76d3fff640 1 -- 172.21.15.60:0/2987099904 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76d40407f0 con 0x7f76dc15e560 2023-11-18T20:44:07.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.653+0000 7f76d3fff640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76bc06ae40 0x7f76bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.654+0000 7f76d3fff640 1 -- 172.21.15.60:0/2987099904 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f76d40ac4c0 con 0x7f76dc15e560 2023-11-18T20:44:07.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.654+0000 7f76db7fe640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76bc06ae40 0x7f76bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.654+0000 7f76db7fe640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76bc06ae40 0x7f76bc06d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f76dc15edf0 tx=0x7f76cc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.659 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.658+0000 7f76d3fff640 1 -- 172.21.15.60:0/2987099904 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76dc153880 con 0x7f76dc15e560 2023-11-18T20:44:07.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.837+0000 7f76e1ff5640 1 -- 172.21.15.60:0/2987099904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76dc153880 con 0x7f76dc15e560 2023-11-18T20:44:07.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.837+0000 7f76d3fff640 1 -- 172.21.15.60:0/2987099904 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76dc153880 con 0x7f76dc15e560 2023-11-18T20:44:07.839 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 -- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76bc06ae40 msgr2=0x7f76bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76bc06ae40 0x7f76bc06d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f76dc15edf0 tx=0x7f76cc008040 comp rx=0 tx=0).stop 2023-11-18T20:44:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 -- 172.21.15.60:0/2987099904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76dc15e560 msgr2=0x7f76dc15ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76dc15e560 0x7f76dc15ee40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009eb0 tx=0x7f76d4004a40 comp rx=0 tx=0).stop 2023-11-18T20:44:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 -- 172.21.15.60:0/2987099904 shutdown_connections 2023-11-18T20:44:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76bc06ae40 0x7f76bc06d300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76dc15e560 0x7f76dc15ee40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76dc13cf70 0x7f76dc166970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 --2- 172.21.15.60:0/2987099904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76dc0721c0 0x7f76dc1f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 -- 172.21.15.60:0/2987099904 >> 172.21.15.60:0/2987099904 conn(0x7f76dc06c520 msgr2=0x7f76dc154c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:07.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 -- 172.21.15.60:0/2987099904 shutdown_connections 2023-11-18T20:44:07.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.839+0000 7f76e1ff5640 1 -- 172.21.15.60:0/2987099904 wait complete. 2023-11-18T20:44:07.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:44:07.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.960+0000 7fd4c0c83640 1 Processor -- start 2023-11-18T20:44:07.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.960+0000 7fd4c0c83640 1 -- start start 2023-11-18T20:44:07.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4c0c83640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd4bc143490 0x7fd4bc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4c0c83640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc15e730 0x7fd4bc0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4c0c83640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4bc15e2a0 con 0x7fd4bc15a4e0 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4c0c83640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4bc15b520 con 0x7fd4bc143490 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4c0c83640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc143870 con 0x7fd4bc15e730 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4bad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd4bc15a4e0 0x7fd4bc15a970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47820/0 (socket says 172.21.15.60:47820) 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4bad9d640 1 -- 172.21.15.60:0/3281314073 learned_addr learned my addr 172.21.15.60:0/3281314073 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4ba59c640 1 --2- 172.21.15.60:0/3281314073 >> v2:172.21.15.100:3300/0 conn(0x7fd4bc143490 0x7fd4bc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4b1d9b640 1 --2- 172.21.15.60:0/3281314073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc15e730 0x7fd4bc0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4b9d9b640 1 -- 172.21.15.60:0/3281314073 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785330106 0 0) 0x7fd4bc15e2a0 con 0x7fd4bc15a4e0 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4b9d9b640 1 -- 172.21.15.60:0/3281314073 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4a8003520 con 0x7fd4bc15a4e0 2023-11-18T20:44:07.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4ba59c640 1 -- 172.21.15.60:0/3281314073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc15e730 msgr2=0x7fd4bc0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:07.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4ba59c640 1 --2- 172.21.15.60:0/3281314073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc15e730 0x7fd4bc0b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4ba59c640 1 -- 172.21.15.60:0/3281314073 >> v1:172.21.15.60:6789/0 conn(0x7fd4bc15a4e0 legacy=0x7fd4bc15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4ba59c640 1 -- 172.21.15.60:0/3281314073 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc15b240 con 0x7fd4bc143490 2023-11-18T20:44:07.964 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.961+0000 7fd4b1d9b640 1 --2- 172.21.15.60:0/3281314073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc15e730 0x7fd4bc0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:07.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.962+0000 7fd4ba59c640 1 --2- 172.21.15.60:0/3281314073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4bc143490 0x7fd4bc0b0cf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0097a0 tx=0x7fd4ac02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e122adbbd2fba90 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.962+0000 7fd4b9d9b640 1 -- 172.21.15.60:0/3281314073 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac031830 con 0x7fd4bc143490 2023-11-18T20:44:07.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.962+0000 7fd4b9d9b640 1 -- 172.21.15.60:0/3281314073 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ac031990 con 0x7fd4bc143490 2023-11-18T20:44:07.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.962+0000 7fd4b9d9b640 1 -- 172.21.15.60:0/3281314073 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac031d20 con 0x7fd4bc143490 2023-11-18T20:44:07.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.963+0000 7fd4c0c83640 1 -- 172.21.15.60:0/3281314073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4bc143490 msgr2=0x7fd4bc0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.963+0000 7fd4c0c83640 1 --2- 172.21.15.60:0/3281314073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4bc143490 0x7fd4bc0b0cf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0097a0 tx=0x7fd4ac02b8d0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.963+0000 7fd4c0c83640 1 -- 172.21.15.60:0/3281314073 shutdown_connections 2023-11-18T20:44:07.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.963+0000 7fd4c0c83640 1 --2- 172.21.15.60:0/3281314073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc15e730 0x7fd4bc0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.963+0000 7fd4c0c83640 1 --2- 172.21.15.60:0/3281314073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4bc143490 0x7fd4bc0b0cf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.963+0000 7fd4c0c83640 1 -- 172.21.15.60:0/3281314073 >> 172.21.15.60:0/3281314073 conn(0x7fd4bc06a4d0 msgr2=0x7fd4bc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:07.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.963+0000 7fd4c0c83640 1 -- 172.21.15.60:0/3281314073 shutdown_connections 2023-11-18T20:44:07.966 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.963+0000 7fd4c0c83640 1 -- 172.21.15.60:0/3281314073 wait complete. 2023-11-18T20:44:07.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.964+0000 7fd4c0c83640 1 Processor -- start 2023-11-18T20:44:07.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.965+0000 7fd4c0c83640 1 -- start start 2023-11-18T20:44:07.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.965+0000 7fd4c0c83640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc143490 0x7fd4bc1598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.965+0000 7fd4c0c83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4bc15a4e0 0x7fd4bc1e40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.965+0000 7fd4c0c83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4bc15e730 0x7fd4bc1e4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.965+0000 7fd4c0c83640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc1e7600 con 0x7fd4bc15e730 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.965+0000 7fd4c0c83640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc1e6980 con 0x7fd4bc15a4e0 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.965+0000 7fd4c0c83640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc1e6c60 con 0x7fd4bc143490 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.965+0000 7fd4bad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4bc15e730 0x7fd4bc1e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.965+0000 7fd4bad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4bc15e730 0x7fd4bc1e4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55034/0 (socket says 172.21.15.60:55034) 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.965+0000 7fd4bad9d640 1 -- 172.21.15.60:0/2640561289 learned_addr learned my addr 172.21.15.60:0/2640561289 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4b1d9b640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4bc15a4e0 0x7fd4bc1e40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4ba59c640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc143490 0x7fd4bc1598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4bad9d640 1 -- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc143490 msgr2=0x7fd4bc1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4bad9d640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc143490 0x7fd4bc1598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4bad9d640 1 -- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4bc15a4e0 msgr2=0x7fd4bc1e40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:07.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4bad9d640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4bc15a4e0 0x7fd4bc1e40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:07.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4bad9d640 1 -- 172.21.15.60:0/2640561289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a4009400 con 0x7fd4bc15e730 2023-11-18T20:44:07.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4ba59c640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc143490 0x7fd4bc1598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:07.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4bad9d640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4bc15e730 0x7fd4bc1e4600 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400cb40 tx=0x7fd4b400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4b37fe640 1 -- 172.21.15.60:0/2640561289 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b400aca0 con 0x7fd4bc15e730 2023-11-18T20:44:07.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4b37fe640 1 -- 172.21.15.60:0/2640561289 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b400ae00 con 0x7fd4bc15e730 2023-11-18T20:44:07.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4c0c83640 1 -- 172.21.15.60:0/2640561289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac009400 con 0x7fd4bc15e730 2023-11-18T20:44:07.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4b37fe640 1 -- 172.21.15.60:0/2640561289 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4012620 con 0x7fd4bc15e730 2023-11-18T20:44:07.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.966+0000 7fd4c0c83640 1 -- 172.21.15.60:0/2640561289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4bc1e7220 con 0x7fd4bc15e730 2023-11-18T20:44:07.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.967+0000 7fd4c0c83640 1 -- 172.21.15.60:0/2640561289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4bc1e7740 con 0x7fd4bc15e730 2023-11-18T20:44:07.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.968+0000 7fd4b37fe640 1 -- 172.21.15.60:0/2640561289 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4b4007070 con 0x7fd4bc15e730 2023-11-18T20:44:07.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.968+0000 7fd4b37fe640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd49c06ae40 0x7fd49c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:07.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.968+0000 7fd4b37fe640 1 -- 172.21.15.60:0/2640561289 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd4b4089840 con 0x7fd4bc15e730 2023-11-18T20:44:07.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.968+0000 7fd4ba59c640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd49c06ae40 0x7fd49c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:07.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.969+0000 7fd4ba59c640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd49c06ae40 0x7fd49c06d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac036000 tx=0x7fd4ac008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:07.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:07.972+0000 7fd4b37fe640 1 -- 172.21.15.60:0/2640561289 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4bc1e7740 con 0x7fd4bc15e730 2023-11-18T20:44:08.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.148+0000 7fd4c0c83640 1 -- 172.21.15.60:0/2640561289 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4bc1e7740 con 0x7fd4bc15e730 2023-11-18T20:44:08.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.148+0000 7fd4b37fe640 1 -- 172.21.15.60:0/2640561289 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4b4017020 con 0x7fd4bc15e730 2023-11-18T20:44:08.149 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:44:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 -- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd49c06ae40 msgr2=0x7fd49c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd49c06ae40 0x7fd49c06d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac036000 tx=0x7fd4ac008030 comp rx=0 tx=0).stop 2023-11-18T20:44:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 -- 172.21.15.60:0/2640561289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4bc15e730 msgr2=0x7fd4bc1e4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4bc15e730 0x7fd4bc1e4600 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400cb40 tx=0x7fd4b400a060 comp rx=0 tx=0).stop 2023-11-18T20:44:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 -- 172.21.15.60:0/2640561289 shutdown_connections 2023-11-18T20:44:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd49c06ae40 0x7fd49c06d300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4bc15e730 0x7fd4bc1e4600 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4bc15a4e0 0x7fd4bc1e40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 --2- 172.21.15.60:0/2640561289 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4bc143490 0x7fd4bc1598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 -- 172.21.15.60:0/2640561289 >> 172.21.15.60:0/2640561289 conn(0x7fd4bc06a4d0 msgr2=0x7fd4bc15a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 -- 172.21.15.60:0/2640561289 shutdown_connections 2023-11-18T20:44:08.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.150+0000 7fd4c0c83640 1 -- 172.21.15.60:0/2640561289 wait complete. 2023-11-18T20:44:08.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-11-18T20:44:08.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.270+0000 7f629f3d5640 1 Processor -- start 2023-11-18T20:44:08.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.271+0000 7f629f3d5640 1 -- start start 2023-11-18T20:44:08.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.271+0000 7f629f3d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6298140310 0x7f62981618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.271+0000 7f629f3d5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62981409d0 0x7f629806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.271+0000 7f629f3d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62981428d0 0x7f629806aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.271+0000 7f629f3d5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6298161e20 con 0x7f62981409d0 2023-11-18T20:44:08.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.271+0000 7f629f3d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62981406f0 con 0x7f6298140310 2023-11-18T20:44:08.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.271+0000 7f629f3d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6298143960 con 0x7f62981428d0 2023-11-18T20:44:08.274 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.272+0000 7f629c970640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62981409d0 0x7f629806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.272+0000 7f629c970640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f62981409d0 0x7f629806a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55038/0 (socket says 172.21.15.60:55038) 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.272+0000 7f629c970640 1 -- 172.21.15.60:0/3198668841 learned_addr learned my addr 172.21.15.60:0/3198668841 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.272+0000 7f629c970640 1 -- 172.21.15.60:0/3198668841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62981428d0 msgr2=0x7f629806aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.272+0000 7f629c970640 1 --2- 172.21.15.60:0/3198668841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62981428d0 0x7f629806aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.272+0000 7f629d171640 1 --2- 172.21.15.60:0/3198668841 >> v2:172.21.15.100:3300/0 conn(0x7f6298140310 0x7f62981618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.272+0000 7f629c970640 1 -- 172.21.15.60:0/3198668841 >> v2:172.21.15.100:3300/0 conn(0x7f6298140310 msgr2=0x7f62981618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.272+0000 7f629c970640 1 --2- 172.21.15.60:0/3198668841 >> v2:172.21.15.100:3300/0 conn(0x7f6298140310 0x7f62981618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.272+0000 7f629c970640 1 -- 172.21.15.60:0/3198668841 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6298142de0 con 0x7f62981409d0 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.272+0000 7f629d171640 1 --2- 172.21.15.60:0/3198668841 >> v2:172.21.15.100:3300/0 conn(0x7f6298140310 0x7f62981618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f629c970640 1 --2- 172.21.15.60:0/3198668841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62981409d0 0x7f629806a4d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f62980b5840 tx=0x7f628802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e617b900a6ac537 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f628ffff640 1 -- 172.21.15.60:0/3198668841 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62880317e0 con 0x7f62981409d0 2023-11-18T20:44:08.275 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f628ffff640 1 -- 172.21.15.60:0/3198668841 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288031940 con 0x7f62981409d0 2023-11-18T20:44:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f628ffff640 1 -- 172.21.15.60:0/3198668841 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288031cd0 con 0x7f62981409d0 2023-11-18T20:44:08.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f629f3d5640 1 -- 172.21.15.60:0/3198668841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62981409d0 msgr2=0x7f629806a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f629f3d5640 1 --2- 172.21.15.60:0/3198668841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62981409d0 0x7f629806a4d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f62980b5840 tx=0x7f628802b880 comp rx=0 tx=0).stop 2023-11-18T20:44:08.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f629f3d5640 1 -- 172.21.15.60:0/3198668841 shutdown_connections 2023-11-18T20:44:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f629f3d5640 1 --2- 172.21.15.60:0/3198668841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62981428d0 0x7f629806aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f629f3d5640 1 --2- 172.21.15.60:0/3198668841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62981409d0 0x7f629806a4d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f629f3d5640 1 --2- 172.21.15.60:0/3198668841 >> v2:172.21.15.100:3300/0 conn(0x7f6298140310 0x7f62981618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.273+0000 7f629f3d5640 1 -- 172.21.15.60:0/3198668841 >> 172.21.15.60:0/3198668841 conn(0x7f62980b1e50 msgr2=0x7f62980b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.274+0000 7f629f3d5640 1 -- 172.21.15.60:0/3198668841 shutdown_connections 2023-11-18T20:44:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.274+0000 7f629f3d5640 1 -- 172.21.15.60:0/3198668841 wait complete. 2023-11-18T20:44:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.275+0000 7f629f3d5640 1 Processor -- start 2023-11-18T20:44:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.275+0000 7f629f3d5640 1 -- start start 2023-11-18T20:44:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.275+0000 7f629f3d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6298140310 0x7f629815ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.275+0000 7f629f3d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62981409d0 0x7f6298144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.275+0000 7f629f3d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62981428d0 0x7f62980ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.275+0000 7f629f3d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629815cfa0 con 0x7f6298140310 2023-11-18T20:44:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.275+0000 7f629f3d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629815db80 con 0x7f62981409d0 2023-11-18T20:44:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.275+0000 7f629f3d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629815de60 con 0x7f62981428d0 2023-11-18T20:44:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.275+0000 7f629d171640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6298140310 0x7f629815ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629d171640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6298140310 0x7f629815ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55054/0 (socket says 172.21.15.60:55054) 2023-11-18T20:44:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629d171640 1 -- 172.21.15.60:0/2051954022 learned_addr learned my addr 172.21.15.60:0/2051954022 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629c970640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62981409d0 0x7f6298144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629d972640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62981428d0 0x7f62980ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629d171640 1 -- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62981428d0 msgr2=0x7f62980ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629d171640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62981428d0 0x7f62980ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629d171640 1 -- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62981409d0 msgr2=0x7f6298144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629d171640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62981409d0 0x7f6298144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629d171640 1 -- 172.21.15.60:0/2051954022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62880094d0 con 0x7f6298140310 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629d171640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6298140310 0x7f629815ca60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f629400c7f0 tx=0x7f629400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f628dffb640 1 -- 172.21.15.60:0/2051954022 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294007820 con 0x7f6298140310 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f628dffb640 1 -- 172.21.15.60:0/2051954022 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6294007980 con 0x7f6298140310 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f628dffb640 1 -- 172.21.15.60:0/2051954022 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294007cf0 con 0x7f6298140310 2023-11-18T20:44:08.282 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629f3d5640 1 -- 172.21.15.60:0/2051954022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629815e140 con 0x7f6298140310 2023-11-18T20:44:08.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.276+0000 7f629f3d5640 1 -- 172.21.15.60:0/2051954022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629815e700 con 0x7f6298140310 2023-11-18T20:44:08.283 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.277+0000 7f629f3d5640 1 -- 172.21.15.60:0/2051954022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6298069400 con 0x7f6298140310 2023-11-18T20:44:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.278+0000 7f628dffb640 1 -- 172.21.15.60:0/2051954022 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62940113f0 con 0x7f6298140310 2023-11-18T20:44:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.278+0000 7f628dffb640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f627c06ae40 0x7f627c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.278+0000 7f628dffb640 1 -- 172.21.15.60:0/2051954022 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f6294089a30 con 0x7f6298140310 2023-11-18T20:44:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.278+0000 7f629c970640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f627c06ae40 0x7f627c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.279+0000 7f629c970640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f627c06ae40 0x7f627c06d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6288036000 tx=0x7f628803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:08.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.283+0000 7f628dffb640 1 -- 172.21.15.60:0/2051954022 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6294054a00 con 0x7f6298140310 2023-11-18T20:44:08.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.460+0000 7f629f3d5640 1 -- 172.21.15.60:0/2051954022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f629806ea60 con 0x7f6298140310 2023-11-18T20:44:08.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.461+0000 7f628dffb640 1 -- 172.21.15.60:0/2051954022 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7f629401f0a0 con 0x7f6298140310 2023-11-18T20:44:08.462 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:08.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 -- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f627c06ae40 msgr2=0x7f627c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f627c06ae40 0x7f627c06d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6288036000 tx=0x7f628803d040 comp rx=0 tx=0).stop 2023-11-18T20:44:08.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 -- 172.21.15.60:0/2051954022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6298140310 msgr2=0x7f629815ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6298140310 0x7f629815ca60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f629400c7f0 tx=0x7f629400ccb0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 -- 172.21.15.60:0/2051954022 shutdown_connections 2023-11-18T20:44:08.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f627c06ae40 0x7f627c06d300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62981428d0 0x7f62980ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62981409d0 0x7f6298144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 --2- 172.21.15.60:0/2051954022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6298140310 0x7f629815ca60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 -- 172.21.15.60:0/2051954022 >> 172.21.15.60:0/2051954022 conn(0x7f62980b1e50 msgr2=0x7f62980714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:08.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 -- 172.21.15.60:0/2051954022 shutdown_connections 2023-11-18T20:44:08.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.463+0000 7f629f3d5640 1 -- 172.21.15.60:0/2051954022 wait complete. 2023-11-18T20:44:08.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:44:08.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.581+0000 7f6e8178e640 1 Processor -- start 2023-11-18T20:44:08.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.581+0000 7f6e8178e640 1 -- start start 2023-11-18T20:44:08.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e8178e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6e7c06ea50 0x7f6e7c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.584 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e8178e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6e7c0d0ed0 0x7f6e7c0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e8178e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c161d20 con 0x7f6e7c160fb0 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e8178e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c06f370 con 0x7f6e7c06ea50 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e8178e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c15e760 con 0x7f6e7c0d0ed0 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e7affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6e7c06ea50 0x7f6e7c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e7a7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6e7c160fb0 0x7f6e7c161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47834/0 (socket says 172.21.15.60:47834) 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e7a7fc640 1 -- 172.21.15.60:0/617849818 learned_addr learned my addr 172.21.15.60:0/617849818 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e7affd640 1 -- 172.21.15.60:0/617849818 >> v2:172.21.15.100:3300/0 conn(0x7f6e7c0d0ed0 msgr2=0x7f6e7c0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e7b7fe640 1 --2- 172.21.15.60:0/617849818 >> v2:172.21.15.100:3300/0 conn(0x7f6e7c0d0ed0 0x7f6e7c0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e7affd640 1 --2- 172.21.15.60:0/617849818 >> v2:172.21.15.100:3300/0 conn(0x7f6e7c0d0ed0 0x7f6e7c0d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e7affd640 1 -- 172.21.15.60:0/617849818 >> v1:172.21.15.60:6789/0 conn(0x7f6e7c160fb0 legacy=0x7f6e7c161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e79ffb640 1 -- 172.21.15.60:0/617849818 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630517586 0 0) 0x7f6e7c161d20 con 0x7f6e7c160fb0 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e7affd640 1 -- 172.21.15.60:0/617849818 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c15d9a0 con 0x7f6e7c06ea50 2023-11-18T20:44:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.582+0000 7f6e7b7fe640 1 --2- 172.21.15.60:0/617849818 >> v2:172.21.15.100:3300/0 conn(0x7f6e7c0d0ed0 0x7f6e7c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:08.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.583+0000 7f6e7affd640 1 --2- 172.21.15.60:0/617849818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 0x7f6e7c06ee30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c1407f0 tx=0x7f6e7402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5662d7080c9c9f85 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:08.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.583+0000 7f6e79ffb640 1 -- 172.21.15.60:0/617849818 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7402e070 con 0x7f6e7c06ea50 2023-11-18T20:44:08.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.583+0000 7f6e79ffb640 1 -- 172.21.15.60:0/617849818 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e740315a0 con 0x7f6e7c06ea50 2023-11-18T20:44:08.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.583+0000 7f6e79ffb640 1 -- 172.21.15.60:0/617849818 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74031930 con 0x7f6e7c06ea50 2023-11-18T20:44:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.584+0000 7f6e8178e640 1 -- 172.21.15.60:0/617849818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 msgr2=0x7f6e7c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.584+0000 7f6e8178e640 1 --2- 172.21.15.60:0/617849818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 0x7f6e7c06ee30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c1407f0 tx=0x7f6e7402b840 comp rx=0 tx=0).stop 2023-11-18T20:44:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.584+0000 7f6e8178e640 1 -- 172.21.15.60:0/617849818 shutdown_connections 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.584+0000 7f6e8178e640 1 --2- 172.21.15.60:0/617849818 >> v2:172.21.15.100:3300/0 conn(0x7f6e7c0d0ed0 0x7f6e7c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.584+0000 7f6e8178e640 1 --2- 172.21.15.60:0/617849818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 0x7f6e7c06ee30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.584+0000 7f6e8178e640 1 -- 172.21.15.60:0/617849818 >> 172.21.15.60:0/617849818 conn(0x7f6e7c06a6b0 msgr2=0x7f6e7c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.584+0000 7f6e8178e640 1 -- 172.21.15.60:0/617849818 shutdown_connections 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.584+0000 7f6e8178e640 1 -- 172.21.15.60:0/617849818 wait complete. 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.585+0000 7f6e8178e640 1 Processor -- start 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.585+0000 7f6e8178e640 1 -- start start 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e8178e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 0x7f6e7c13f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e8178e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0ed0 0x7f6e7c1e7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e8178e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7c160fb0 0x7f6e7c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e8178e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1591e0 con 0x7f6e7c06ea50 2023-11-18T20:44:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e8178e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1e7980 con 0x7f6e7c160fb0 2023-11-18T20:44:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e8178e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1e7e90 con 0x7f6e7c0d0ed0 2023-11-18T20:44:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 0x7f6e7c13f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 0x7f6e7c13f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55070/0 (socket says 172.21.15.60:55070) 2023-11-18T20:44:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7affd640 1 -- 172.21.15.60:0/456834675 learned_addr learned my addr 172.21.15.60:0/456834675 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7a7fc640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0ed0 0x7f6e7c1e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7b7fe640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7c160fb0 0x7f6e7c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7affd640 1 -- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0ed0 msgr2=0x7f6e7c1e7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7affd640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0ed0 0x7f6e7c1e7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7affd640 1 -- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7c160fb0 msgr2=0x7f6e7c1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7affd640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7c160fb0 0x7f6e7c1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7affd640 1 -- 172.21.15.60:0/456834675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e74009400 con 0x7f6e7c06ea50 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.586+0000 7f6e7a7fc640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0ed0 0x7f6e7c1e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.587+0000 7f6e7affd640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 0x7f6e7c13f6e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009890 tx=0x7f6e740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.587+0000 7f6e737fe640 1 -- 172.21.15.60:0/456834675 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74032900 con 0x7f6e7c06ea50 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.587+0000 7f6e737fe640 1 -- 172.21.15.60:0/456834675 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e74032a60 con 0x7f6e7c06ea50 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.587+0000 7f6e737fe640 1 -- 172.21.15.60:0/456834675 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74032e10 con 0x7f6e7c06ea50 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.587+0000 7f6e8178e640 1 -- 172.21.15.60:0/456834675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c1e7bb0 con 0x7f6e7c06ea50 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.587+0000 7f6e8178e640 1 -- 172.21.15.60:0/456834675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c1596f0 con 0x7f6e7c06ea50 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.588+0000 7f6e8178e640 1 -- 172.21.15.60:0/456834675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c069400 con 0x7f6e7c06ea50 2023-11-18T20:44:08.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.588+0000 7f6e737fe640 1 -- 172.21.15.60:0/456834675 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e7403b3f0 con 0x7f6e7c06ea50 2023-11-18T20:44:08.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.589+0000 7f6e737fe640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e5c06ae40 0x7f6e5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.589+0000 7f6e737fe640 1 -- 172.21.15.60:0/456834675 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f6e740ab920 con 0x7f6e7c06ea50 2023-11-18T20:44:08.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.589+0000 7f6e7a7fc640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e5c06ae40 0x7f6e5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.590+0000 7f6e7a7fc640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e5c06ae40 0x7f6e5c06d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c06d410 tx=0x7f6e64007c10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:08.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.593+0000 7f6e737fe640 1 -- 172.21.15.60:0/456834675 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e74076840 con 0x7f6e7c06ea50 2023-11-18T20:44:08.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.784+0000 7f6e8178e640 1 -- 172.21.15.60:0/456834675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6e7c0b4df0 con 0x7f6e7c06ea50 2023-11-18T20:44:08.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.785+0000 7f6e737fe640 1 -- 172.21.15.60:0/456834675 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6e74079e60 con 0x7f6e7c06ea50 2023-11-18T20:44:08.786 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:44:08.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 -- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e5c06ae40 msgr2=0x7f6e5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e5c06ae40 0x7f6e5c06d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c06d410 tx=0x7f6e64007c10 comp rx=0 tx=0).stop 2023-11-18T20:44:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 -- 172.21.15.60:0/456834675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 msgr2=0x7f6e7c13f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 0x7f6e7c13f6e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009890 tx=0x7f6e740046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 -- 172.21.15.60:0/456834675 shutdown_connections 2023-11-18T20:44:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e5c06ae40 0x7f6e5c06d300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e7c160fb0 0x7f6e7c1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e7c0d0ed0 0x7f6e7c1e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 --2- 172.21.15.60:0/456834675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e7c06ea50 0x7f6e7c13f6e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 -- 172.21.15.60:0/456834675 >> 172.21.15.60:0/456834675 conn(0x7f6e7c06a6b0 msgr2=0x7f6e7c15cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 -- 172.21.15.60:0/456834675 shutdown_connections 2023-11-18T20:44:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.787+0000 7f6e8178e640 1 -- 172.21.15.60:0/456834675 wait complete. 2023-11-18T20:44:08.799 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:08.800 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:08.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:08.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.960+0000 7f227a5b0640 1 Processor -- start 2023-11-18T20:44:08.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.960+0000 7f227a5b0640 1 -- start start 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.961+0000 7f227a5b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22740ae480 0x7f227406d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.961+0000 7f227a5b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740ae910 con 0x7f2274072390 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.961+0000 7f227a5b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740b2200 con 0x7f227415d2b0 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.961+0000 7f227a5b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227406df90 con 0x7f22740ae480 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.961+0000 7f22737fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22740ae480 0x7f227406d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.961+0000 7f2278b4d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2274072390 0x7f227406d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47838/0 (socket says 172.21.15.60:47838) 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.961+0000 7f2278b4d640 1 -- 172.21.15.60:0/3879906289 learned_addr learned my addr 172.21.15.60:0/3879906289 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.961+0000 7f22737fe640 1 -- 172.21.15.60:0/3879906289 >> v1:172.21.15.100:6789/0 conn(0x7f227415d2b0 legacy=0x7f2274071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.961+0000 7f22737fe640 1 -- 172.21.15.60:0/3879906289 >> v1:172.21.15.60:6789/0 conn(0x7f2274072390 legacy=0x7f227406d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.962+0000 7f22737fe640 1 -- 172.21.15.60:0/3879906289 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22740b24e0 con 0x7f22740ae480 2023-11-18T20:44:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.962+0000 7f2272ffd640 1 -- 172.21.15.60:0/3879906289 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646338836 0 0) 0x7f22740ae910 con 0x7f2274072390 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.962+0000 7f22737fe640 1 --2- 172.21.15.60:0/3879906289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740ae480 0x7f227406d340 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f22640097a0 tx=0x7f226402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5bbf1e5955a0f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.962+0000 7f2272ffd640 1 -- 172.21.15.60:0/3879906289 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226402e070 con 0x7f22740ae480 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.962+0000 7f2272ffd640 1 -- 172.21.15.60:0/3879906289 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22640315a0 con 0x7f22740ae480 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.962+0000 7f2272ffd640 1 -- 172.21.15.60:0/3879906289 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264031930 con 0x7f22740ae480 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.963+0000 7f227a5b0640 1 -- 172.21.15.60:0/3879906289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740ae480 msgr2=0x7f227406d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.963+0000 7f227a5b0640 1 --2- 172.21.15.60:0/3879906289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740ae480 0x7f227406d340 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f22640097a0 tx=0x7f226402b840 comp rx=0 tx=0).stop 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.963+0000 7f227a5b0640 1 -- 172.21.15.60:0/3879906289 shutdown_connections 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.963+0000 7f227a5b0640 1 --2- 172.21.15.60:0/3879906289 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740ae480 0x7f227406d340 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.963+0000 7f227a5b0640 1 -- 172.21.15.60:0/3879906289 >> 172.21.15.60:0/3879906289 conn(0x7f22740b4720 msgr2=0x7f22740b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.963+0000 7f227a5b0640 1 -- 172.21.15.60:0/3879906289 shutdown_connections 2023-11-18T20:44:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.963+0000 7f227a5b0640 1 -- 172.21.15.60:0/3879906289 wait complete. 2023-11-18T20:44:08.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.964+0000 7f227a5b0640 1 Processor -- start 2023-11-18T20:44:08.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.964+0000 7f227a5b0640 1 -- start start 2023-11-18T20:44:08.967 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f227a5b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274072390 0x7f22741f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f227a5b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227415d2b0 0x7f227415a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f227a5b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2274158dd0 0x7f2274159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f227a5b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22741f2990 con 0x7f2274158dd0 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f227a5b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2274159890 con 0x7f2274072390 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f227a5b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2274159b70 con 0x7f227415d2b0 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f2278b4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2274158dd0 0x7f2274159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f2278b4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2274158dd0 0x7f2274159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55074/0 (socket says 172.21.15.60:55074) 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f2278b4d640 1 -- 172.21.15.60:0/3358577281 learned_addr learned my addr 172.21.15.60:0/3358577281 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f2273fff640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274072390 0x7f22741f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f22737fe640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227415d2b0 0x7f227415a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f2278b4d640 1 -- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227415d2b0 msgr2=0x7f227415a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f2278b4d640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227415d2b0 0x7f227415a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f2278b4d640 1 -- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274072390 msgr2=0x7f22741f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f2278b4d640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274072390 0x7f22741f2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.965+0000 7f2278b4d640 1 -- 172.21.15.60:0/3358577281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009400 con 0x7f2274158dd0 2023-11-18T20:44:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.966+0000 7f2278b4d640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2274158dd0 0x7f2274159280 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f226c00c7e0 tx=0x7f226c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.966+0000 7f2270ff9640 1 -- 172.21.15.60:0/3358577281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00a640 con 0x7f2274158dd0 2023-11-18T20:44:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.966+0000 7f2270ff9640 1 -- 172.21.15.60:0/3358577281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c00a7c0 con 0x7f2274158dd0 2023-11-18T20:44:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.966+0000 7f2270ff9640 1 -- 172.21.15.60:0/3358577281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00ab70 con 0x7f2274158dd0 2023-11-18T20:44:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.966+0000 7f227a5b0640 1 -- 172.21.15.60:0/3358577281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227409bf60 con 0x7f2274158dd0 2023-11-18T20:44:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.966+0000 7f227a5b0640 1 -- 172.21.15.60:0/3358577281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227414bf00 con 0x7f2274158dd0 2023-11-18T20:44:08.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.967+0000 7f227a5b0640 1 -- 172.21.15.60:0/3358577281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227409c480 con 0x7f2274158dd0 2023-11-18T20:44:08.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.967+0000 7f2270ff9640 1 -- 172.21.15.60:0/3358577281 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f226c00acd0 con 0x7f2274158dd0 2023-11-18T20:44:08.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.968+0000 7f2270ff9640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f226006ae70 0x7f226006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:08.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.968+0000 7f2270ff9640 1 -- 172.21.15.60:0/3358577281 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f226c088b90 con 0x7f2274158dd0 2023-11-18T20:44:08.970 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.968+0000 7f2273fff640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f226006ae70 0x7f226006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.968+0000 7f2273fff640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f226006ae70 0x7f226006d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f226800a750 tx=0x7f2268008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:08.973 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:08.972+0000 7f2270ff9640 1 -- 172.21.15.60:0/3358577281 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f227409c480 con 0x7f2274158dd0 2023-11-18T20:44:09.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:44:09.118 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:44:09.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.148+0000 7f227a5b0640 1 -- 172.21.15.60:0/3358577281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f227409c480 con 0x7f2274158dd0 2023-11-18T20:44:09.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.150+0000 7f2270ff9640 1 -- 172.21.15.60:0/3358577281 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18336 (secure 0 0 0) 0x7f227409c480 con 0x7f2274158dd0 2023-11-18T20:44:09.151 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:09.151 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:42.814654+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-11-18T20:43:40.650315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 -- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f226006ae70 msgr2=0x7f226006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f226006ae70 0x7f226006d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f226800a750 tx=0x7f2268008040 comp rx=0 tx=0).stop 2023-11-18T20:44:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 -- 172.21.15.60:0/3358577281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2274158dd0 msgr2=0x7f2274159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2274158dd0 0x7f2274159280 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f226c00c7e0 tx=0x7f226c00cca0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 -- 172.21.15.60:0/3358577281 shutdown_connections 2023-11-18T20:44:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f226006ae70 0x7f226006d330 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2274158dd0 0x7f2274159280 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227415d2b0 0x7f227415a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 --2- 172.21.15.60:0/3358577281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2274072390 0x7f22741f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 -- 172.21.15.60:0/3358577281 >> 172.21.15.60:0/3358577281 conn(0x7f22740b4720 msgr2=0x7f22741581f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 -- 172.21.15.60:0/3358577281 shutdown_connections 2023-11-18T20:44:09.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.152+0000 7f227a5b0640 1 -- 172.21.15.60:0/3358577281 wait complete. 2023-11-18T20:44:09.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:43:31.703619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:43:32.121610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-11-18T20:43:40.650315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:44:09.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:44:09.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.327+0000 7f2d8bb3a640 1 Processor -- start 2023-11-18T20:44:09.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.327+0000 7f2d8bb3a640 1 -- start start 2023-11-18T20:44:09.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.327+0000 7f2d8bb3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d84143490 0x7f2d840b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.327+0000 7f2d8bb3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d8415abf0 0x7f2d840b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.328+0000 7f2d8bb3a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8415e2a0 con 0x7f2d8415e730 2023-11-18T20:44:09.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.328+0000 7f2d8bb3a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8415b520 con 0x7f2d84143490 2023-11-18T20:44:09.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.328+0000 7f2d8bb3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d84143870 con 0x7f2d8415abf0 2023-11-18T20:44:09.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.328+0000 7f2d898d6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d84143490 0x7f2d840b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.328+0000 7f2d898d6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2d84143490 0x7f2d840b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55088/0 (socket says 172.21.15.60:55088) 2023-11-18T20:44:09.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.328+0000 7f2d898d6640 1 -- 172.21.15.60:0/2936865776 learned_addr learned my addr 172.21.15.60:0/2936865776 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:09.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.328+0000 7f2d898d6640 1 -- 172.21.15.60:0/2936865776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d8415abf0 msgr2=0x7f2d840b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:09.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.328+0000 7f2d898d6640 1 --2- 172.21.15.60:0/2936865776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d8415abf0 0x7f2d840b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.328+0000 7f2d898d6640 1 -- 172.21.15.60:0/2936865776 >> v1:172.21.15.100:6789/0 conn(0x7f2d8415e730 legacy=0x7f2d8415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:09.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.328+0000 7f2d898d6640 1 -- 172.21.15.60:0/2936865776 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8415b240 con 0x7f2d84143490 2023-11-18T20:44:09.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.329+0000 7f2d898d6640 1 --2- 172.21.15.60:0/2936865776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d84143490 0x7f2d840b0cf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009870 tx=0x7f2d8002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e6721014fcebdb server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:09.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.329+0000 7f2d888d4640 1 -- 172.21.15.60:0/2936865776 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80031640 con 0x7f2d84143490 2023-11-18T20:44:09.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.329+0000 7f2d888d4640 1 -- 172.21.15.60:0/2936865776 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d800317a0 con 0x7f2d84143490 2023-11-18T20:44:09.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.329+0000 7f2d888d4640 1 -- 172.21.15.60:0/2936865776 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80031b30 con 0x7f2d84143490 2023-11-18T20:44:09.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.329+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/2936865776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d84143490 msgr2=0x7f2d840b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.329+0000 7f2d8bb3a640 1 --2- 172.21.15.60:0/2936865776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d84143490 0x7f2d840b0cf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009870 tx=0x7f2d8002b8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.330+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/2936865776 shutdown_connections 2023-11-18T20:44:09.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.330+0000 7f2d8bb3a640 1 --2- 172.21.15.60:0/2936865776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d8415abf0 0x7f2d840b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.330+0000 7f2d8bb3a640 1 --2- 172.21.15.60:0/2936865776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d84143490 0x7f2d840b0cf0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.330+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/2936865776 >> 172.21.15.60:0/2936865776 conn(0x7f2d8406a4d0 msgr2=0x7f2d8406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:09.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.330+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/2936865776 shutdown_connections 2023-11-18T20:44:09.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.330+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/2936865776 wait complete. 2023-11-18T20:44:09.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.331+0000 7f2d8bb3a640 1 Processor -- start 2023-11-18T20:44:09.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.331+0000 7f2d8bb3a640 1 -- start start 2023-11-18T20:44:09.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.331+0000 7f2d8bb3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d84143490 0x7f2d840a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8bb3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d8415abf0 0x7f2d841e5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8bb3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8415e730 0x7f2d841f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8bb3a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841e5a60 con 0x7f2d8415e730 2023-11-18T20:44:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8bb3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841e5c90 con 0x7f2d84143490 2023-11-18T20:44:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8bb3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841e5f70 con 0x7f2d8415abf0 2023-11-18T20:44:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8a0d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8415e730 0x7f2d841f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8a0d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8415e730 0x7f2d841f0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55102/0 (socket says 172.21.15.60:55102) 2023-11-18T20:44:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8a0d7640 1 -- 172.21.15.60:0/3870182610 learned_addr learned my addr 172.21.15.60:0/3870182610 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d898d6640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d84143490 0x7f2d840a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d890d5640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d8415abf0 0x7f2d841e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8a0d7640 1 -- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d8415abf0 msgr2=0x7f2d841e5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8a0d7640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d8415abf0 0x7f2d841e5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8a0d7640 1 -- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d84143490 msgr2=0x7f2d840a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8a0d7640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d84143490 0x7f2d840a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8a0d7640 1 -- 172.21.15.60:0/3870182610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d800094d0 con 0x7f2d8415e730 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d890d5640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d8415abf0 0x7f2d841e5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d898d6640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d84143490 0x7f2d840a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8a0d7640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8415e730 0x7f2d841f0fd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00b970 tx=0x7f2d7c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d7a7fc640 1 -- 172.21.15.60:0/3870182610 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00cb40 con 0x7f2d8415e730 2023-11-18T20:44:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d7a7fc640 1 -- 172.21.15.60:0/3870182610 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c00ccc0 con 0x7f2d8415e730 2023-11-18T20:44:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/3870182610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d841e6250 con 0x7f2d8415e730 2023-11-18T20:44:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d7a7fc640 1 -- 172.21.15.60:0/3870182610 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c014680 con 0x7f2d8415e730 2023-11-18T20:44:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.332+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/3870182610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d841e6810 con 0x7f2d8415e730 2023-11-18T20:44:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.333+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/3870182610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d841e6a30 con 0x7f2d8415e730 2023-11-18T20:44:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.334+0000 7f2d7a7fc640 1 -- 172.21.15.60:0/3870182610 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d7c01d3b0 con 0x7f2d8415e730 2023-11-18T20:44:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.334+0000 7f2d7a7fc640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d6806ae40 0x7f2d6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.334+0000 7f2d7a7fc640 1 -- 172.21.15.60:0/3870182610 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f2d7c089710 con 0x7f2d8415e730 2023-11-18T20:44:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.334+0000 7f2d898d6640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d6806ae40 0x7f2d6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.335+0000 7f2d898d6640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d6806ae40 0x7f2d6806d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2d80036000 tx=0x7f2d8003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:09.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.338+0000 7f2d7a7fc640 1 -- 172.21.15.60:0/3870182610 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d7c0580c0 con 0x7f2d8415e730 2023-11-18T20:44:09.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.514+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/3870182610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d84069400 con 0x7f2d8415e730 2023-11-18T20:44:09.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.514+0000 7f2d7a7fc640 1 -- 172.21.15.60:0/3870182610 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d7c019020 con 0x7f2d8415e730 2023-11-18T20:44:09.515 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:44:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d6806ae40 msgr2=0x7f2d6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d6806ae40 0x7f2d6806d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2d80036000 tx=0x7f2d8003c040 comp rx=0 tx=0).stop 2023-11-18T20:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/3870182610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8415e730 msgr2=0x7f2d841f0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8415e730 0x7f2d841f0fd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00b970 tx=0x7f2d7c00be30 comp rx=0 tx=0).stop 2023-11-18T20:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/3870182610 shutdown_connections 2023-11-18T20:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2d6806ae40 0x7f2d6806d300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8415e730 0x7f2d841f0fd0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d8415abf0 0x7f2d841e5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 --2- 172.21.15.60:0/3870182610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d84143490 0x7f2d840a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/3870182610 >> 172.21.15.60:0/3870182610 conn(0x7f2d8406a4d0 msgr2=0x7f2d84070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/3870182610 shutdown_connections 2023-11-18T20:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.516+0000 7f2d8bb3a640 1 -- 172.21.15.60:0/3870182610 wait complete. 2023-11-18T20:44:09.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:44:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.635+0000 7fe2b10a2640 1 Processor -- start 2023-11-18T20:44:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.636+0000 7fe2b10a2640 1 -- start start 2023-11-18T20:44:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.636+0000 7fe2b10a2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2ac15d330 0x7fe2ac0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.636+0000 7fe2b10a2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac0b2b10 con 0x7fe2ac0d3900 2023-11-18T20:44:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.636+0000 7fe2b10a2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac0d1590 con 0x7fe2ac0d2910 2023-11-18T20:44:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.636+0000 7fe2b10a2640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ac0d1870 con 0x7fe2ac15d330 2023-11-18T20:44:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.636+0000 7fe2aad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2ac15d330 0x7fe2ac0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.636+0000 7fe2ab59e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe2ac0d3900 0x7fe2ac0b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47840/0 (socket says 172.21.15.60:47840) 2023-11-18T20:44:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.636+0000 7fe2aad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2ac15d330 0x7fe2ac0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55112/0 (socket says 172.21.15.60:55112) 2023-11-18T20:44:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.636+0000 7fe2ab59e640 1 -- 172.21.15.60:0/941897799 learned_addr learned my addr 172.21.15.60:0/941897799 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:09.638 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.637+0000 7fe2aad9d640 1 -- 172.21.15.60:0/941897799 >> v1:172.21.15.100:6789/0 conn(0x7fe2ac0d2910 legacy=0x7fe2ac0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.637+0000 7fe2aad9d640 1 -- 172.21.15.60:0/941897799 >> v1:172.21.15.60:6789/0 conn(0x7fe2ac0d3900 legacy=0x7fe2ac0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.637+0000 7fe2aa59c640 1 -- 172.21.15.60:0/941897799 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164919131 0 0) 0x7fe2ac0b2b10 con 0x7fe2ac0d3900 2023-11-18T20:44:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.637+0000 7fe2aad9d640 1 -- 172.21.15.60:0/941897799 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac0d1b50 con 0x7fe2ac15d330 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.637+0000 7fe2aad9d640 1 --2- 172.21.15.60:0/941897799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac15d330 0x7fe2ac0b8b80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0097a0 tx=0x7fe29c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8398a8e6a89118a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.638+0000 7fe2aa59c640 1 -- 172.21.15.60:0/941897799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c02e070 con 0x7fe2ac15d330 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.638+0000 7fe2aa59c640 1 -- 172.21.15.60:0/941897799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c0315a0 con 0x7fe2ac15d330 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.638+0000 7fe2aa59c640 1 -- 172.21.15.60:0/941897799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c031930 con 0x7fe2ac15d330 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.638+0000 7fe2b10a2640 1 -- 172.21.15.60:0/941897799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac15d330 msgr2=0x7fe2ac0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.638+0000 7fe2b10a2640 1 --2- 172.21.15.60:0/941897799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac15d330 0x7fe2ac0b8b80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0097a0 tx=0x7fe29c02b840 comp rx=0 tx=0).stop 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.638+0000 7fe2b10a2640 1 -- 172.21.15.60:0/941897799 shutdown_connections 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.638+0000 7fe2b10a2640 1 --2- 172.21.15.60:0/941897799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac15d330 0x7fe2ac0b8b80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.638+0000 7fe2b10a2640 1 -- 172.21.15.60:0/941897799 >> 172.21.15.60:0/941897799 conn(0x7fe2ac072020 msgr2=0x7fe2ac072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.638+0000 7fe2b10a2640 1 -- 172.21.15.60:0/941897799 shutdown_connections 2023-11-18T20:44:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.638+0000 7fe2b10a2640 1 -- 172.21.15.60:0/941897799 wait complete. 2023-11-18T20:44:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.640+0000 7fe2b10a2640 1 Processor -- start 2023-11-18T20:44:09.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.640+0000 7fe2b10a2640 1 -- start start 2023-11-18T20:44:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.640+0000 7fe2b10a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac0d2910 0x7fe2ac149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.640+0000 7fe2b10a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ac0d3900 0x7fe2ac0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.640+0000 7fe2b10a2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ac15d330 0x7fe2ac1f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.640+0000 7fe2b10a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1f2aa0 con 0x7fe2ac0d2910 2023-11-18T20:44:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.640+0000 7fe2b10a2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0cd6a0 con 0x7fe2ac0d3900 2023-11-18T20:44:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.640+0000 7fe2b10a2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac15eb00 con 0x7fe2ac15d330 2023-11-18T20:44:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.640+0000 7fe2aad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac0d2910 0x7fe2ac149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2aad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac0d2910 0x7fe2ac149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55124/0 (socket says 172.21.15.60:55124) 2023-11-18T20:44:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2aad9d640 1 -- 172.21.15.60:0/761402601 learned_addr learned my addr 172.21.15.60:0/761402601 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2ab59e640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ac15d330 0x7fe2ac1f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2a3fff640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ac0d3900 0x7fe2ac0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2aad9d640 1 -- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ac15d330 msgr2=0x7fe2ac1f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2aad9d640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ac15d330 0x7fe2ac1f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2aad9d640 1 -- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ac0d3900 msgr2=0x7fe2ac0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2aad9d640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ac0d3900 0x7fe2ac0ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2aad9d640 1 -- 172.21.15.60:0/761402601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c009400 con 0x7fe2ac0d2910 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2aad9d640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac0d2910 0x7fe2ac149210 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe29c034040 tx=0x7fe29c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2a37fe640 1 -- 172.21.15.60:0/761402601 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c032900 con 0x7fe2ac0d2910 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2a37fe640 1 -- 172.21.15.60:0/761402601 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c032a80 con 0x7fe2ac0d2910 2023-11-18T20:44:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2b10a2640 1 -- 172.21.15.60:0/761402601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac0cce00 con 0x7fe2ac0d2910 2023-11-18T20:44:09.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2a37fe640 1 -- 172.21.15.60:0/761402601 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c032e30 con 0x7fe2ac0d2910 2023-11-18T20:44:09.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.641+0000 7fe2b10a2640 1 -- 172.21.15.60:0/761402601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac0cd3c0 con 0x7fe2ac0d2910 2023-11-18T20:44:09.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.642+0000 7fe2b10a2640 1 -- 172.21.15.60:0/761402601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ac15ecc0 con 0x7fe2ac0d2910 2023-11-18T20:44:09.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.643+0000 7fe2a37fe640 1 -- 172.21.15.60:0/761402601 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe29c03b3f0 con 0x7fe2ac0d2910 2023-11-18T20:44:09.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.643+0000 7fe2a37fe640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe28c06ae40 0x7fe28c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.643+0000 7fe2a37fe640 1 -- 172.21.15.60:0/761402601 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe29c0ab820 con 0x7fe2ac0d2910 2023-11-18T20:44:09.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.643+0000 7fe2a3fff640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe28c06ae40 0x7fe28c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.644+0000 7fe2a3fff640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe28c06ae40 0x7fe28c06d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac15d070 tx=0x7fe294009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:09.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.648+0000 7fe2a37fe640 1 -- 172.21.15.60:0/761402601 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe29c0767d0 con 0x7fe2ac0d2910 2023-11-18T20:44:09.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.822+0000 7fe2b10a2640 1 -- 172.21.15.60:0/761402601 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2ac0d2da0 con 0x7fe2ac0d2910 2023-11-18T20:44:09.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.822+0000 7fe2a37fe640 1 -- 172.21.15.60:0/761402601 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe29c079df0 con 0x7fe2ac0d2910 2023-11-18T20:44:09.824 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:09.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 -- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe28c06ae40 msgr2=0x7fe28c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe28c06ae40 0x7fe28c06d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac15d070 tx=0x7fe294009040 comp rx=0 tx=0).stop 2023-11-18T20:44:09.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 -- 172.21.15.60:0/761402601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac0d2910 msgr2=0x7fe2ac149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac0d2910 0x7fe2ac149210 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe29c034040 tx=0x7fe29c0046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 -- 172.21.15.60:0/761402601 shutdown_connections 2023-11-18T20:44:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe28c06ae40 0x7fe28c06d300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ac15d330 0x7fe2ac1f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ac0d3900 0x7fe2ac0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 --2- 172.21.15.60:0/761402601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ac0d2910 0x7fe2ac149210 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 -- 172.21.15.60:0/761402601 >> 172.21.15.60:0/761402601 conn(0x7fe2ac072020 msgr2=0x7fe2ac072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 -- 172.21.15.60:0/761402601 shutdown_connections 2023-11-18T20:44:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.825+0000 7fe2b10a2640 1 -- 172.21.15.60:0/761402601 wait complete. 2023-11-18T20:44:09.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:44:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.946+0000 7f675c353640 1 Processor -- start 2023-11-18T20:44:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.946+0000 7f675c353640 1 -- start start 2023-11-18T20:44:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f675c353640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6754160fb0 0x7f6754161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f675c353640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67540d0ed0 0x7f67540d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f675c353640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6754161d20 con 0x7f675406ea50 2023-11-18T20:44:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f675c353640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675415e760 con 0x7f6754160fb0 2023-11-18T20:44:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f675c353640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675415d9a0 con 0x7f67540d0ed0 2023-11-18T20:44:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f675a0ef640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f675406ea50 0x7f675406ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47854/0 (socket says 172.21.15.60:47854) 2023-11-18T20:44:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f675a0ef640 1 -- 172.21.15.60:0/1795768271 learned_addr learned my addr 172.21.15.60:0/1795768271 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635832590 0 0) 0x7f6754161d20 con 0x7f675406ea50 2023-11-18T20:44:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67598ee640 1 --2- 172.21.15.60:0/1795768271 >> v2:172.21.15.100:3300/0 conn(0x7f6754160fb0 0x7f6754161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f675a8f0640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67540d0ed0 0x7f67540d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6740003520 con 0x7f675406ea50 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2213862293 0 0) 0x7f6740003520 con 0x7f675406ea50 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67540d0ed0 msgr2=0x7f67540d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67540d0ed0 0x7f67540d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 >> v2:172.21.15.100:3300/0 conn(0x7f6754160fb0 msgr2=0x7f6754161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 --2- 172.21.15.60:0/1795768271 >> v2:172.21.15.100:3300/0 conn(0x7f6754160fb0 0x7f6754161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675415dc80 con 0x7f675406ea50 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f6750001670 con 0x7f675406ea50 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 >> v1:172.21.15.60:6789/0 conn(0x7f675406ea50 legacy=0x7f675406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6740003970 0x7f6740003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.947+0000 7f67590ed640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67400047c0 0x7f6740004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f67590ed640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67540d0ed0 0x7f6740005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675415e760 con 0x7f67400047c0 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675415d9a0 con 0x7f6740003970 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674000b980 con 0x7f67540d0ed0 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f67598ee640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67400047c0 0x7f6740004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f675a0ef640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6740003970 0x7f6740003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f675a8f0640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67540d0ed0 0x7f6740005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f67598ee640 1 -- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67540d0ed0 msgr2=0x7f6740005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f67598ee640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67540d0ed0 0x7f6740005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f67598ee640 1 -- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6740003970 msgr2=0x7f6740003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f67598ee640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6740003970 0x7f6740003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.948+0000 7f67598ee640 1 -- 172.21.15.60:0/1795768271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740003520 con 0x7f67400047c0 2023-11-18T20:44:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.949+0000 7f67598ee640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67400047c0 0x7f6740004c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6744009ac0 tx=0x7f67400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e52c6d67faf9b09 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.949+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674402c710 con 0x7f67400047c0 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.949+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674402c870 con 0x7f67400047c0 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.949+0000 7f67590ed640 1 -- 172.21.15.60:0/1795768271 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674402cc00 con 0x7f67400047c0 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.949+0000 7f675c353640 1 -- 172.21.15.60:0/1795768271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67400047c0 msgr2=0x7f6740004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.949+0000 7f675c353640 1 --2- 172.21.15.60:0/1795768271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67400047c0 0x7f6740004c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6744009ac0 tx=0x7f67400057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.949+0000 7f675a8f0640 1 -- 172.21.15.60:0/1795768271 reap_dead start 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.949+0000 7f675c353640 1 -- 172.21.15.60:0/1795768271 shutdown_connections 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.949+0000 7f675c353640 1 -- 172.21.15.60:0/1795768271 >> 172.21.15.60:0/1795768271 conn(0x7f675406a6b0 msgr2=0x7f675406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.949+0000 7f675c353640 1 -- 172.21.15.60:0/1795768271 shutdown_connections 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.950+0000 7f675c353640 1 -- 172.21.15.60:0/1795768271 wait complete. 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.951+0000 7f675c353640 1 Processor -- start 2023-11-18T20:44:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.951+0000 7f675c353640 1 -- start start 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.951+0000 7f675c353640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67400047c0 0x7f675413d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.951+0000 7f675c353640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67541585b0 0x7f67540ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.951+0000 7f675c353640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f675406c630 0x7f675406cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.951+0000 7f675c353640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675406cff0 con 0x7f675406c630 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.951+0000 7f675c353640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6754158ab0 con 0x7f67541585b0 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.951+0000 7f675c353640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6754159630 con 0x7f67400047c0 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.951+0000 7f675a8f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f675406c630 0x7f675406cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675a8f0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f675406c630 0x7f675406cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55138/0 (socket says 172.21.15.60:55138) 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675a8f0640 1 -- 172.21.15.60:0/1422692696 learned_addr learned my addr 172.21.15.60:0/1422692696 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f67598ee640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67541585b0 0x7f67540ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675a0ef640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67400047c0 0x7f675413d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675a8f0640 1 -- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67400047c0 msgr2=0x7f675413d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675a8f0640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67400047c0 0x7f675413d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675a8f0640 1 -- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67541585b0 msgr2=0x7f67540ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675a8f0640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67541585b0 0x7f67540ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675a8f0640 1 -- 172.21.15.60:0/1422692696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754158d00 con 0x7f675406c630 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f67598ee640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67541585b0 0x7f67540ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675a8f0640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f675406c630 0x7f675406cab0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f674c011b40 tx=0x7f674c00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f674affd640 1 -- 172.21.15.60:0/1422692696 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c00e040 con 0x7f675406c630 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f674affd640 1 -- 172.21.15.60:0/1422692696 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674c00f040 con 0x7f675406c630 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675c353640 1 -- 172.21.15.60:0/1422692696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6754158fe0 con 0x7f675406c630 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f674affd640 1 -- 172.21.15.60:0/1422692696 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c015430 con 0x7f675406c630 2023-11-18T20:44:09.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.952+0000 7f675c353640 1 -- 172.21.15.60:0/1422692696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675413de60 con 0x7f675406c630 2023-11-18T20:44:09.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.953+0000 7f675c353640 1 -- 172.21.15.60:0/1422692696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67540b5330 con 0x7f675406c630 2023-11-18T20:44:09.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.954+0000 7f674affd640 1 -- 172.21.15.60:0/1422692696 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f674c015590 con 0x7f675406c630 2023-11-18T20:44:09.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.954+0000 7f674affd640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f673806ae70 0x7f673806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:09.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.954+0000 7f674affd640 1 -- 172.21.15.60:0/1422692696 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f674c08a010 con 0x7f675406c630 2023-11-18T20:44:09.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.954+0000 7f675a0ef640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f673806ae70 0x7f673806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:09.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.955+0000 7f675a0ef640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f673806ae70 0x7f673806d330 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6750000fb0 tx=0x7f6750009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:09.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:09.958+0000 7f674affd640 1 -- 172.21.15.60:0/1422692696 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f674c054fc0 con 0x7f675406c630 2023-11-18T20:44:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.137+0000 7f675c353640 1 -- 172.21.15.60:0/1422692696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6754069400 con 0x7f675406c630 2023-11-18T20:44:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.137+0000 7f674affd640 1 -- 172.21.15.60:0/1422692696 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f674c0585e0 con 0x7f675406c630 2023-11-18T20:44:10.138 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 -- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f673806ae70 msgr2=0x7f673806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f673806ae70 0x7f673806d330 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6750000fb0 tx=0x7f6750009040 comp rx=0 tx=0).stop 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 -- 172.21.15.60:0/1422692696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f675406c630 msgr2=0x7f675406cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f675406c630 0x7f675406cab0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f674c011b40 tx=0x7f674c00f1b0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 -- 172.21.15.60:0/1422692696 shutdown_connections 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f673806ae70 0x7f673806d330 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f675406c630 0x7f675406cab0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67541585b0 0x7f67540ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 --2- 172.21.15.60:0/1422692696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67400047c0 0x7f675413d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 -- 172.21.15.60:0/1422692696 >> 172.21.15.60:0/1422692696 conn(0x7f675406a6b0 msgr2=0x7f67540d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:10.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 -- 172.21.15.60:0/1422692696 shutdown_connections 2023-11-18T20:44:10.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.139+0000 7f675c353640 1 -- 172.21.15.60:0/1422692696 wait complete. 2023-11-18T20:44:10.151 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-11-18T20:44:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.260+0000 7fa142f7c640 1 Processor -- start 2023-11-18T20:44:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.260+0000 7fa142f7c640 1 -- start start 2023-11-18T20:44:10.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.260+0000 7fa142f7c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa13c0d2910 0x7fa13c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.260+0000 7fa142f7c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c0b2b10 con 0x7fa13c0d3730 2023-11-18T20:44:10.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.260+0000 7fa142f7c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c0d1590 con 0x7fa13c15d330 2023-11-18T20:44:10.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.260+0000 7fa142f7c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13c0d3bc0 con 0x7fa13c0d2910 2023-11-18T20:44:10.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.261+0000 7fa13bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa13c0d2910 0x7fa13c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.261+0000 7fa141519640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa13c0d3730 0x7fa13c0b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47856/0 (socket says 172.21.15.60:47856) 2023-11-18T20:44:10.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.261+0000 7fa141519640 1 -- 172.21.15.60:0/358175724 learned_addr learned my addr 172.21.15.60:0/358175724 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:10.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.261+0000 7fa13bfff640 1 -- 172.21.15.60:0/358175724 >> v1:172.21.15.100:6789/0 conn(0x7fa13c15d330 legacy=0x7fa13c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.261+0000 7fa13bfff640 1 -- 172.21.15.60:0/358175724 >> v1:172.21.15.60:6789/0 conn(0x7fa13c0d3730 legacy=0x7fa13c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.261+0000 7fa13b7fe640 1 -- 172.21.15.60:0/358175724 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961363317 0 0) 0x7fa13c0b2b10 con 0x7fa13c0d3730 2023-11-18T20:44:10.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.261+0000 7fa13bfff640 1 -- 172.21.15.60:0/358175724 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c0d1870 con 0x7fa13c0d2910 2023-11-18T20:44:10.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa13bfff640 1 --2- 172.21.15.60:0/358175724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 0x7fa13c0d31f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa1240097a0 tx=0x7fa12402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c162f72a8b1988a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa13b7fe640 1 -- 172.21.15.60:0/358175724 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124031830 con 0x7fa13c0d2910 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa13b7fe640 1 -- 172.21.15.60:0/358175724 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa124031990 con 0x7fa13c0d2910 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa13b7fe640 1 -- 172.21.15.60:0/358175724 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124031d20 con 0x7fa13c0d2910 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa142f7c640 1 -- 172.21.15.60:0/358175724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 msgr2=0x7fa13c0d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa142f7c640 1 --2- 172.21.15.60:0/358175724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 0x7fa13c0d31f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa1240097a0 tx=0x7fa12402b8d0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa142f7c640 1 -- 172.21.15.60:0/358175724 shutdown_connections 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa142f7c640 1 --2- 172.21.15.60:0/358175724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 0x7fa13c0d31f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa142f7c640 1 -- 172.21.15.60:0/358175724 >> 172.21.15.60:0/358175724 conn(0x7fa13c072020 msgr2=0x7fa13c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa142f7c640 1 -- 172.21.15.60:0/358175724 shutdown_connections 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.262+0000 7fa142f7c640 1 -- 172.21.15.60:0/358175724 wait complete. 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.263+0000 7fa142f7c640 1 Processor -- start 2023-11-18T20:44:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.263+0000 7fa142f7c640 1 -- start start 2023-11-18T20:44:10.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.264+0000 7fa142f7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 0x7fa13c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.264+0000 7fa142f7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa13c0d3730 0x7fa13c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.264+0000 7fa142f7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa13c15d330 0x7fa13c15f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.264+0000 7fa142f7c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c15bdb0 con 0x7fa13c0d2910 2023-11-18T20:44:10.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.264+0000 7fa142f7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0cb4e0 con 0x7fa13c15d330 2023-11-18T20:44:10.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.264+0000 7fa142f7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0cb7c0 con 0x7fa13c0d3730 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.264+0000 7fa140d18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 0x7fa13c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa140d18640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 0x7fa13c15ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55156/0 (socket says 172.21.15.60:55156) 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa140d18640 1 -- 172.21.15.60:0/1625612460 learned_addr learned my addr 172.21.15.60:0/1625612460 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa13bfff640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa13c0d3730 0x7fa13c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa141519640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa13c15d330 0x7fa13c15f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa140d18640 1 -- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa13c0d3730 msgr2=0x7fa13c15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa140d18640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa13c0d3730 0x7fa13c15ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa140d18640 1 -- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa13c15d330 msgr2=0x7fa13c15f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa140d18640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa13c15d330 0x7fa13c15f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa140d18640 1 -- 172.21.15.60:0/1625612460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa124009400 con 0x7fa13c0d2910 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa140d18640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 0x7fa13c15ad50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa13400c730 tx=0x7fa13400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:10.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa1397fa640 1 -- 172.21.15.60:0/1625612460 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134003b70 con 0x7fa13c0d2910 2023-11-18T20:44:10.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa1397fa640 1 -- 172.21.15.60:0/1625612460 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa134003cf0 con 0x7fa13c0d2910 2023-11-18T20:44:10.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa142f7c640 1 -- 172.21.15.60:0/1625612460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c0cbaa0 con 0x7fa13c0d2910 2023-11-18T20:44:10.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa1397fa640 1 -- 172.21.15.60:0/1625612460 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134003ea0 con 0x7fa13c0d2910 2023-11-18T20:44:10.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.265+0000 7fa142f7c640 1 -- 172.21.15.60:0/1625612460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c0cc060 con 0x7fa13c0d2910 2023-11-18T20:44:10.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.266+0000 7fa142f7c640 1 -- 172.21.15.60:0/1625612460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c069400 con 0x7fa13c0d2910 2023-11-18T20:44:10.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.267+0000 7fa1397fa640 1 -- 172.21.15.60:0/1625612460 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa13400aa90 con 0x7fa13c0d2910 2023-11-18T20:44:10.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.267+0000 7fa1397fa640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa11c06ae40 0x7fa11c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.267+0000 7fa1397fa640 1 -- 172.21.15.60:0/1625612460 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fa134089640 con 0x7fa13c0d2910 2023-11-18T20:44:10.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.268+0000 7fa13bfff640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa11c06ae40 0x7fa11c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.268+0000 7fa13bfff640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa11c06ae40 0x7fa11c06d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa124036000 tx=0x7fa124008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:10.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.272+0000 7fa1397fa640 1 -- 172.21.15.60:0/1625612460 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa134054610 con 0x7fa13c0d2910 2023-11-18T20:44:10.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.446+0000 7fa142f7c640 1 -- 172.21.15.60:0/1625612460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fa13c072650 con 0x7fa13c0d2910 2023-11-18T20:44:10.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.447+0000 7fa1397fa640 1 -- 172.21.15.60:0/1625612460 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7fa134057c30 con 0x7fa13c0d2910 2023-11-18T20:44:10.448 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 -- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa11c06ae40 msgr2=0x7fa11c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa11c06ae40 0x7fa11c06d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa124036000 tx=0x7fa124008030 comp rx=0 tx=0).stop 2023-11-18T20:44:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 -- 172.21.15.60:0/1625612460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 msgr2=0x7fa13c15ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 0x7fa13c15ad50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa13400c730 tx=0x7fa13400cbf0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 -- 172.21.15.60:0/1625612460 shutdown_connections 2023-11-18T20:44:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa11c06ae40 0x7fa11c06d300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa13c15d330 0x7fa13c15f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa13c0d3730 0x7fa13c15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 --2- 172.21.15.60:0/1625612460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa13c0d2910 0x7fa13c15ad50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 -- 172.21.15.60:0/1625612460 >> 172.21.15.60:0/1625612460 conn(0x7fa13c072020 msgr2=0x7fa13c0b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:10.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 -- 172.21.15.60:0/1625612460 shutdown_connections 2023-11-18T20:44:10.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.449+0000 7fa142f7c640 1 -- 172.21.15.60:0/1625612460 wait complete. 2023-11-18T20:44:10.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:44:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.567+0000 7f4d56e90640 1 Processor -- start 2023-11-18T20:44:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.567+0000 7f4d56e90640 1 -- start start 2023-11-18T20:44:10.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d56e90640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4d500ad640 0x7f4d5006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d56e90640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d500adae0 con 0x7f4d5015d1d0 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d56e90640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d500addc0 con 0x7f4d5006bbf0 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d56e90640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d500ae0a0 con 0x7f4d500ad640 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d54c2c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4d5015d1d0 0x7f4d5006b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47862/0 (socket says 172.21.15.60:47862) 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d54c2c640 1 -- 172.21.15.60:0/1026585031 learned_addr learned my addr 172.21.15.60:0/1026585031 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296414546 0 0) 0x7f4d500adae0 con 0x7f4d5015d1d0 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d40003520 con 0x7f4d5015d1d0 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4ffff640 1 --2- 172.21.15.60:0/1026585031 >> v2:172.21.15.100:3300/0 conn(0x7f4d500ad640 0x7f4d5006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3136244762 0 0) 0x7f4d40003520 con 0x7f4d5015d1d0 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 >> v2:172.21.15.100:3300/0 conn(0x7f4d500ad640 msgr2=0x7f4d5006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 --2- 172.21.15.60:0/1026585031 >> v2:172.21.15.100:3300/0 conn(0x7f4d500ad640 0x7f4d5006d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 >> v1:172.21.15.100:6789/0 conn(0x7f4d5006bbf0 legacy=0x7f4d5006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d500ae380 con 0x7f4d5015d1d0 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f4d44001670 con 0x7f4d5015d1d0 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 >> v1:172.21.15.60:6789/0 conn(0x7f4d5015d1d0 legacy=0x7f4d5006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d40003970 0x7f4d40003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d40004370 0x7f4d40004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d40005040 0x7f4d4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.568+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500ae0a0 con 0x7f4d40003970 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4000bfa0 con 0x7f4d40005040 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4000c160 con 0x7f4d40004370 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d54c2c640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d40003970 0x7f4d40003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d4ffff640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d40004370 0x7f4d40004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d5542d640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d40005040 0x7f4d4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d54c2c640 1 -- 172.21.15.60:0/1026585031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d40004370 msgr2=0x7f4d40004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d54c2c640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d40004370 0x7f4d40004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d54c2c640 1 -- 172.21.15.60:0/1026585031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d40005040 msgr2=0x7f4d4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d54c2c640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d40005040 0x7f4d4000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d54c2c640 1 -- 172.21.15.60:0/1026585031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40003520 con 0x7f4d40003970 2023-11-18T20:44:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.569+0000 7f4d5542d640 1 -- 172.21.15.60:0/1026585031 reap_dead start 2023-11-18T20:44:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d54c2c640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d40003970 0x7f4d40003d50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c210 tx=0x7f4d400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7769db0bb10eb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4402cae0 con 0x7f4d40003970 2023-11-18T20:44:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4402cc60 con 0x7f4d40003970 2023-11-18T20:44:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d4f7fe640 1 -- 172.21.15.60:0/1026585031 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44005b10 con 0x7f4d40003970 2023-11-18T20:44:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d56e90640 1 -- 172.21.15.60:0/1026585031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d40003970 msgr2=0x7f4d40003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d56e90640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d40003970 0x7f4d40003d50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c210 tx=0x7f4d400057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d56e90640 1 -- 172.21.15.60:0/1026585031 shutdown_connections 2023-11-18T20:44:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d56e90640 1 --2- 172.21.15.60:0/1026585031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d40003970 0x7f4d40003d50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d56e90640 1 -- 172.21.15.60:0/1026585031 >> 172.21.15.60:0/1026585031 conn(0x7f4d50070e70 msgr2=0x7f4d50071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d56e90640 1 -- 172.21.15.60:0/1026585031 shutdown_connections 2023-11-18T20:44:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.570+0000 7f4d56e90640 1 -- 172.21.15.60:0/1026585031 wait complete. 2023-11-18T20:44:10.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.571+0000 7f4d56e90640 1 Processor -- start 2023-11-18T20:44:10.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.571+0000 7f4d56e90640 1 -- start start 2023-11-18T20:44:10.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.572+0000 7f4d56e90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d40003970 0x7f4d5015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.572+0000 7f4d56e90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d5015e790 0x7f4d501f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.572+0000 7f4d56e90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d50148030 0x7f4d501484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.572+0000 7f4d56e90640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5014a560 con 0x7f4d50148030 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.572+0000 7f4d56e90640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5014a800 con 0x7f4d40003970 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.572+0000 7f4d56e90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d501489f0 con 0x7f4d5015e790 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.572+0000 7f4d5542d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d50148030 0x7f4d501484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.572+0000 7f4d5542d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d50148030 0x7f4d501484b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55162/0 (socket says 172.21.15.60:55162) 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.572+0000 7f4d5542d640 1 -- 172.21.15.60:0/3860519822 learned_addr learned my addr 172.21.15.60:0/3860519822 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d54c2c640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d40003970 0x7f4d5015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d4ffff640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d5015e790 0x7f4d501f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d5542d640 1 -- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d5015e790 msgr2=0x7f4d501f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d5542d640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d5015e790 0x7f4d501f64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d5542d640 1 -- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d40003970 msgr2=0x7f4d5015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d5542d640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d40003970 0x7f4d5015c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d5542d640 1 -- 172.21.15.60:0/3860519822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5015f780 con 0x7f4d50148030 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d54c2c640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d40003970 0x7f4d5015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d5542d640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d50148030 0x7f4d501484b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4d50160e20 tx=0x7f4d4800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d4d7fa640 1 -- 172.21.15.60:0/3860519822 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48003970 con 0x7f4d50148030 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d4d7fa640 1 -- 172.21.15.60:0/3860519822 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d48003ad0 con 0x7f4d50148030 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d56e90640 1 -- 172.21.15.60:0/3860519822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5015eee0 con 0x7f4d50148030 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d4d7fa640 1 -- 172.21.15.60:0/3860519822 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48003e80 con 0x7f4d50148030 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.573+0000 7f4d56e90640 1 -- 172.21.15.60:0/3860519822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5015f4a0 con 0x7f4d50148030 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.574+0000 7f4d56e90640 1 -- 172.21.15.60:0/3860519822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5014a970 con 0x7f4d50148030 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.575+0000 7f4d4d7fa640 1 -- 172.21.15.60:0/3860519822 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d480050d0 con 0x7f4d50148030 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.575+0000 7f4d4d7fa640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d3406ae40 0x7f4d3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:10.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.575+0000 7f4d4d7fa640 1 -- 172.21.15.60:0/3860519822 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f4d480895c0 con 0x7f4d50148030 2023-11-18T20:44:10.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.575+0000 7f4d54c2c640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d3406ae40 0x7f4d3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:10.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.576+0000 7f4d54c2c640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d3406ae40 0x7f4d3406d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4d44034000 tx=0x7f4d44009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:10.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.579+0000 7f4d4d7fa640 1 -- 172.21.15.60:0/3860519822 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d5014a970 con 0x7f4d50148030 2023-11-18T20:44:10.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.767+0000 7f4d56e90640 1 -- 172.21.15.60:0/3860519822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4d5014a970 con 0x7f4d50148030 2023-11-18T20:44:10.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.767+0000 7f4d4d7fa640 1 -- 172.21.15.60:0/3860519822 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4d5014a970 con 0x7f4d50148030 2023-11-18T20:44:10.768 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:44:10.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 -- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d3406ae40 msgr2=0x7f4d3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d3406ae40 0x7f4d3406d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4d44034000 tx=0x7f4d44009030 comp rx=0 tx=0).stop 2023-11-18T20:44:10.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 -- 172.21.15.60:0/3860519822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d50148030 msgr2=0x7f4d501484b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:10.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d50148030 0x7f4d501484b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4d50160e20 tx=0x7f4d4800eb90 comp rx=0 tx=0).stop 2023-11-18T20:44:10.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 -- 172.21.15.60:0/3860519822 shutdown_connections 2023-11-18T20:44:10.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d3406ae40 0x7f4d3406d300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d50148030 0x7f4d501484b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d5015e790 0x7f4d501f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 --2- 172.21.15.60:0/3860519822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d40003970 0x7f4d5015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:10.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 -- 172.21.15.60:0/3860519822 >> 172.21.15.60:0/3860519822 conn(0x7f4d50070e70 msgr2=0x7f4d50160e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:10.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 -- 172.21.15.60:0/3860519822 shutdown_connections 2023-11-18T20:44:10.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:10.769+0000 7f4d56e90640 1 -- 172.21.15.60:0/3860519822 wait complete. 2023-11-18T20:44:10.781 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-11-18T20:44:10.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:10.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:10.810 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[103499]: fuse finished with error 0 and tester_r 0 2023-11-18T20:44:10.812 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:44:16.913 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:44:16.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:16.914 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-18T20:44:16.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2023-11-18T20:44:17.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.097+0000 7fbff6533640 1 Processor -- start 2023-11-18T20:44:17.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.097+0000 7fbff6533640 1 -- start start 2023-11-18T20:44:17.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbff6533640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbff015f8c0 0x7fbff00b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbff6533640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbff0144600 0x7fbff00b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbff6533640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff0141d10 con 0x7fbff0141930 2023-11-18T20:44:17.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbff6533640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff015fdc0 con 0x7fbff015f8c0 2023-11-18T20:44:17.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbff6533640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff00cd320 con 0x7fbff0144600 2023-11-18T20:44:17.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbfef7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbff015f8c0 0x7fbff00b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbfef7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbff015f8c0 0x7fbff00b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33392/0 (socket says 172.21.15.60:33392) 2023-11-18T20:44:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbfef7fe640 1 -- 172.21.15.60:0/2051168995 learned_addr learned my addr 172.21.15.60:0/2051168995 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbff4ad0640 1 --2- 172.21.15.60:0/2051168995 >> v2:172.21.15.100:3300/0 conn(0x7fbff0144600 0x7fbff00b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbfef7fe640 1 -- 172.21.15.60:0/2051168995 >> v2:172.21.15.100:3300/0 conn(0x7fbff0144600 msgr2=0x7fbff00b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbfef7fe640 1 --2- 172.21.15.60:0/2051168995 >> v2:172.21.15.100:3300/0 conn(0x7fbff0144600 0x7fbff00b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbfef7fe640 1 -- 172.21.15.60:0/2051168995 >> v1:172.21.15.100:6789/0 conn(0x7fbff0141930 legacy=0x7fbff00ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:17.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.098+0000 7fbfef7fe640 1 -- 172.21.15.60:0/2051168995 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00cd600 con 0x7fbff015f8c0 2023-11-18T20:44:17.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.099+0000 7fbfef7fe640 1 --2- 172.21.15.60:0/2051168995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015f8c0 0x7fbff00b0bf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbff00ccaa0 tx=0x7fbfe402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e036d22cde4dfd server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:17.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.099+0000 7fbfeeffd640 1 -- 172.21.15.60:0/2051168995 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4031750 con 0x7fbff015f8c0 2023-11-18T20:44:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.099+0000 7fbfeeffd640 1 -- 172.21.15.60:0/2051168995 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe40318b0 con 0x7fbff015f8c0 2023-11-18T20:44:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.099+0000 7fbfeeffd640 1 -- 172.21.15.60:0/2051168995 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4031c40 con 0x7fbff015f8c0 2023-11-18T20:44:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.100+0000 7fbff6533640 1 -- 172.21.15.60:0/2051168995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015f8c0 msgr2=0x7fbff00b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.100+0000 7fbff6533640 1 --2- 172.21.15.60:0/2051168995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015f8c0 0x7fbff00b0bf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbff00ccaa0 tx=0x7fbfe402b7f0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.100+0000 7fbff6533640 1 -- 172.21.15.60:0/2051168995 shutdown_connections 2023-11-18T20:44:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.100+0000 7fbff6533640 1 --2- 172.21.15.60:0/2051168995 >> v2:172.21.15.100:3300/0 conn(0x7fbff0144600 0x7fbff00b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.100+0000 7fbff6533640 1 --2- 172.21.15.60:0/2051168995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff015f8c0 0x7fbff00b0bf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.100+0000 7fbff6533640 1 -- 172.21.15.60:0/2051168995 >> 172.21.15.60:0/2051168995 conn(0x7fbff006d9c0 msgr2=0x7fbff006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:17.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.100+0000 7fbff6533640 1 -- 172.21.15.60:0/2051168995 shutdown_connections 2023-11-18T20:44:17.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.100+0000 7fbff6533640 1 -- 172.21.15.60:0/2051168995 wait complete. 2023-11-18T20:44:17.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.101+0000 7fbff6533640 1 Processor -- start 2023-11-18T20:44:17.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.101+0000 7fbff6533640 1 -- start start 2023-11-18T20:44:17.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbff6533640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0141930 0x7fbff015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbff6533640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0144600 0x7fbff00ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbff6533640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff015f8c0 0x7fbff00ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbff6533640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00ad5e0 con 0x7fbff0144600 2023-11-18T20:44:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbff6533640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff015c210 con 0x7fbff015f8c0 2023-11-18T20:44:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbff6533640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00acd40 con 0x7fbff0141930 2023-11-18T20:44:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbfef7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0144600 0x7fbff00ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbfef7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0144600 0x7fbff00ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33396/0 (socket says 172.21.15.60:33396) 2023-11-18T20:44:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbfef7fe640 1 -- 172.21.15.60:0/3886527863 learned_addr learned my addr 172.21.15.60:0/3886527863 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbff4ad0640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff015f8c0 0x7fbff00ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbfeffff640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0141930 0x7fbff015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbfef7fe640 1 -- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0141930 msgr2=0x7fbff015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbfef7fe640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0141930 0x7fbff015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbfef7fe640 1 -- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff015f8c0 msgr2=0x7fbff00ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbfef7fe640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff015f8c0 0x7fbff00ac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.102+0000 7fbfef7fe640 1 -- 172.21.15.60:0/3886527863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40094d0 con 0x7fbff0144600 2023-11-18T20:44:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.103+0000 7fbfef7fe640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0144600 0x7fbff00ac180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbfe402b830 tx=0x7fbfe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.103+0000 7fbfecff9640 1 -- 172.21.15.60:0/3886527863 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4033c80 con 0x7fbff0144600 2023-11-18T20:44:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.103+0000 7fbfecff9640 1 -- 172.21.15.60:0/3886527863 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4033de0 con 0x7fbff0144600 2023-11-18T20:44:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.103+0000 7fbff6533640 1 -- 172.21.15.60:0/3886527863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff00ad020 con 0x7fbff0144600 2023-11-18T20:44:17.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.103+0000 7fbfecff9640 1 -- 172.21.15.60:0/3886527863 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe403c620 con 0x7fbff0144600 2023-11-18T20:44:17.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.103+0000 7fbff6533640 1 -- 172.21.15.60:0/3886527863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff00a8600 con 0x7fbff0144600 2023-11-18T20:44:17.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.104+0000 7fbff6533640 1 -- 172.21.15.60:0/3886527863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff0069400 con 0x7fbff0144600 2023-11-18T20:44:17.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.104+0000 7fbfecff9640 1 -- 172.21.15.60:0/3886527863 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbfe403a040 con 0x7fbff0144600 2023-11-18T20:44:17.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.105+0000 7fbfecff9640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbfdc06ae40 0x7fbfdc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.105+0000 7fbfecff9640 1 -- 172.21.15.60:0/3886527863 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fbfe40ab380 con 0x7fbff0144600 2023-11-18T20:44:17.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.105+0000 7fbfeffff640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbfdc06ae40 0x7fbfdc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.105+0000 7fbfeffff640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbfdc06ae40 0x7fbfdc06d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0006fd0 tx=0x7fbfe0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.109+0000 7fbfecff9640 1 -- 172.21.15.60:0/3886527863 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbfe4076330 con 0x7fbff0144600 2023-11-18T20:44:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.353+0000 7fbff6533640 1 -- 172.21.15.60:0/3886527863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbff00cc6d0 con 0x7fbff0144600 2023-11-18T20:44:17.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.355+0000 7fbfecff9640 1 -- 172.21.15.60:0/3886527863 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+141 (secure 0 0 0) 0x7fbfe4079950 con 0x7fbff0144600 2023-11-18T20:44:17.357 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-11-18T20:44:17.357 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA== 2023-11-18T20:44:17.357 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-11-18T20:44:17.357 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-11-18T20:44:17.357 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw pool=cephfs_data" 2023-11-18T20:44:17.357 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-11-18T20:44:17.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.357+0000 7fbff6533640 1 -- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbfdc06ae40 msgr2=0x7fbfdc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:17.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.357+0000 7fbff6533640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbfdc06ae40 0x7fbfdc06d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0006fd0 tx=0x7fbfe0008040 comp rx=0 tx=0).stop 2023-11-18T20:44:17.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.357+0000 7fbff6533640 1 -- 172.21.15.60:0/3886527863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0144600 msgr2=0x7fbff00ac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:17.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.357+0000 7fbff6533640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0144600 0x7fbff00ac180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbfe402b830 tx=0x7fbfe40046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.357+0000 7fbff6533640 1 -- 172.21.15.60:0/3886527863 shutdown_connections 2023-11-18T20:44:17.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.357+0000 7fbff6533640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbfdc06ae40 0x7fbfdc06d300 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.357+0000 7fbff6533640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff015f8c0 0x7fbff00ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.357+0000 7fbff6533640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0144600 0x7fbff00ac180 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.357+0000 7fbff6533640 1 --2- 172.21.15.60:0/3886527863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0141930 0x7fbff015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.357+0000 7fbff6533640 1 -- 172.21.15.60:0/3886527863 >> 172.21.15.60:0/3886527863 conn(0x7fbff006d9c0 msgr2=0x7fbff015ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:17.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.358+0000 7fbff6533640 1 -- 172.21.15.60:0/3886527863 shutdown_connections 2023-11-18T20:44:17.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.358+0000 7fbff6533640 1 -- 172.21.15.60:0/3886527863 wait complete. 2023-11-18T20:44:17.369 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-18T20:44:17.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-18T20:44:17.412 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-18T20:44:17.465 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-18T20:44:17.466 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:17.466 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:17.481 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=344 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249488k,nr_inodes=812372,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-18T20:44:17.481 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:44:17.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:17.481 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-18T20:44:17.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:44:17.535 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-18T20:44:17.536 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:44:17.536 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:17.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056339,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=480 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-18T20:44:17.553 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:44:17.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:17.553 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-18T20:44:17.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:44:17.609 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-18T20:44:17.609 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:17.609 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:17.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=344 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249488k,nr_inodes=812372,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-18T20:44:17.625 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:44:17.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:17.625 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-18T20:44:17.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:44:17.681 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-18T20:44:17.681 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:44:17.681 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:17.698 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056339,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=480 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-18T20:44:17.698 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:44:17.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:17.699 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-18T20:44:17.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:44:17.755 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-18T20:44:17.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:44:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.861+0000 7f15c7b08640 1 Processor -- start 2023-11-18T20:44:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.861+0000 7f15c7b08640 1 -- start start 2023-11-18T20:44:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.862+0000 7f15c7b08640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15c0150560 0x7f15c00b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.862+0000 7f15c7b08640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c0150940 0x7f15c00ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.862+0000 7f15c7b08640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c00b49f0 con 0x7f15c00b3390 2023-11-18T20:44:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.862+0000 7f15c7b08640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c00b3de0 con 0x7f15c0150560 2023-11-18T20:44:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.862+0000 7f15c7b08640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00b40c0 con 0x7f15c0150940 2023-11-18T20:44:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.862+0000 7f15c60a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c0150940 0x7f15c00ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c58a4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15c0150560 0x7f15c00b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c50a3640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f15c00b3390 0x7f15c015d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:44340/0 (socket says 172.21.15.60:44340) 2023-11-18T20:44:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c50a3640 1 -- 172.21.15.60:0/1932464091 learned_addr learned my addr 172.21.15.60:0/1932464091 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c60a5640 1 --2- 172.21.15.60:0/1932464091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c0150940 0x7f15c00ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:48004/0 (socket says 172.21.15.60:48004) 2023-11-18T20:44:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c48a2640 1 -- 172.21.15.60:0/1932464091 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863507846 0 0) 0x7f15c00b49f0 con 0x7f15c00b3390 2023-11-18T20:44:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c48a2640 1 -- 172.21.15.60:0/1932464091 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15a8003520 con 0x7f15c00b3390 2023-11-18T20:44:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c60a5640 1 -- 172.21.15.60:0/1932464091 >> v2:172.21.15.100:3300/0 conn(0x7f15c0150560 msgr2=0x7f15c00b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c60a5640 1 --2- 172.21.15.60:0/1932464091 >> v2:172.21.15.100:3300/0 conn(0x7f15c0150560 0x7f15c00b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c60a5640 1 -- 172.21.15.60:0/1932464091 >> v1:172.21.15.100:6789/0 conn(0x7f15c00b3390 legacy=0x7f15c015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c60a5640 1 -- 172.21.15.60:0/1932464091 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c00b43a0 con 0x7f15c0150940 2023-11-18T20:44:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.863+0000 7f15c58a4640 1 --2- 172.21.15.60:0/1932464091 >> v2:172.21.15.100:3300/0 conn(0x7f15c0150560 0x7f15c00b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.864+0000 7f15c60a5640 1 --2- 172.21.15.60:0/1932464091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c0150940 0x7f15c00ce0e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009d90 tx=0x7f15b002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb6e02a25929ae1a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.864+0000 7f15c48a2640 1 -- 172.21.15.60:0/1932464091 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b002cc60 con 0x7f15c0150940 2023-11-18T20:44:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.864+0000 7f15c48a2640 1 -- 172.21.15.60:0/1932464091 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b002cdc0 con 0x7f15c0150940 2023-11-18T20:44:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.864+0000 7f15c48a2640 1 -- 172.21.15.60:0/1932464091 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0034640 con 0x7f15c0150940 2023-11-18T20:44:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.864+0000 7f15c7b08640 1 -- 172.21.15.60:0/1932464091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c0150940 msgr2=0x7f15c00ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.864+0000 7f15c7b08640 1 --2- 172.21.15.60:0/1932464091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c0150940 0x7f15c00ce0e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009d90 tx=0x7f15b002beb0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.864+0000 7f15c7b08640 1 -- 172.21.15.60:0/1932464091 shutdown_connections 2023-11-18T20:44:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.864+0000 7f15c7b08640 1 --2- 172.21.15.60:0/1932464091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c0150940 0x7f15c00ce0e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.864+0000 7f15c7b08640 1 --2- 172.21.15.60:0/1932464091 >> v2:172.21.15.100:3300/0 conn(0x7f15c0150560 0x7f15c00b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.864+0000 7f15c7b08640 1 -- 172.21.15.60:0/1932464091 >> 172.21.15.60:0/1932464091 conn(0x7f15c006a4d0 msgr2=0x7f15c006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.865+0000 7f15c7b08640 1 -- 172.21.15.60:0/1932464091 shutdown_connections 2023-11-18T20:44:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.865+0000 7f15c7b08640 1 -- 172.21.15.60:0/1932464091 wait complete. 2023-11-18T20:44:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.866+0000 7f15c7b08640 1 Processor -- start 2023-11-18T20:44:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.866+0000 7f15c7b08640 1 -- start start 2023-11-18T20:44:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.866+0000 7f15c7b08640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c00b3390 0x7f15c0158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.866+0000 7f15c7b08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15c0150560 0x7f15c014db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.866+0000 7f15c7b08640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c014e0c0 0x7f15c00ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.866+0000 7f15c7b08640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0158f90 con 0x7f15c0150560 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.866+0000 7f15c7b08640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c014e6d0 con 0x7f15c00b3390 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.866+0000 7f15c7b08640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c014e9b0 con 0x7f15c014e0c0 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c50a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15c0150560 0x7f15c014db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c50a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15c0150560 0x7f15c014db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33408/0 (socket says 172.21.15.60:33408) 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c50a3640 1 -- 172.21.15.60:0/3337906407 learned_addr learned my addr 172.21.15.60:0/3337906407 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c58a4640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c00b3390 0x7f15c0158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c60a5640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c014e0c0 0x7f15c00ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c50a3640 1 -- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c014e0c0 msgr2=0x7f15c00ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c50a3640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c014e0c0 0x7f15c00ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c50a3640 1 -- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c00b3390 msgr2=0x7f15c0158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:17.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c50a3640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c00b3390 0x7f15c0158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c50a3640 1 -- 172.21.15.60:0/3337906407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b8009400 con 0x7f15c0150560 2023-11-18T20:44:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c50a3640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15c0150560 0x7f15c014db80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f15b400c430 tx=0x7f15b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15be7fc640 1 -- 172.21.15.60:0/3337906407 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4005ce0 con 0x7f15c0150560 2023-11-18T20:44:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15be7fc640 1 -- 172.21.15.60:0/3337906407 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b4005e40 con 0x7f15c0150560 2023-11-18T20:44:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c7b08640 1 -- 172.21.15.60:0/3337906407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b0009a70 con 0x7f15c0150560 2023-11-18T20:44:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15be7fc640 1 -- 172.21.15.60:0/3337906407 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b40072f0 con 0x7f15c0150560 2023-11-18T20:44:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.867+0000 7f15c7b08640 1 -- 172.21.15.60:0/3337906407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c014ef70 con 0x7f15c0150560 2023-11-18T20:44:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.868+0000 7f15c7b08640 1 -- 172.21.15.60:0/3337906407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c0069400 con 0x7f15c0150560 2023-11-18T20:44:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.869+0000 7f15be7fc640 1 -- 172.21.15.60:0/3337906407 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15b4007450 con 0x7f15c0150560 2023-11-18T20:44:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.869+0000 7f15be7fc640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15ac06ae70 0x7f15ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.869+0000 7f15be7fc640 1 -- 172.21.15.60:0/3337906407 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f15b4088d80 con 0x7f15c0150560 2023-11-18T20:44:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.869+0000 7f15c58a4640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15ac06ae70 0x7f15ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.870+0000 7f15c58a4640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15ac06ae70 0x7f15ac06d330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009cd0 tx=0x7f15b8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:17.873+0000 7f15be7fc640 1 -- 172.21.15.60:0/3337906407 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15b4053d40 con 0x7f15c0150560 2023-11-18T20:44:18.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.083+0000 7f15c7b08640 1 -- 172.21.15.60:0/3337906407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15c00b5200 con 0x7f15c0150560 2023-11-18T20:44:18.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.084+0000 7f15be7fc640 1 -- 172.21.15.60:0/3337906407 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f15b4017020 con 0x7f15c0150560 2023-11-18T20:44:18.085 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:18.085 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4814,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3885363174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3885363174},{"type":"v1","addr":"172.21.15.100:6837","nonce":3885363174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1102236552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1102236552},{"type":"v1","addr":"172.21.15.60:6835","nonce":1102236552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2680685354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2680685354},{"type":"v1","addr":"172.21.15.100:6836","nonce":2680685354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:43:33.146258+0000","modified":"2023-11-18T20:43:41.162003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5065":{"gid":5065,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3762312847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3762312847},{"type":"v1","addr":"172.21.15.60:6834","nonce":3762312847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-18T20:44:18.086 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 14 2023-11-18T20:44:18.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 -- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15ac06ae70 msgr2=0x7f15ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15ac06ae70 0x7f15ac06d330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009cd0 tx=0x7f15b8009190 comp rx=0 tx=0).stop 2023-11-18T20:44:18.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 -- 172.21.15.60:0/3337906407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15c0150560 msgr2=0x7f15c014db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15c0150560 0x7f15c014db80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f15b400c430 tx=0x7f15b400c8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 -- 172.21.15.60:0/3337906407 shutdown_connections 2023-11-18T20:44:18.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15ac06ae70 0x7f15ac06d330 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15c014e0c0 0x7f15c00ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15c0150560 0x7f15c014db80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 --2- 172.21.15.60:0/3337906407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15c00b3390 0x7f15c0158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 -- 172.21.15.60:0/3337906407 >> 172.21.15.60:0/3337906407 conn(0x7f15c006a4d0 msgr2=0x7f15c014f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:18.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 -- 172.21.15.60:0/3337906407 shutdown_connections 2023-11-18T20:44:18.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.086+0000 7f15c7b08640 1 -- 172.21.15.60:0/3337906407 wait complete. 2023-11-18T20:44:18.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:18.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.203+0000 7fe668644640 1 Processor -- start 2023-11-18T20:44:18.204 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.203+0000 7fe668644640 1 -- start start 2023-11-18T20:44:18.205 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.204+0000 7fe668644640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe66015d1d0 0x7fe66006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.204+0000 7fe668644640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6600ad640 0x7fe66006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.204+0000 7fe668644640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6600adae0 con 0x7fe6600aeb80 2023-11-18T20:44:18.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.204+0000 7fe668644640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6600addc0 con 0x7fe6600ad640 2023-11-18T20:44:18.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.204+0000 7fe668644640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600ae0a0 con 0x7fe66015d1d0 2023-11-18T20:44:18.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.204+0000 7fe6663e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe66015d1d0 0x7fe66006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.205+0000 7fe665bdf640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6600ad640 0x7fe66006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.205+0000 7fe666be1640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe6600aeb80 0x7fe660160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:44354/0 (socket says 172.21.15.60:44354) 2023-11-18T20:44:18.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.205+0000 7fe6663e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe66015d1d0 0x7fe66006b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:48018/0 (socket says 172.21.15.60:48018) 2023-11-18T20:44:18.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.205+0000 7fe666be1640 1 -- 172.21.15.60:0/4167395954 learned_addr learned my addr 172.21.15.60:0/4167395954 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:18.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.205+0000 7fe6663e0640 1 -- 172.21.15.60:0/4167395954 >> v2:172.21.15.100:3300/0 conn(0x7fe6600ad640 msgr2=0x7fe66006ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:18.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.205+0000 7fe6663e0640 1 --2- 172.21.15.60:0/4167395954 >> v2:172.21.15.100:3300/0 conn(0x7fe6600ad640 0x7fe66006ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.205+0000 7fe6663e0640 1 -- 172.21.15.60:0/4167395954 >> v1:172.21.15.100:6789/0 conn(0x7fe6600aeb80 legacy=0x7fe660160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.205+0000 7fe6663e0640 1 -- 172.21.15.60:0/4167395954 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600ae380 con 0x7fe66015d1d0 2023-11-18T20:44:18.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.205+0000 7fe665bdf640 1 --2- 172.21.15.60:0/4167395954 >> v2:172.21.15.100:3300/0 conn(0x7fe6600ad640 0x7fe66006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:18.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.206+0000 7fe6663e0640 1 --2- 172.21.15.60:0/4167395954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe66015d1d0 0x7fe66006b4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe650009d90 tx=0x7fe65002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9487f2508076a87 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:18.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.206+0000 7fe6653de640 1 -- 172.21.15.60:0/4167395954 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65002cc50 con 0x7fe66015d1d0 2023-11-18T20:44:18.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.206+0000 7fe6653de640 1 -- 172.21.15.60:0/4167395954 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65002cdb0 con 0x7fe66015d1d0 2023-11-18T20:44:18.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.206+0000 7fe6653de640 1 -- 172.21.15.60:0/4167395954 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650034660 con 0x7fe66015d1d0 2023-11-18T20:44:18.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.206+0000 7fe668644640 1 -- 172.21.15.60:0/4167395954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe66015d1d0 msgr2=0x7fe66006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.206+0000 7fe668644640 1 --2- 172.21.15.60:0/4167395954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe66015d1d0 0x7fe66006b4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe650009d90 tx=0x7fe65002bea0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.207+0000 7fe668644640 1 -- 172.21.15.60:0/4167395954 shutdown_connections 2023-11-18T20:44:18.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.207+0000 7fe668644640 1 --2- 172.21.15.60:0/4167395954 >> v2:172.21.15.100:3300/0 conn(0x7fe6600ad640 0x7fe66006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.207+0000 7fe668644640 1 --2- 172.21.15.60:0/4167395954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe66015d1d0 0x7fe66006b4e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.207+0000 7fe668644640 1 -- 172.21.15.60:0/4167395954 >> 172.21.15.60:0/4167395954 conn(0x7fe660070e70 msgr2=0x7fe660071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:18.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.207+0000 7fe668644640 1 -- 172.21.15.60:0/4167395954 shutdown_connections 2023-11-18T20:44:18.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.207+0000 7fe668644640 1 -- 172.21.15.60:0/4167395954 wait complete. 2023-11-18T20:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.208+0000 7fe668644640 1 Processor -- start 2023-11-18T20:44:18.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.208+0000 7fe668644640 1 -- start start 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.208+0000 7fe668644640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6600ad640 0x7fe660147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.208+0000 7fe668644640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6600aeb80 0x7fe6600cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.208+0000 7fe668644640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe66015d1d0 0x7fe66006e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.208+0000 7fe668644640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660147890 con 0x7fe66015d1d0 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe668644640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660150790 con 0x7fe6600aeb80 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe668644640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600c4a80 con 0x7fe6600ad640 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe666be1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe66015d1d0 0x7fe66006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe6663e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6600ad640 0x7fe660147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe665bdf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6600aeb80 0x7fe6600cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe6663e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6600ad640 0x7fe660147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:48030/0 (socket says 172.21.15.60:48030) 2023-11-18T20:44:18.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe666be1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe66015d1d0 0x7fe66006e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33416/0 (socket says 172.21.15.60:33416) 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe6663e0640 1 -- 172.21.15.60:0/1103863687 learned_addr learned my addr 172.21.15.60:0/1103863687 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe6663e0640 1 -- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6600aeb80 msgr2=0x7fe6600cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe6663e0640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6600aeb80 0x7fe6600cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe6663e0640 1 -- 172.21.15.60:0/1103863687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe66015d1d0 msgr2=0x7fe66006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe6663e0640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe66015d1d0 0x7fe66006e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe6663e0640 1 -- 172.21.15.60:0/1103863687 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe654009400 con 0x7fe6600ad640 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe6663e0640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6600ad640 0x7fe660147350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe650009d90 tx=0x7fe650004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.209+0000 7fe65effd640 1 -- 172.21.15.60:0/1103863687 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650040650 con 0x7fe6600ad640 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.210+0000 7fe65effd640 1 -- 172.21.15.60:0/1103863687 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6500407b0 con 0x7fe6600ad640 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.210+0000 7fe668644640 1 -- 172.21.15.60:0/1103863687 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe650009a70 con 0x7fe6600ad640 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.210+0000 7fe65effd640 1 -- 172.21.15.60:0/1103863687 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650040b60 con 0x7fe6600ad640 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.210+0000 7fe668644640 1 -- 172.21.15.60:0/1103863687 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6600c5040 con 0x7fe6600ad640 2023-11-18T20:44:18.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.211+0000 7fe668644640 1 -- 172.21.15.60:0/1103863687 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe660069400 con 0x7fe6600ad640 2023-11-18T20:44:18.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.211+0000 7fe65effd640 1 -- 172.21.15.60:0/1103863687 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe650040cc0 con 0x7fe6600ad640 2023-11-18T20:44:18.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.211+0000 7fe65effd640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe64c06ae40 0x7fe64c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.212+0000 7fe65effd640 1 -- 172.21.15.60:0/1103863687 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe6500ac420 con 0x7fe6600ad640 2023-11-18T20:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.212+0000 7fe665bdf640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe64c06ae40 0x7fe64c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.212+0000 7fe665bdf640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe64c06ae40 0x7fe64c06d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe660151150 tx=0x7fe654009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:18.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.216+0000 7fe65effd640 1 -- 172.21.15.60:0/1103863687 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6500775a0 con 0x7fe6600ad640 2023-11-18T20:44:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.392+0000 7fe668644640 1 -- 172.21.15.60:0/1103863687 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe660151180 con 0x7fe6600ad640 2023-11-18T20:44:18.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.394+0000 7fe65effd640 1 -- 172.21.15.60:0/1103863687 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18336 (secure 0 0 0) 0x7fe65007abc0 con 0x7fe6600ad640 2023-11-18T20:44:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:18.396 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:42.814654+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-11-18T20:43:40.650315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:18.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 -- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe64c06ae40 msgr2=0x7fe64c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe64c06ae40 0x7fe64c06d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe660151150 tx=0x7fe654009190 comp rx=0 tx=0).stop 2023-11-18T20:44:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 -- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6600ad640 msgr2=0x7fe660147350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6600ad640 0x7fe660147350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe650009d90 tx=0x7fe650004a40 comp rx=0 tx=0).stop 2023-11-18T20:44:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 -- 172.21.15.60:0/1103863687 shutdown_connections 2023-11-18T20:44:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe64c06ae40 0x7fe64c06d300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe66015d1d0 0x7fe66006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6600aeb80 0x7fe6600cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 --2- 172.21.15.60:0/1103863687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6600ad640 0x7fe660147350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 -- 172.21.15.60:0/1103863687 >> 172.21.15.60:0/1103863687 conn(0x7fe660070e70 msgr2=0x7fe66015fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 -- 172.21.15.60:0/1103863687 shutdown_connections 2023-11-18T20:44:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.396+0000 7fe668644640 1 -- 172.21.15.60:0/1103863687 wait complete. 2023-11-18T20:44:18.408 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:43:31.703619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:43:32.121610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2023-11-18T20:43:40.650315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:44:18.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:44:18.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.517+0000 7f3b08851640 1 Processor -- start 2023-11-18T20:44:18.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.517+0000 7f3b08851640 1 -- start start 2023-11-18T20:44:18.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.517+0000 7f3b08851640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3b040d2910 0x7f3b040d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.517+0000 7f3b08851640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b040b2b10 con 0x7f3b040d3730 2023-11-18T20:44:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.517+0000 7f3b08851640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b040d1590 con 0x7f3b0415d330 2023-11-18T20:44:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.517+0000 7f3b08851640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b040d3bc0 con 0x7f3b040d2910 2023-11-18T20:44:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b02d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3b040d3730 0x7f3b040b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55302/0 (socket says 172.21.15.60:55302) 2023-11-18T20:44:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b02d9d640 1 -- 172.21.15.60:0/1766540209 learned_addr learned my addr 172.21.15.60:0/1766540209 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3afbfff640 1 --2- 172.21.15.60:0/1766540209 >> v2:172.21.15.100:3300/0 conn(0x7f3b040d2910 0x7f3b040d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225422506 0 0) 0x7f3b040b2b10 con 0x7f3b040d3730 2023-11-18T20:44:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3af0003520 con 0x7f3b040d3730 2023-11-18T20:44:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 252216070 0 0) 0x7f3af0003520 con 0x7f3b040d3730 2023-11-18T20:44:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 >> v2:172.21.15.100:3300/0 conn(0x7f3b040d2910 msgr2=0x7f3b040d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 --2- 172.21.15.60:0/1766540209 >> v2:172.21.15.100:3300/0 conn(0x7f3b040d2910 0x7f3b040d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 >> v1:172.21.15.100:6789/0 conn(0x7f3b0415d330 legacy=0x7f3b040b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b040d1870 con 0x7f3b040d3730 2023-11-18T20:44:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f3afc0019f0 con 0x7f3b040d3730 2023-11-18T20:44:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 >> v1:172.21.15.60:6789/0 conn(0x7f3b040d3730 legacy=0x7f3b040b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3af0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3af0005180 0x7f3af000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040d3bc0 con 0x7f3af00047c0 2023-11-18T20:44:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af000c120 con 0x7f3af0003970 2023-11-18T20:44:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af000c2e0 con 0x7f3af0005180 2023-11-18T20:44:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3afbfff640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3af0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b0259c640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.518+0000 7f3b02d9d640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3af0005180 0x7f3af000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.519+0000 7f3afbfff640 1 -- 172.21.15.60:0/1766540209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3af0005180 msgr2=0x7f3af000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.519+0000 7f3afbfff640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3af0005180 0x7f3af000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.519+0000 7f3afbfff640 1 -- 172.21.15.60:0/1766540209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0003970 msgr2=0x7f3af0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.519+0000 7f3afbfff640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.519+0000 7f3b02d9d640 1 -- 172.21.15.60:0/1766540209 reap_dead start 2023-11-18T20:44:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.519+0000 7f3afbfff640 1 -- 172.21.15.60:0/1766540209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af0003520 con 0x7f3af00047c0 2023-11-18T20:44:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.519+0000 7f3afbfff640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3af0004c40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009ac0 tx=0x7f3af00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe072b05191a16cc server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.520+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec02c6b0 con 0x7f3af00047c0 2023-11-18T20:44:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.520+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec02c810 con 0x7f3af00047c0 2023-11-18T20:44:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.520+0000 7f3b01d9b640 1 -- 172.21.15.60:0/1766540209 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec02cba0 con 0x7f3af00047c0 2023-11-18T20:44:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.520+0000 7f3b08851640 1 -- 172.21.15.60:0/1766540209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 msgr2=0x7f3af0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.520+0000 7f3b08851640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3af0004c40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009ac0 tx=0x7f3af00057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.520+0000 7f3b08851640 1 -- 172.21.15.60:0/1766540209 shutdown_connections 2023-11-18T20:44:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.520+0000 7f3b08851640 1 --2- 172.21.15.60:0/1766540209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3af0004c40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.520+0000 7f3b08851640 1 -- 172.21.15.60:0/1766540209 >> 172.21.15.60:0/1766540209 conn(0x7f3b04072020 msgr2=0x7f3b04072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.520+0000 7f3b08851640 1 -- 172.21.15.60:0/1766540209 shutdown_connections 2023-11-18T20:44:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.520+0000 7f3b08851640 1 -- 172.21.15.60:0/1766540209 wait complete. 2023-11-18T20:44:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.521+0000 7f3b08851640 1 Processor -- start 2023-11-18T20:44:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.522+0000 7f3b08851640 1 -- start start 2023-11-18T20:44:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.522+0000 7f3b08851640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3b040cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.522+0000 7f3b08851640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b04153880 0x7f3b041438d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.522+0000 7f3b08851640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b04153d00 0x7f3b04143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.522+0000 7f3b08851640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040cc170 con 0x7f3af00047c0 2023-11-18T20:44:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.522+0000 7f3b08851640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040cc450 con 0x7f3b04153d00 2023-11-18T20:44:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.522+0000 7f3b08851640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040cc730 con 0x7f3b04153880 2023-11-18T20:44:18.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.522+0000 7f3b0259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3b040cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.522+0000 7f3b0259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3b040cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33438/0 (socket says 172.21.15.60:33438) 2023-11-18T20:44:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.522+0000 7f3b0259c640 1 -- 172.21.15.60:0/506533665 learned_addr learned my addr 172.21.15.60:0/506533665 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3b02d9d640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b04153d00 0x7f3b04143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3afbfff640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b04153880 0x7f3b041438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3b0259c640 1 -- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b04153880 msgr2=0x7f3b041438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3b0259c640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b04153880 0x7f3b041438d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3b0259c640 1 -- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b04153d00 msgr2=0x7f3b04143e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3b0259c640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b04153d00 0x7f3b04143e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3b0259c640 1 -- 172.21.15.60:0/506533665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b040cca10 con 0x7f3af00047c0 2023-11-18T20:44:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3b02d9d640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b04153d00 0x7f3b04143e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3afbfff640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b04153880 0x7f3b041438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3b0259c640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3b040cb920 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3af4007810 tx=0x7f3af400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3afaffd640 1 -- 172.21.15.60:0/506533665 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af40050d0 con 0x7f3af00047c0 2023-11-18T20:44:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3afaffd640 1 -- 172.21.15.60:0/506533665 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af4005230 con 0x7f3af00047c0 2023-11-18T20:44:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3afaffd640 1 -- 172.21.15.60:0/506533665 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af400d730 con 0x7f3af00047c0 2023-11-18T20:44:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3b08851640 1 -- 172.21.15.60:0/506533665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b040cccf0 con 0x7f3af00047c0 2023-11-18T20:44:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.523+0000 7f3b08851640 1 -- 172.21.15.60:0/506533665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b040cbe60 con 0x7f3af00047c0 2023-11-18T20:44:18.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.524+0000 7f3b08851640 1 -- 172.21.15.60:0/506533665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b04069400 con 0x7f3af00047c0 2023-11-18T20:44:18.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.524+0000 7f3afaffd640 1 -- 172.21.15.60:0/506533665 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3af400d890 con 0x7f3af00047c0 2023-11-18T20:44:18.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.525+0000 7f3afaffd640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3ae406ae40 0x7f3ae406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.525+0000 7f3afaffd640 1 -- 172.21.15.60:0/506533665 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f3af40896f0 con 0x7f3af00047c0 2023-11-18T20:44:18.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.525+0000 7f3afbfff640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3ae406ae40 0x7f3ae406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.526+0000 7f3afbfff640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3ae406ae40 0x7f3ae406d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3aec036000 tx=0x7f3aec030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:18.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.530+0000 7f3afaffd640 1 -- 172.21.15.60:0/506533665 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3af4054570 con 0x7f3af00047c0 2023-11-18T20:44:18.707 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.706+0000 7f3b08851640 1 -- 172.21.15.60:0/506533665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b040c97a0 con 0x7f3af00047c0 2023-11-18T20:44:18.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.707+0000 7f3afaffd640 1 -- 172.21.15.60:0/506533665 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3af4057b90 con 0x7f3af00047c0 2023-11-18T20:44:18.708 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:44:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 -- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3ae406ae40 msgr2=0x7f3ae406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3ae406ae40 0x7f3ae406d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3aec036000 tx=0x7f3aec030000 comp rx=0 tx=0).stop 2023-11-18T20:44:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 -- 172.21.15.60:0/506533665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 msgr2=0x7f3b040cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3b040cb920 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3af4007810 tx=0x7f3af400fc00 comp rx=0 tx=0).stop 2023-11-18T20:44:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 -- 172.21.15.60:0/506533665 shutdown_connections 2023-11-18T20:44:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3ae406ae40 0x7f3ae406d300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b04153d00 0x7f3b04143e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b04153880 0x7f3b041438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 --2- 172.21.15.60:0/506533665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00047c0 0x7f3b040cb920 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 -- 172.21.15.60:0/506533665 >> 172.21.15.60:0/506533665 conn(0x7f3b04072020 msgr2=0x7f3b040c70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 -- 172.21.15.60:0/506533665 shutdown_connections 2023-11-18T20:44:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.709+0000 7f3b08851640 1 -- 172.21.15.60:0/506533665 wait complete. 2023-11-18T20:44:18.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:44:18.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.829+0000 7fea130e7640 1 Processor -- start 2023-11-18T20:44:18.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.829+0000 7fea130e7640 1 -- start start 2023-11-18T20:44:18.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.829+0000 7fea130e7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fea0c0d2910 0x7fea0c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.829+0000 7fea130e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea0c0d3730 0x7fea0c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.829+0000 7fea130e7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0c0b2b10 con 0x7fea0c15d330 2023-11-18T20:44:18.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.829+0000 7fea130e7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0c0d3bc0 con 0x7fea0c0d3730 2023-11-18T20:44:18.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.829+0000 7fea130e7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0c0d1590 con 0x7fea0c0d2910 2023-11-18T20:44:18.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.830+0000 7fea11684640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea0c0d3730 0x7fea0c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.830+0000 7fea11684640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea0c0d3730 0x7fea0c0b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33450/0 (socket says 172.21.15.60:33450) 2023-11-18T20:44:18.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.830+0000 7fea11684640 1 -- 172.21.15.60:0/3076267427 learned_addr learned my addr 172.21.15.60:0/3076267427 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:18.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.830+0000 7fea0bfff640 1 --2- 172.21.15.60:0/3076267427 >> v2:172.21.15.100:3300/0 conn(0x7fea0c0d2910 0x7fea0c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.830+0000 7fea11684640 1 -- 172.21.15.60:0/3076267427 >> v2:172.21.15.100:3300/0 conn(0x7fea0c0d2910 msgr2=0x7fea0c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:18.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.830+0000 7fea11684640 1 --2- 172.21.15.60:0/3076267427 >> v2:172.21.15.100:3300/0 conn(0x7fea0c0d2910 0x7fea0c0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.830+0000 7fea11684640 1 -- 172.21.15.60:0/3076267427 >> v1:172.21.15.100:6789/0 conn(0x7fea0c15d330 legacy=0x7fea0c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.830+0000 7fea11684640 1 -- 172.21.15.60:0/3076267427 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c0d1870 con 0x7fea0c0d3730 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea11684640 1 --2- 172.21.15.60:0/3076267427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 0x7fea0c0b6390 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0d1540 tx=0x7fe9fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03260aab60af94d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea0b7fe640 1 -- 172.21.15.60:0/3076267427 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc031870 con 0x7fea0c0d3730 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea0b7fe640 1 -- 172.21.15.60:0/3076267427 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc0319d0 con 0x7fea0c0d3730 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea0b7fe640 1 -- 172.21.15.60:0/3076267427 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc031d60 con 0x7fea0c0d3730 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea130e7640 1 -- 172.21.15.60:0/3076267427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 msgr2=0x7fea0c0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea130e7640 1 --2- 172.21.15.60:0/3076267427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 0x7fea0c0b6390 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0d1540 tx=0x7fe9fc02b910 comp rx=0 tx=0).stop 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea130e7640 1 -- 172.21.15.60:0/3076267427 shutdown_connections 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea130e7640 1 --2- 172.21.15.60:0/3076267427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 0x7fea0c0b6390 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea130e7640 1 --2- 172.21.15.60:0/3076267427 >> v2:172.21.15.100:3300/0 conn(0x7fea0c0d2910 0x7fea0c0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea130e7640 1 -- 172.21.15.60:0/3076267427 >> 172.21.15.60:0/3076267427 conn(0x7fea0c072020 msgr2=0x7fea0c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:18.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea130e7640 1 -- 172.21.15.60:0/3076267427 shutdown_connections 2023-11-18T20:44:18.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.831+0000 7fea130e7640 1 -- 172.21.15.60:0/3076267427 wait complete. 2023-11-18T20:44:18.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.833+0000 7fea130e7640 1 Processor -- start 2023-11-18T20:44:18.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.833+0000 7fea130e7640 1 -- start start 2023-11-18T20:44:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.833+0000 7fea130e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea0c0d2910 0x7fea0c1f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.833+0000 7fea130e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 0x7fea0c1600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.833+0000 7fea130e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea0c15d330 0x7fea0c14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea130e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c1f3480 con 0x7fea0c0d3730 2023-11-18T20:44:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea130e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c0c87d0 con 0x7fea0c0d2910 2023-11-18T20:44:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea130e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c0c7920 con 0x7fea0c15d330 2023-11-18T20:44:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 0x7fea0c1600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 0x7fea0c1600e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33466/0 (socket says 172.21.15.60:33466) 2023-11-18T20:44:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea0bfff640 1 -- 172.21.15.60:0/101326017 learned_addr learned my addr 172.21.15.60:0/101326017 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:18.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea10e83640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea0c0d2910 0x7fea0c1f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea11684640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea0c15d330 0x7fea0c14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea0bfff640 1 -- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea0c15d330 msgr2=0x7fea0c14c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea0bfff640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea0c15d330 0x7fea0c14c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea0bfff640 1 -- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea0c0d2910 msgr2=0x7fea0c1f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea0bfff640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea0c0d2910 0x7fea0c1f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea0bfff640 1 -- 172.21.15.60:0/101326017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc0094d0 con 0x7fea0c0d3730 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea0bfff640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 0x7fea0c1600e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fea0000e9b0 tx=0x7fea0000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea097fa640 1 -- 172.21.15.60:0/101326017 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00009670 con 0x7fea0c0d3730 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea097fa640 1 -- 172.21.15.60:0/101326017 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0000c9a0 con 0x7fea0c0d3730 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea097fa640 1 -- 172.21.15.60:0/101326017 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0000cd50 con 0x7fea0c0d3730 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea130e7640 1 -- 172.21.15.60:0/101326017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c0c7bc0 con 0x7fea0c0d3730 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.834+0000 7fea130e7640 1 -- 172.21.15.60:0/101326017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0c0c8180 con 0x7fea0c0d3730 2023-11-18T20:44:18.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.835+0000 7fea130e7640 1 -- 172.21.15.60:0/101326017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0c0c8960 con 0x7fea0c0d3730 2023-11-18T20:44:18.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.836+0000 7fea097fa640 1 -- 172.21.15.60:0/101326017 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea00013470 con 0x7fea0c0d3730 2023-11-18T20:44:18.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.836+0000 7fea097fa640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9f806ae40 0x7fe9f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:18.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.836+0000 7fea097fa640 1 -- 172.21.15.60:0/101326017 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fea000899a0 con 0x7fea0c0d3730 2023-11-18T20:44:18.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.836+0000 7fea10e83640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9f806ae40 0x7fe9f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:18.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.837+0000 7fea10e83640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9f806ae40 0x7fe9f806d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fea04001e70 tx=0x7fea04009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:18.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:18.840+0000 7fea097fa640 1 -- 172.21.15.60:0/101326017 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea00054970 con 0x7fea0c0d3730 2023-11-18T20:44:19.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.018+0000 7fea130e7640 1 -- 172.21.15.60:0/101326017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea0c069400 con 0x7fea0c0d3730 2023-11-18T20:44:19.019 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.018+0000 7fea097fa640 1 -- 172.21.15.60:0/101326017 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea00017020 con 0x7fea0c0d3730 2023-11-18T20:44:19.020 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:19.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 -- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9f806ae40 msgr2=0x7fe9f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9f806ae40 0x7fe9f806d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fea04001e70 tx=0x7fea04009040 comp rx=0 tx=0).stop 2023-11-18T20:44:19.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 -- 172.21.15.60:0/101326017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 msgr2=0x7fea0c1600e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 0x7fea0c1600e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fea0000e9b0 tx=0x7fea0000ee70 comp rx=0 tx=0).stop 2023-11-18T20:44:19.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 -- 172.21.15.60:0/101326017 shutdown_connections 2023-11-18T20:44:19.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9f806ae40 0x7fe9f806d300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea0c15d330 0x7fea0c14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea0c0d3730 0x7fea0c1600e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 --2- 172.21.15.60:0/101326017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea0c0d2910 0x7fea0c1f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 -- 172.21.15.60:0/101326017 >> 172.21.15.60:0/101326017 conn(0x7fea0c072020 msgr2=0x7fea0c0b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:19.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 -- 172.21.15.60:0/101326017 shutdown_connections 2023-11-18T20:44:19.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.020+0000 7fea130e7640 1 -- 172.21.15.60:0/101326017 wait complete. 2023-11-18T20:44:19.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:44:19.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.140+0000 7ff77a2d0640 1 Processor -- start 2023-11-18T20:44:19.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.140+0000 7ff77a2d0640 1 -- start start 2023-11-18T20:44:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff77a2d0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff774144980 0x7ff77415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff77a2d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 0x7ff7740cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff77a2d0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff774071770 con 0x7ff7740b4370 2023-11-18T20:44:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff77a2d0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7740ccdc0 con 0x7ff774144980 2023-11-18T20:44:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff77a2d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740cd0a0 con 0x7ff7740ceb70 2023-11-18T20:44:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff773fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff774144980 0x7ff77415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff773fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff774144980 0x7ff77415c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33478/0 (socket says 172.21.15.60:33478) 2023-11-18T20:44:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff773fff640 1 -- 172.21.15.60:0/587560729 learned_addr learned my addr 172.21.15.60:0/587560729 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff7737fe640 1 --2- 172.21.15.60:0/587560729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 0x7ff7740cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff773fff640 1 -- 172.21.15.60:0/587560729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 msgr2=0x7ff7740cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:19.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff773fff640 1 --2- 172.21.15.60:0/587560729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 0x7ff7740cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff773fff640 1 -- 172.21.15.60:0/587560729 >> v1:172.21.15.100:6789/0 conn(0x7ff7740b4370 legacy=0x7ff774071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff773fff640 1 -- 172.21.15.60:0/587560729 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740cd380 con 0x7ff774144980 2023-11-18T20:44:19.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.141+0000 7ff7737fe640 1 --2- 172.21.15.60:0/587560729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 0x7ff7740cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:19.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.142+0000 7ff773fff640 1 --2- 172.21.15.60:0/587560729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff774144980 0x7ff77415c7c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009870 tx=0x7ff75c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6144200ffc6bc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:19.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.142+0000 7ff772ffd640 1 -- 172.21.15.60:0/587560729 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c030840 con 0x7ff774144980 2023-11-18T20:44:19.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.142+0000 7ff772ffd640 1 -- 172.21.15.60:0/587560729 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c0309a0 con 0x7ff774144980 2023-11-18T20:44:19.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.142+0000 7ff772ffd640 1 -- 172.21.15.60:0/587560729 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c030d30 con 0x7ff774144980 2023-11-18T20:44:19.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.143+0000 7ff77a2d0640 1 -- 172.21.15.60:0/587560729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff774144980 msgr2=0x7ff77415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.143+0000 7ff77a2d0640 1 --2- 172.21.15.60:0/587560729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff774144980 0x7ff77415c7c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009870 tx=0x7ff75c02b980 comp rx=0 tx=0).stop 2023-11-18T20:44:19.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.143+0000 7ff77a2d0640 1 -- 172.21.15.60:0/587560729 shutdown_connections 2023-11-18T20:44:19.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.143+0000 7ff77a2d0640 1 --2- 172.21.15.60:0/587560729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 0x7ff7740cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.143+0000 7ff77a2d0640 1 --2- 172.21.15.60:0/587560729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff774144980 0x7ff77415c7c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.143+0000 7ff77a2d0640 1 -- 172.21.15.60:0/587560729 >> 172.21.15.60:0/587560729 conn(0x7ff77406a4d0 msgr2=0x7ff77406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:19.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.143+0000 7ff77a2d0640 1 -- 172.21.15.60:0/587560729 shutdown_connections 2023-11-18T20:44:19.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.143+0000 7ff77a2d0640 1 -- 172.21.15.60:0/587560729 wait complete. 2023-11-18T20:44:19.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.144+0000 7ff77a2d0640 1 Processor -- start 2023-11-18T20:44:19.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.144+0000 7ff77a2d0640 1 -- start start 2023-11-18T20:44:19.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff77a2d0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7740b4370 0x7ff7740b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff77a2d0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 0x7ff774141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff77a2d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff774144980 0x7ff7741593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff77a2d0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740b1bc0 con 0x7ff7740b4370 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff77a2d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740a8890 con 0x7ff774144980 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff77a2d0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740a8b70 con 0x7ff7740ceb70 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff773fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7740b4370 0x7ff7740b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff773fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7740b4370 0x7ff7740b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33480/0 (socket says 172.21.15.60:33480) 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff773fff640 1 -- 172.21.15.60:0/157921337 learned_addr learned my addr 172.21.15.60:0/157921337 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff77886d640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff774144980 0x7ff7741593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff7737fe640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 0x7ff774141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff773fff640 1 -- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 msgr2=0x7ff774141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff773fff640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 0x7ff774141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff773fff640 1 -- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff774144980 msgr2=0x7ff7741593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff773fff640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff774144980 0x7ff7741593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff773fff640 1 -- 172.21.15.60:0/157921337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0094d0 con 0x7ff7740b4370 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff7737fe640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 0x7ff774141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.145+0000 7ff773fff640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7740b4370 0x7ff7740b1680 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff75c000c00 tx=0x7ff75c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.146+0000 7ff770ff9640 1 -- 172.21.15.60:0/157921337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c039a90 con 0x7ff7740b4370 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.146+0000 7ff770ff9640 1 -- 172.21.15.60:0/157921337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c039bf0 con 0x7ff7740b4370 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.146+0000 7ff77a2d0640 1 -- 172.21.15.60:0/157921337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7740a8e50 con 0x7ff7740b4370 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.146+0000 7ff770ff9640 1 -- 172.21.15.60:0/157921337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c043410 con 0x7ff7740b4370 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.146+0000 7ff77a2d0640 1 -- 172.21.15.60:0/157921337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7740a9410 con 0x7ff7740b4370 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.147+0000 7ff77a2d0640 1 -- 172.21.15.60:0/157921337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff774069400 con 0x7ff7740b4370 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.147+0000 7ff770ff9640 1 -- 172.21.15.60:0/157921337 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff75c04c3b0 con 0x7ff7740b4370 2023-11-18T20:44:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.147+0000 7ff770ff9640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff75406ae40 0x7ff75406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.147+0000 7ff770ff9640 1 -- 172.21.15.60:0/157921337 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7ff75c0abbb0 con 0x7ff7740b4370 2023-11-18T20:44:19.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.148+0000 7ff7737fe640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff75406ae40 0x7ff75406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.148+0000 7ff7737fe640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff75406ae40 0x7ff75406d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff7741f41b0 tx=0x7ff764008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:19.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.152+0000 7ff770ff9640 1 -- 172.21.15.60:0/157921337 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff75c076b80 con 0x7ff7740b4370 2023-11-18T20:44:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.327+0000 7ff77a2d0640 1 -- 172.21.15.60:0/157921337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff774142450 con 0x7ff7740b4370 2023-11-18T20:44:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.327+0000 7ff770ff9640 1 -- 172.21.15.60:0/157921337 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff75c07a1a0 con 0x7ff7740b4370 2023-11-18T20:44:19.328 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:44:19.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 -- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff75406ae40 msgr2=0x7ff75406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff75406ae40 0x7ff75406d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff7741f41b0 tx=0x7ff764008040 comp rx=0 tx=0).stop 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 -- 172.21.15.60:0/157921337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7740b4370 msgr2=0x7ff7740b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7740b4370 0x7ff7740b1680 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff75c000c00 tx=0x7ff75c004730 comp rx=0 tx=0).stop 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 -- 172.21.15.60:0/157921337 shutdown_connections 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff75406ae40 0x7ff75406d300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff774144980 0x7ff7741593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7740ceb70 0x7ff774141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 --2- 172.21.15.60:0/157921337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7740b4370 0x7ff7740b1680 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 -- 172.21.15.60:0/157921337 >> 172.21.15.60:0/157921337 conn(0x7ff77406a4d0 msgr2=0x7ff7740cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 -- 172.21.15.60:0/157921337 shutdown_connections 2023-11-18T20:44:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.329+0000 7ff77a2d0640 1 -- 172.21.15.60:0/157921337 wait complete. 2023-11-18T20:44:19.341 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-11-18T20:44:19.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.449+0000 7fe981d01640 1 Processor -- start 2023-11-18T20:44:19.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.449+0000 7fe981d01640 1 -- start start 2023-11-18T20:44:19.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.450+0000 7fe981d01640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe97c15d330 0x7fe97c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.450+0000 7fe981d01640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d3900 0x7fe97c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.450+0000 7fe981d01640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97c0b2b10 con 0x7fe97c0d2910 2023-11-18T20:44:19.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.450+0000 7fe981d01640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97c0d1590 con 0x7fe97c15d330 2023-11-18T20:44:19.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.450+0000 7fe981d01640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c0d1870 con 0x7fe97c0d3900 2023-11-18T20:44:19.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.450+0000 7fe97b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe97c15d330 0x7fe97c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.450+0000 7fe97b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe97c15d330 0x7fe97c0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33490/0 (socket says 172.21.15.60:33490) 2023-11-18T20:44:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.450+0000 7fe97b7fe640 1 -- 172.21.15.60:0/394073386 learned_addr learned my addr 172.21.15.60:0/394073386 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.450+0000 7fe97affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe97c0d2910 0x7fe97c0d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55306/0 (socket says 172.21.15.60:55306) 2023-11-18T20:44:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.451+0000 7fe97bfff640 1 --2- 172.21.15.60:0/394073386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d3900 0x7fe97c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.451+0000 7fe97b7fe640 1 -- 172.21.15.60:0/394073386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d3900 msgr2=0x7fe97c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.451+0000 7fe97b7fe640 1 --2- 172.21.15.60:0/394073386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d3900 0x7fe97c0b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.451+0000 7fe97b7fe640 1 -- 172.21.15.60:0/394073386 >> v1:172.21.15.60:6789/0 conn(0x7fe97c0d2910 legacy=0x7fe97c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.451+0000 7fe97b7fe640 1 -- 172.21.15.60:0/394073386 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c0d1b50 con 0x7fe97c15d330 2023-11-18T20:44:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.451+0000 7fe97a7fc640 1 -- 172.21.15.60:0/394073386 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560448605 0 0) 0x7fe97c0b2b10 con 0x7fe97c0d2910 2023-11-18T20:44:19.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.451+0000 7fe97bfff640 1 --2- 172.21.15.60:0/394073386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d3900 0x7fe97c0b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.451+0000 7fe97b7fe640 1 --2- 172.21.15.60:0/394073386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c15d330 0x7fe97c0b8b80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009870 tx=0x7fe96c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86b96709d187c80 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe97a7fc640 1 -- 172.21.15.60:0/394073386 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c031640 con 0x7fe97c15d330 2023-11-18T20:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe97a7fc640 1 -- 172.21.15.60:0/394073386 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c0317a0 con 0x7fe97c15d330 2023-11-18T20:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe97a7fc640 1 -- 172.21.15.60:0/394073386 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c031b30 con 0x7fe97c15d330 2023-11-18T20:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe981d01640 1 -- 172.21.15.60:0/394073386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c15d330 msgr2=0x7fe97c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe981d01640 1 --2- 172.21.15.60:0/394073386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c15d330 0x7fe97c0b8b80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009870 tx=0x7fe96c02b8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe981d01640 1 -- 172.21.15.60:0/394073386 shutdown_connections 2023-11-18T20:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe981d01640 1 --2- 172.21.15.60:0/394073386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d3900 0x7fe97c0b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe981d01640 1 --2- 172.21.15.60:0/394073386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c15d330 0x7fe97c0b8b80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe981d01640 1 -- 172.21.15.60:0/394073386 >> 172.21.15.60:0/394073386 conn(0x7fe97c072020 msgr2=0x7fe97c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:19.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe981d01640 1 -- 172.21.15.60:0/394073386 shutdown_connections 2023-11-18T20:44:19.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.452+0000 7fe981d01640 1 -- 172.21.15.60:0/394073386 wait complete. 2023-11-18T20:44:19.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.453+0000 7fe981d01640 1 Processor -- start 2023-11-18T20:44:19.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.454+0000 7fe981d01640 1 -- start start 2023-11-18T20:44:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.454+0000 7fe981d01640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d2910 0x7fe97c1f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.454+0000 7fe981d01640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c0d3900 0x7fe97c1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.454+0000 7fe981d01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe97c15d330 0x7fe97c0ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.454+0000 7fe981d01640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c0b1af0 con 0x7fe97c0d3900 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.454+0000 7fe981d01640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c1f3ef0 con 0x7fe97c15d330 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.454+0000 7fe981d01640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c0cc680 con 0x7fe97c0d2910 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.454+0000 7fe97affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c0d3900 0x7fe97c1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c0d3900 0x7fe97c1590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33498/0 (socket says 172.21.15.60:33498) 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97affd640 1 -- 172.21.15.60:0/4239451640 learned_addr learned my addr 172.21.15.60:0/4239451640 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97b7fe640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d2910 0x7fe97c1f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97bfff640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe97c15d330 0x7fe97c0ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97affd640 1 -- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d2910 msgr2=0x7fe97c1f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97affd640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d2910 0x7fe97c1f39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97affd640 1 -- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe97c15d330 msgr2=0x7fe97c0ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97affd640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe97c15d330 0x7fe97c0ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97affd640 1 -- 172.21.15.60:0/4239451640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0094d0 con 0x7fe97c0d3900 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97bfff640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe97c15d330 0x7fe97c0ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe97affd640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c0d3900 0x7fe97c1590e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe97000c430 tx=0x7fe97000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe963fff640 1 -- 172.21.15.60:0/4239451640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9700074e0 con 0x7fe97c0d3900 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe963fff640 1 -- 172.21.15.60:0/4239451640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe970007660 con 0x7fe97c0d3900 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe981d01640 1 -- 172.21.15.60:0/4239451640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97c0cc960 con 0x7fe97c0d3900 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe963fff640 1 -- 172.21.15.60:0/4239451640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970004a20 con 0x7fe97c0d3900 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.455+0000 7fe981d01640 1 -- 172.21.15.60:0/4239451640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c0ccf20 con 0x7fe97c0d3900 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.456+0000 7fe981d01640 1 -- 172.21.15.60:0/4239451640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97c0d23b0 con 0x7fe97c0d3900 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.457+0000 7fe963fff640 1 -- 172.21.15.60:0/4239451640 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe970004b80 con 0x7fe97c0d3900 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.457+0000 7fe963fff640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe96806ae40 0x7fe96806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.457+0000 7fe963fff640 1 -- 172.21.15.60:0/4239451640 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fe970089520 con 0x7fe97c0d3900 2023-11-18T20:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.457+0000 7fe97b7fe640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe96806ae40 0x7fe96806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.458+0000 7fe97b7fe640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe96806ae40 0x7fe96806d300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe96c036000 tx=0x7fe96c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.461+0000 7fe963fff640 1 -- 172.21.15.60:0/4239451640 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9700544d0 con 0x7fe97c0d3900 2023-11-18T20:44:19.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.640+0000 7fe981d01640 1 -- 172.21.15.60:0/4239451640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fe97c0cd140 con 0x7fe97c0d3900 2023-11-18T20:44:19.642 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.641+0000 7fe963fff640 1 -- 172.21.15.60:0/4239451640 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) v1 ==== 97+0+11 (secure 0 0 0) 0x7fe970057af0 con 0x7fe97c0d3900 2023-11-18T20:44:19.642 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.642+0000 7fe981d01640 1 -- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe96806ae40 msgr2=0x7fe96806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.642+0000 7fe981d01640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe96806ae40 0x7fe96806d300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe96c036000 tx=0x7fe96c03c040 comp rx=0 tx=0).stop 2023-11-18T20:44:19.645 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.642+0000 7fe981d01640 1 -- 172.21.15.60:0/4239451640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c0d3900 msgr2=0x7fe97c1590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.642+0000 7fe981d01640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c0d3900 0x7fe97c1590e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe97000c430 tx=0x7fe97000c8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.643+0000 7fe981d01640 1 -- 172.21.15.60:0/4239451640 shutdown_connections 2023-11-18T20:44:19.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.643+0000 7fe981d01640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe96806ae40 0x7fe96806d300 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.643+0000 7fe981d01640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe97c15d330 0x7fe97c0ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.643+0000 7fe981d01640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe97c0d3900 0x7fe97c1590e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.643+0000 7fe981d01640 1 --2- 172.21.15.60:0/4239451640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe97c0d2910 0x7fe97c1f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.643+0000 7fe981d01640 1 -- 172.21.15.60:0/4239451640 >> 172.21.15.60:0/4239451640 conn(0x7fe97c072020 msgr2=0x7fe97c0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:19.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.643+0000 7fe981d01640 1 -- 172.21.15.60:0/4239451640 shutdown_connections 2023-11-18T20:44:19.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.643+0000 7fe981d01640 1 -- 172.21.15.60:0/4239451640 wait complete. 2023-11-18T20:44:19.655 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:44:19.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.759+0000 7fd264365640 1 Processor -- start 2023-11-18T20:44:19.761 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.759+0000 7fd264365640 1 -- start start 2023-11-18T20:44:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.760+0000 7fd264365640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd25c15d330 0x7fd25c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.760+0000 7fd264365640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0d2910 0x7fd25c0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.760+0000 7fd264365640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c0b2b10 con 0x7fd25c0d3730 2023-11-18T20:44:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.760+0000 7fd264365640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c0d3b50 con 0x7fd25c15d330 2023-11-18T20:44:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.760+0000 7fd264365640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0d1590 con 0x7fd25c0d2910 2023-11-18T20:44:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.760+0000 7fd262902640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd25c0d3730 0x7fd25c0b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55310/0 (socket says 172.21.15.60:55310) 2023-11-18T20:44:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.760+0000 7fd262902640 1 -- 172.21.15.60:0/800735817 learned_addr learned my addr 172.21.15.60:0/800735817 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd262101640 1 --2- 172.21.15.60:0/800735817 >> v2:172.21.15.100:3300/0 conn(0x7fd25c15d330 0x7fd25c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd261900640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0d2910 0x7fd25c0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493661790 0 0) 0x7fd25c0b2b10 con 0x7fd25c0d3730 2023-11-18T20:44:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd244003520 con 0x7fd25c0d3730 2023-11-18T20:44:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2218284585 0 0) 0x7fd244003520 con 0x7fd25c0d3730 2023-11-18T20:44:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0d2910 msgr2=0x7fd25c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0d2910 0x7fd25c0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 >> v2:172.21.15.100:3300/0 conn(0x7fd25c15d330 msgr2=0x7fd25c0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 --2- 172.21.15.60:0/800735817 >> v2:172.21.15.100:3300/0 conn(0x7fd25c15d330 0x7fd25c0b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0d1870 con 0x7fd25c0d3730 2023-11-18T20:44:19.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fd24c0019f0 con 0x7fd25c0d3730 2023-11-18T20:44:19.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd262101640 1 --2- 172.21.15.60:0/800735817 >> v2:172.21.15.100:3300/0 conn(0x7fd25c15d330 0x7fd25c0b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:19.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd261900640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0d2910 0x7fd25c0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:19.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 >> v1:172.21.15.60:6789/0 conn(0x7fd25c0d3730 legacy=0x7fd25c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244003970 0x7fd244003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 0x7fd244004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0d2910 0x7fd244005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0d3b50 con 0x7fd2440047c0 2023-11-18T20:44:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0d1590 con 0x7fd244003970 2023-11-18T20:44:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24400b980 con 0x7fd25c0d2910 2023-11-18T20:44:19.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd261900640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 0x7fd244004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd262101640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244003970 0x7fd244003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd262902640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0d2910 0x7fd244005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.761+0000 7fd261900640 1 -- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0d2910 msgr2=0x7fd244005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:19.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.762+0000 7fd261900640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0d2910 0x7fd244005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.762+0000 7fd261900640 1 -- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244003970 msgr2=0x7fd244003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:19.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.762+0000 7fd261900640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd244003970 0x7fd244003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.762+0000 7fd261900640 1 -- 172.21.15.60:0/800735817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250009400 con 0x7fd2440047c0 2023-11-18T20:44:19.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.762+0000 7fd261900640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 0x7fd244004c40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd250009c90 tx=0x7fd2440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6347655dbb83d39b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:19.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.762+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25002c8f0 con 0x7fd2440047c0 2023-11-18T20:44:19.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.762+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25002ca50 con 0x7fd2440047c0 2023-11-18T20:44:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.762+0000 7fd2610ff640 1 -- 172.21.15.60:0/800735817 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25002cdc0 con 0x7fd2440047c0 2023-11-18T20:44:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.763+0000 7fd264365640 1 -- 172.21.15.60:0/800735817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 msgr2=0x7fd244004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.763+0000 7fd264365640 1 --2- 172.21.15.60:0/800735817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 0x7fd244004c40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd250009c90 tx=0x7fd2440057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.763+0000 7fd262902640 1 -- 172.21.15.60:0/800735817 reap_dead start 2023-11-18T20:44:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.763+0000 7fd264365640 1 -- 172.21.15.60:0/800735817 shutdown_connections 2023-11-18T20:44:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.763+0000 7fd264365640 1 -- 172.21.15.60:0/800735817 >> 172.21.15.60:0/800735817 conn(0x7fd25c072020 msgr2=0x7fd25c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.763+0000 7fd264365640 1 -- 172.21.15.60:0/800735817 shutdown_connections 2023-11-18T20:44:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.763+0000 7fd264365640 1 -- 172.21.15.60:0/800735817 wait complete. 2023-11-18T20:44:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.764+0000 7fd264365640 1 Processor -- start 2023-11-18T20:44:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.764+0000 7fd264365640 1 -- start start 2023-11-18T20:44:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd264365640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 0x7fd25c15fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd264365640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ccf50 0x7fd25c154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd264365640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25c14fca0 0x7fd25c15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd264365640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0ca3d0 con 0x7fd2440047c0 2023-11-18T20:44:19.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd264365640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0ca6b0 con 0x7fd25c14fca0 2023-11-18T20:44:19.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd264365640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0ca990 con 0x7fd25c0ccf50 2023-11-18T20:44:19.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd262101640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 0x7fd25c15fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd262101640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 0x7fd25c15fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33516/0 (socket says 172.21.15.60:33516) 2023-11-18T20:44:19.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd262101640 1 -- 172.21.15.60:0/1520742507 learned_addr learned my addr 172.21.15.60:0/1520742507 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:19.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd262902640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25c14fca0 0x7fd25c15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd261900640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ccf50 0x7fd25c154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd262101640 1 -- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ccf50 msgr2=0x7fd25c154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd262101640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ccf50 0x7fd25c154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd262101640 1 -- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25c14fca0 msgr2=0x7fd25c15dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd262101640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25c14fca0 0x7fd25c15dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd262101640 1 -- 172.21.15.60:0/1520742507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0cac70 con 0x7fd2440047c0 2023-11-18T20:44:19.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.765+0000 7fd261900640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ccf50 0x7fd25c154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:19.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.766+0000 7fd262101640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 0x7fd25c15fd30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd254011cd0 tx=0x7fd25400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:19.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.766+0000 7fd25affd640 1 -- 172.21.15.60:0/1520742507 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25400fd20 con 0x7fd2440047c0 2023-11-18T20:44:19.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.766+0000 7fd25affd640 1 -- 172.21.15.60:0/1520742507 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25400fe80 con 0x7fd2440047c0 2023-11-18T20:44:19.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.766+0000 7fd264365640 1 -- 172.21.15.60:0/1520742507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c1606a0 con 0x7fd2440047c0 2023-11-18T20:44:19.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.766+0000 7fd25affd640 1 -- 172.21.15.60:0/1520742507 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254004270 con 0x7fd2440047c0 2023-11-18T20:44:19.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.766+0000 7fd264365640 1 -- 172.21.15.60:0/1520742507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c160450 con 0x7fd2440047c0 2023-11-18T20:44:19.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.767+0000 7fd264365640 1 -- 172.21.15.60:0/1520742507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c0ca510 con 0x7fd2440047c0 2023-11-18T20:44:19.783 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.768+0000 7fd25affd640 1 -- 172.21.15.60:0/1520742507 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2540043d0 con 0x7fd2440047c0 2023-11-18T20:44:19.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.768+0000 7fd25affd640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd24806ae40 0x7fd24806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:19.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.768+0000 7fd25affd640 1 -- 172.21.15.60:0/1520742507 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fd254089f00 con 0x7fd2440047c0 2023-11-18T20:44:19.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.768+0000 7fd261900640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd24806ae40 0x7fd24806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:19.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.769+0000 7fd261900640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd24806ae40 0x7fd24806d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd250036000 tx=0x7fd250030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:19.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.772+0000 7fd25affd640 1 -- 172.21.15.60:0/1520742507 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd25c0ca510 con 0x7fd2440047c0 2023-11-18T20:44:19.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.982+0000 7fd264365640 1 -- 172.21.15.60:0/1520742507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd25c0ca510 con 0x7fd2440047c0 2023-11-18T20:44:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.983+0000 7fd25affd640 1 -- 172.21.15.60:0/1520742507 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fd25c0ca510 con 0x7fd2440047c0 2023-11-18T20:44:19.984 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:19.985 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4814,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3885363174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3885363174},{"type":"v1","addr":"172.21.15.100:6837","nonce":3885363174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1102236552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1102236552},{"type":"v1","addr":"172.21.15.60:6835","nonce":1102236552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2680685354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2680685354},{"type":"v1","addr":"172.21.15.100:6836","nonce":2680685354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:43:33.146258+0000","modified":"2023-11-18T20:43:41.162003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5065":{"gid":5065,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3762312847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3762312847},{"type":"v1","addr":"172.21.15.60:6834","nonce":3762312847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-18T20:44:19.985 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 14 2023-11-18T20:44:19.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 -- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd24806ae40 msgr2=0x7fd24806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd24806ae40 0x7fd24806d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd250036000 tx=0x7fd250030000 comp rx=0 tx=0).stop 2023-11-18T20:44:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 -- 172.21.15.60:0/1520742507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 msgr2=0x7fd25c15fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 0x7fd25c15fd30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd254011cd0 tx=0x7fd25400f610 comp rx=0 tx=0).stop 2023-11-18T20:44:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 -- 172.21.15.60:0/1520742507 shutdown_connections 2023-11-18T20:44:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd24806ae40 0x7fd24806d300 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25c14fca0 0x7fd25c15dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ccf50 0x7fd25c154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 --2- 172.21.15.60:0/1520742507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440047c0 0x7fd25c15fd30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 -- 172.21.15.60:0/1520742507 >> 172.21.15.60:0/1520742507 conn(0x7fd25c072020 msgr2=0x7fd25c0c85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 -- 172.21.15.60:0/1520742507 shutdown_connections 2023-11-18T20:44:19.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:19.985+0000 7fd264365640 1 -- 172.21.15.60:0/1520742507 wait complete. 2023-11-18T20:44:19.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.101+0000 7fb46fb33640 1 Processor -- start 2023-11-18T20:44:20.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.102+0000 7fb46fb33640 1 -- start start 2023-11-18T20:44:20.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.102+0000 7fb46fb33640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb46815d330 0x7fb4680b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.102+0000 7fb46fb33640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4680d2910 0x7fb4680d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.102+0000 7fb46fb33640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb4680d3730 0x7fb4680b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.102+0000 7fb46fb33640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4680b2b10 con 0x7fb4680d3730 2023-11-18T20:44:20.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.102+0000 7fb46fb33640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4680d3b50 con 0x7fb46815d330 2023-11-18T20:44:20.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.102+0000 7fb46fb33640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680d1590 con 0x7fb4680d2910 2023-11-18T20:44:20.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.102+0000 7fb46e0d0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb4680d3730 0x7fb4680b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.102+0000 7fb46e0d0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb4680d3730 0x7fb4680b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33518/0 (socket says 172.21.15.60:33518) 2023-11-18T20:44:20.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.102+0000 7fb46e0d0640 1 -- 172.21.15.60:0/3527313513 learned_addr learned my addr 172.21.15.60:0/3527313513 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:20.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.103+0000 7fb46d0ce640 1 --2- 172.21.15.60:0/3527313513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4680d2910 0x7fb4680d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.103+0000 7fb46d8cf640 1 --2- 172.21.15.60:0/3527313513 >> v2:172.21.15.100:3300/0 conn(0x7fb46815d330 0x7fb4680b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.103+0000 7fb46e0d0640 1 -- 172.21.15.60:0/3527313513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4680d2910 msgr2=0x7fb4680d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:20.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.103+0000 7fb46e0d0640 1 --2- 172.21.15.60:0/3527313513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4680d2910 0x7fb4680d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.105 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.103+0000 7fb46e0d0640 1 -- 172.21.15.60:0/3527313513 >> v2:172.21.15.100:3300/0 conn(0x7fb46815d330 msgr2=0x7fb4680b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:20.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.103+0000 7fb46e0d0640 1 --2- 172.21.15.60:0/3527313513 >> v2:172.21.15.100:3300/0 conn(0x7fb46815d330 0x7fb4680b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.103+0000 7fb46e0d0640 1 -- 172.21.15.60:0/3527313513 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680d1870 con 0x7fb4680d3730 2023-11-18T20:44:20.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.103+0000 7fb46d0ce640 1 --2- 172.21.15.60:0/3527313513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4680d2910 0x7fb4680d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46e0d0640 1 --2- 172.21.15.60:0/3527313513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d3730 0x7fb4680b6390 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb4680d0d10 tx=0x7fb46002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33f2d40846f966 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46c8cd640 1 -- 172.21.15.60:0/3527313513 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4600317e0 con 0x7fb4680d3730 2023-11-18T20:44:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46c8cd640 1 -- 172.21.15.60:0/3527313513 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460031940 con 0x7fb4680d3730 2023-11-18T20:44:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46c8cd640 1 -- 172.21.15.60:0/3527313513 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460031cd0 con 0x7fb4680d3730 2023-11-18T20:44:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46fb33640 1 -- 172.21.15.60:0/3527313513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d3730 msgr2=0x7fb4680b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46fb33640 1 --2- 172.21.15.60:0/3527313513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d3730 0x7fb4680b6390 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb4680d0d10 tx=0x7fb46002b880 comp rx=0 tx=0).stop 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46fb33640 1 -- 172.21.15.60:0/3527313513 shutdown_connections 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46fb33640 1 --2- 172.21.15.60:0/3527313513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d3730 0x7fb4680b6390 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46fb33640 1 --2- 172.21.15.60:0/3527313513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4680d2910 0x7fb4680d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46fb33640 1 --2- 172.21.15.60:0/3527313513 >> v2:172.21.15.100:3300/0 conn(0x7fb46815d330 0x7fb4680b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46fb33640 1 -- 172.21.15.60:0/3527313513 >> 172.21.15.60:0/3527313513 conn(0x7fb468072020 msgr2=0x7fb468072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46fb33640 1 -- 172.21.15.60:0/3527313513 shutdown_connections 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.104+0000 7fb46fb33640 1 -- 172.21.15.60:0/3527313513 wait complete. 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.106+0000 7fb46fb33640 1 Processor -- start 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.106+0000 7fb46fb33640 1 -- start start 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.106+0000 7fb46fb33640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d2910 0x7fb468148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.106+0000 7fb46fb33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4680d3730 0x7fb46815f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.106+0000 7fb46fb33640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb46815d330 0x7fb4681430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.106+0000 7fb46fb33640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46815ece0 con 0x7fb4680d2910 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.106+0000 7fb46fb33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680c7500 con 0x7fb4680d3730 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.106+0000 7fb46fb33640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46815e0d0 con 0x7fb46815d330 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.106+0000 7fb46d8cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d2910 0x7fb468148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.106+0000 7fb46d8cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d2910 0x7fb468148b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33526/0 (socket says 172.21.15.60:33526) 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46d8cf640 1 -- 172.21.15.60:0/3067502049 learned_addr learned my addr 172.21.15.60:0/3067502049 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46d0ce640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4680d3730 0x7fb46815f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46e0d0640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb46815d330 0x7fb4681430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46d8cf640 1 -- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb46815d330 msgr2=0x7fb4681430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46d8cf640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb46815d330 0x7fb4681430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46d8cf640 1 -- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4680d3730 msgr2=0x7fb46815f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46d8cf640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4680d3730 0x7fb46815f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46d8cf640 1 -- 172.21.15.60:0/3067502049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4600094d0 con 0x7fb4680d2910 2023-11-18T20:44:20.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46d8cf640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d2910 0x7fb468148b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb458007f20 tx=0x7fb45800e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:20.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb4667fc640 1 -- 172.21.15.60:0/3067502049 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458014620 con 0x7fb4680d2910 2023-11-18T20:44:20.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb4667fc640 1 -- 172.21.15.60:0/3067502049 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb458014780 con 0x7fb4680d2910 2023-11-18T20:44:20.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb4667fc640 1 -- 172.21.15.60:0/3067502049 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458014b30 con 0x7fb4680d2910 2023-11-18T20:44:20.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46fb33640 1 -- 172.21.15.60:0/3067502049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46815e3b0 con 0x7fb4680d2910 2023-11-18T20:44:20.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.107+0000 7fb46fb33640 1 -- 172.21.15.60:0/3067502049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46815e970 con 0x7fb4680d2910 2023-11-18T20:44:20.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.108+0000 7fb46fb33640 1 -- 172.21.15.60:0/3067502049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46815eb30 con 0x7fb4680d2910 2023-11-18T20:44:20.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.109+0000 7fb4667fc640 1 -- 172.21.15.60:0/3067502049 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb458014c90 con 0x7fb4680d2910 2023-11-18T20:44:20.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.109+0000 7fb4667fc640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb45406ae40 0x7fb45406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.109+0000 7fb4667fc640 1 -- 172.21.15.60:0/3067502049 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7fb45808a130 con 0x7fb4680d2910 2023-11-18T20:44:20.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.109+0000 7fb46d0ce640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb45406ae40 0x7fb45406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.110+0000 7fb46d0ce640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb45406ae40 0x7fb45406d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb45c006fd0 tx=0x7fb45c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:20.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.113+0000 7fb4667fc640 1 -- 172.21.15.60:0/3067502049 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4580550e0 con 0x7fb4680d2910 2023-11-18T20:44:20.292 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.291+0000 7fb46fb33640 1 -- 172.21.15.60:0/3067502049 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb468069400 con 0x7fb4680d2910 2023-11-18T20:44:20.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.292+0000 7fb4667fc640 1 -- 172.21.15.60:0/3067502049 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18336 (secure 0 0 0) 0x7fb458058700 con 0x7fb4680d2910 2023-11-18T20:44:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:20.294 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:42.814654+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-11-18T20:43:40.650315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:20.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 -- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb45406ae40 msgr2=0x7fb45406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb45406ae40 0x7fb45406d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb45c006fd0 tx=0x7fb45c008040 comp rx=0 tx=0).stop 2023-11-18T20:44:20.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 -- 172.21.15.60:0/3067502049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d2910 msgr2=0x7fb468148b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d2910 0x7fb468148b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb458007f20 tx=0x7fb45800e570 comp rx=0 tx=0).stop 2023-11-18T20:44:20.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 -- 172.21.15.60:0/3067502049 shutdown_connections 2023-11-18T20:44:20.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb45406ae40 0x7fb45406d300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb46815d330 0x7fb4681430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4680d3730 0x7fb46815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 --2- 172.21.15.60:0/3067502049 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4680d2910 0x7fb468148b80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 -- 172.21.15.60:0/3067502049 >> 172.21.15.60:0/3067502049 conn(0x7fb468072020 msgr2=0x7fb468072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:20.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 -- 172.21.15.60:0/3067502049 shutdown_connections 2023-11-18T20:44:20.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.294+0000 7fb46fb33640 1 -- 172.21.15.60:0/3067502049 wait complete. 2023-11-18T20:44:20.307 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-18T20:44:20.307 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:44:20.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.413+0000 7f52d13f5640 1 Processor -- start 2023-11-18T20:44:20.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.413+0000 7f52d13f5640 1 -- start start 2023-11-18T20:44:20.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52d13f5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f52cc15d2d0 0x7f52cc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52d13f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 0x7f52cc0d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52d13f5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc1614e0 con 0x7f52cc160f70 2023-11-18T20:44:20.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52d13f5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc1617c0 con 0x7f52cc15d2d0 2023-11-18T20:44:20.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52d13f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc161aa0 con 0x7f52cc0ce650 2023-11-18T20:44:20.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52caffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f52cc15d2d0 0x7f52cc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52cb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 0x7f52cc0d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.416 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52c3fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f52cc160f70 0x7f52cc0cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:44396/0 (socket says 172.21.15.60:44396) 2023-11-18T20:44:20.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52c3fff640 1 -- 172.21.15.60:0/3156876006 learned_addr learned my addr 172.21.15.60:0/3156876006 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:20.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52cb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 0x7f52cc0d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:48142/0 (socket says 172.21.15.60:48142) 2023-11-18T20:44:20.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.414+0000 7f52caffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f52cc15d2d0 0x7f52cc0b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:41694/0 (socket says 172.21.15.60:41694) 2023-11-18T20:44:20.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.415+0000 7f52cb7fe640 1 -- 172.21.15.60:0/3156876006 >> v2:172.21.15.100:3300/0 conn(0x7f52cc15d2d0 msgr2=0x7f52cc0b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:20.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.415+0000 7f52cb7fe640 1 --2- 172.21.15.60:0/3156876006 >> v2:172.21.15.100:3300/0 conn(0x7f52cc15d2d0 0x7f52cc0b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.415+0000 7f52cb7fe640 1 -- 172.21.15.60:0/3156876006 >> v1:172.21.15.100:6789/0 conn(0x7f52cc160f70 legacy=0x7f52cc0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.415+0000 7f52cb7fe640 1 -- 172.21.15.60:0/3156876006 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc161d80 con 0x7f52cc0ce650 2023-11-18T20:44:20.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.415+0000 7f52ca7fc640 1 -- 172.21.15.60:0/3156876006 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623115187 0 0) 0x7f52cc1614e0 con 0x7f52cc160f70 2023-11-18T20:44:20.417 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.415+0000 7f52caffd640 1 --2- 172.21.15.60:0/3156876006 >> v2:172.21.15.100:3300/0 conn(0x7f52cc15d2d0 0x7f52cc0b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52cb7fe640 1 --2- 172.21.15.60:0/3156876006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 0x7f52cc0d1af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009d90 tx=0x7f52c402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c3e9e263507c941 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52ca7fc640 1 -- 172.21.15.60:0/3156876006 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c402cc50 con 0x7f52cc0ce650 2023-11-18T20:44:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52ca7fc640 1 -- 172.21.15.60:0/3156876006 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c402cdb0 con 0x7f52cc0ce650 2023-11-18T20:44:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52ca7fc640 1 -- 172.21.15.60:0/3156876006 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4034660 con 0x7f52cc0ce650 2023-11-18T20:44:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52d13f5640 1 -- 172.21.15.60:0/3156876006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 msgr2=0x7f52cc0d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52d13f5640 1 --2- 172.21.15.60:0/3156876006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 0x7f52cc0d1af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009d90 tx=0x7f52c402bea0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52d13f5640 1 -- 172.21.15.60:0/3156876006 shutdown_connections 2023-11-18T20:44:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52d13f5640 1 --2- 172.21.15.60:0/3156876006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 0x7f52cc0d1af0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52d13f5640 1 --2- 172.21.15.60:0/3156876006 >> v2:172.21.15.100:3300/0 conn(0x7f52cc15d2d0 0x7f52cc0b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52d13f5640 1 -- 172.21.15.60:0/3156876006 >> 172.21.15.60:0/3156876006 conn(0x7f52cc072230 msgr2=0x7f52cc0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:20.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52d13f5640 1 -- 172.21.15.60:0/3156876006 shutdown_connections 2023-11-18T20:44:20.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.416+0000 7f52d13f5640 1 -- 172.21.15.60:0/3156876006 wait complete. 2023-11-18T20:44:20.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.418+0000 7f52d13f5640 1 Processor -- start 2023-11-18T20:44:20.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.418+0000 7f52d13f5640 1 -- start start 2023-11-18T20:44:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.418+0000 7f52d13f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 0x7f52cc158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.418+0000 7f52d13f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52cc15d2d0 0x7f52cc159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.418+0000 7f52d13f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52cc160f70 0x7f52cc1f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.418+0000 7f52d13f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc159c40 con 0x7f52cc15d2d0 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.418+0000 7f52d13f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc15db10 con 0x7f52cc160f70 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.418+0000 7f52d13f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc15ddf0 con 0x7f52cc0ce650 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52cc15d2d0 0x7f52cc159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52cc15d2d0 0x7f52cc159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33534/0 (socket says 172.21.15.60:33534) 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c3fff640 1 -- 172.21.15.60:0/664259252 learned_addr learned my addr 172.21.15.60:0/664259252 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52caffd640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 0x7f52cc158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52cb7fe640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52cc160f70 0x7f52cc1f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c3fff640 1 -- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 msgr2=0x7f52cc158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c3fff640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 0x7f52cc158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c3fff640 1 -- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52cc160f70 msgr2=0x7f52cc1f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c3fff640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52cc160f70 0x7f52cc1f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c3fff640 1 -- 172.21.15.60:0/664259252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc009400 con 0x7f52cc15d2d0 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c3fff640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52cc15d2d0 0x7f52cc159420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f52b400c430 tx=0x7f52b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c37fe640 1 -- 172.21.15.60:0/664259252 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40074e0 con 0x7f52cc15d2d0 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c37fe640 1 -- 172.21.15.60:0/664259252 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4007660 con 0x7f52cc15d2d0 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52d13f5640 1 -- 172.21.15.60:0/664259252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c4009a70 con 0x7f52cc15d2d0 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52c37fe640 1 -- 172.21.15.60:0/664259252 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4004a20 con 0x7f52cc15d2d0 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.419+0000 7f52d13f5640 1 -- 172.21.15.60:0/664259252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc15e3b0 con 0x7f52cc15d2d0 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.420+0000 7f52d13f5640 1 -- 172.21.15.60:0/664259252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc069400 con 0x7f52cc15d2d0 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.421+0000 7f52c37fe640 1 -- 172.21.15.60:0/664259252 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f52b4004b80 con 0x7f52cc15d2d0 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.421+0000 7f52c37fe640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f52ac06ae40 0x7f52ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.421+0000 7f52c37fe640 1 -- 172.21.15.60:0/664259252 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f52b40895c0 con 0x7f52cc15d2d0 2023-11-18T20:44:20.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.421+0000 7f52caffd640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f52ac06ae40 0x7f52ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.422+0000 7f52caffd640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f52ac06ae40 0x7f52ac06d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009c60 tx=0x7f52bc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:20.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.426+0000 7f52c37fe640 1 -- 172.21.15.60:0/664259252 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52b40544f0 con 0x7f52cc15d2d0 2023-11-18T20:44:20.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.641+0000 7f52d13f5640 1 -- 172.21.15.60:0/664259252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52cc0d0aa0 con 0x7f52cc15d2d0 2023-11-18T20:44:20.643 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.642+0000 7f52c37fe640 1 -- 172.21.15.60:0/664259252 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f52b4057b10 con 0x7f52cc15d2d0 2023-11-18T20:44:20.643 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:20.644 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4814,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3885363174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3885363174},{"type":"v1","addr":"172.21.15.100:6837","nonce":3885363174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1102236552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1102236552},{"type":"v1","addr":"172.21.15.60:6835","nonce":1102236552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2680685354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2680685354},{"type":"v1","addr":"172.21.15.100:6836","nonce":2680685354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:43:33.146258+0000","modified":"2023-11-18T20:43:41.162003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5065":{"gid":5065,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3762312847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3762312847},{"type":"v1","addr":"172.21.15.60:6834","nonce":3762312847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-18T20:44:20.644 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 14 2023-11-18T20:44:20.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 -- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f52ac06ae40 msgr2=0x7f52ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f52ac06ae40 0x7f52ac06d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009c60 tx=0x7f52bc008040 comp rx=0 tx=0).stop 2023-11-18T20:44:20.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 -- 172.21.15.60:0/664259252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52cc15d2d0 msgr2=0x7f52cc159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52cc15d2d0 0x7f52cc159420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f52b400c430 tx=0x7f52b400c8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 -- 172.21.15.60:0/664259252 shutdown_connections 2023-11-18T20:44:20.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f52ac06ae40 0x7f52ac06d300 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52cc160f70 0x7f52cc1f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52cc15d2d0 0x7f52cc159420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 --2- 172.21.15.60:0/664259252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52cc0ce650 0x7f52cc158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 -- 172.21.15.60:0/664259252 >> 172.21.15.60:0/664259252 conn(0x7f52cc072230 msgr2=0x7f52cc15a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:20.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 -- 172.21.15.60:0/664259252 shutdown_connections 2023-11-18T20:44:20.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.644+0000 7f52d13f5640 1 -- 172.21.15.60:0/664259252 wait complete. 2023-11-18T20:44:20.656 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:44:20.762 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.761+0000 7f6ea8c94640 1 Processor -- start 2023-11-18T20:44:20.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.761+0000 7f6ea8c94640 1 -- start start 2023-11-18T20:44:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.762+0000 7f6ea8c94640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6ea4140340 0x7f6ea4166000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.762+0000 7f6ea8c94640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea415ba40 con 0x7f6ea406f210 2023-11-18T20:44:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.762+0000 7f6ea8c94640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea41416e0 con 0x7f6ea415c2f0 2023-11-18T20:44:20.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.762+0000 7f6ea8c94640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea415c6d0 con 0x7f6ea4140340 2023-11-18T20:44:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.762+0000 7f6ea1d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6ea406f210 0x7f6ea4140800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49422/0 (socket says 172.21.15.60:49422) 2023-11-18T20:44:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.762+0000 7f6ea1d9b640 1 -- 172.21.15.60:0/132174361 learned_addr learned my addr 172.21.15.60:0/132174361 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.762+0000 7f6ea2d9d640 1 --2- 172.21.15.60:0/132174361 >> v2:172.21.15.100:3300/0 conn(0x7f6ea4140340 0x7f6ea4166000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108403815 0 0) 0x7f6ea415ba40 con 0x7f6ea406f210 2023-11-18T20:44:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e8c003520 con 0x7f6ea406f210 2023-11-18T20:44:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3514836881 0 0) 0x7f6e8c003520 con 0x7f6ea406f210 2023-11-18T20:44:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 >> v2:172.21.15.100:3300/0 conn(0x7f6ea4140340 msgr2=0x7f6ea4166000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 --2- 172.21.15.60:0/132174361 >> v2:172.21.15.100:3300/0 conn(0x7f6ea4140340 0x7f6ea4166000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 >> v1:172.21.15.100:6789/0 conn(0x7f6ea415c2f0 legacy=0x7f6ea406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4141400 con 0x7f6ea406f210 2023-11-18T20:44:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f6e98001670 con 0x7f6ea406f210 2023-11-18T20:44:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 >> v1:172.21.15.60:6789/0 conn(0x7f6ea406f210 legacy=0x7f6ea4140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e8c003970 0x7f6e8c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e8c0047c0 0x7f6e8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e8c005180 0x7f6e8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c00c050 con 0x7f6e8c0047c0 2023-11-18T20:44:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c00c190 con 0x7f6e8c003970 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c00c350 con 0x7f6e8c005180 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea1d9b640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e8c0047c0 0x7f6e8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea2d9d640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e8c005180 0x7f6e8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea259c640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e8c003970 0x7f6e8c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea1d9b640 1 -- 172.21.15.60:0/132174361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e8c005180 msgr2=0x7f6e8c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea1d9b640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e8c005180 0x7f6e8c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea1d9b640 1 -- 172.21.15.60:0/132174361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e8c003970 msgr2=0x7f6e8c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea1d9b640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e8c003970 0x7f6e8c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea1d9b640 1 -- 172.21.15.60:0/132174361 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c003520 con 0x7f6e8c0047c0 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea2d9d640 1 -- 172.21.15.60:0/132174361 reap_dead start 2023-11-18T20:44:20.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.763+0000 7f6ea259c640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e8c003970 0x7f6e8c003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:20.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.764+0000 7f6ea1d9b640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e8c0047c0 0x7f6e8c004c40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800c240 tx=0x7f6e8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fdda04bff306bd server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:20.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.764+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98005ad0 con 0x7f6e8c0047c0 2023-11-18T20:44:20.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.764+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e98005c30 con 0x7f6e8c0047c0 2023-11-18T20:44:20.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.764+0000 7f6ea159a640 1 -- 172.21.15.60:0/132174361 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9802c850 con 0x7f6e8c0047c0 2023-11-18T20:44:20.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.765+0000 7f6ea8c94640 1 -- 172.21.15.60:0/132174361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e8c0047c0 msgr2=0x7f6e8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.765+0000 7f6ea8c94640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e8c0047c0 0x7f6e8c004c40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800c240 tx=0x7f6e8c0057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.765+0000 7f6ea8c94640 1 -- 172.21.15.60:0/132174361 shutdown_connections 2023-11-18T20:44:20.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.765+0000 7f6ea8c94640 1 --2- 172.21.15.60:0/132174361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e8c0047c0 0x7f6e8c004c40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.765+0000 7f6ea8c94640 1 -- 172.21.15.60:0/132174361 >> 172.21.15.60:0/132174361 conn(0x7f6ea406a6b0 msgr2=0x7f6ea406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:20.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.765+0000 7f6ea8c94640 1 -- 172.21.15.60:0/132174361 shutdown_connections 2023-11-18T20:44:20.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.765+0000 7f6ea8c94640 1 -- 172.21.15.60:0/132174361 wait complete. 2023-11-18T20:44:20.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.766+0000 7f6ea8c94640 1 Processor -- start 2023-11-18T20:44:20.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.766+0000 7f6ea8c94640 1 -- start start 2023-11-18T20:44:20.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea8c94640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e8c0047c0 0x7f6ea413b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea8c94640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ea41f6d20 0x7f6ea4152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea8c94640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea41530f0 0x7f6ea4154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea8c94640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea40b1e60 con 0x7f6ea41f6d20 2023-11-18T20:44:20.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea8c94640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea40d0320 con 0x7f6ea41530f0 2023-11-18T20:44:20.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea8c94640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea4153720 con 0x7f6e8c0047c0 2023-11-18T20:44:20.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ea41f6d20 0x7f6ea4152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ea41f6d20 0x7f6ea4152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34994/0 (socket says 172.21.15.60:34994) 2023-11-18T20:44:20.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea1d9b640 1 -- 172.21.15.60:0/3927489337 learned_addr learned my addr 172.21.15.60:0/3927489337 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:20.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea259c640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e8c0047c0 0x7f6ea413b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea2d9d640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea41530f0 0x7f6ea4154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea1d9b640 1 -- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e8c0047c0 msgr2=0x7f6ea413b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea1d9b640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e8c0047c0 0x7f6ea413b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea1d9b640 1 -- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea41530f0 msgr2=0x7f6ea4154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea1d9b640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea41530f0 0x7f6ea4154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.767+0000 7f6ea1d9b640 1 -- 172.21.15.60:0/3927489337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea41f6240 con 0x7f6ea41f6d20 2023-11-18T20:44:20.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.768+0000 7f6ea1d9b640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ea41f6d20 0x7f6ea4152bb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6e9802c3f0 tx=0x7f6e9802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:20.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.768+0000 7f6e92ffd640 1 -- 172.21.15.60:0/3927489337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9803c040 con 0x7f6ea41f6d20 2023-11-18T20:44:20.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.768+0000 7f6e92ffd640 1 -- 172.21.15.60:0/3927489337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9802cd20 con 0x7f6ea41f6d20 2023-11-18T20:44:20.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.768+0000 7f6ea8c94640 1 -- 172.21.15.60:0/3927489337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea41f6520 con 0x7f6ea41f6d20 2023-11-18T20:44:20.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.768+0000 7f6e92ffd640 1 -- 172.21.15.60:0/3927489337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98007290 con 0x7f6ea41f6d20 2023-11-18T20:44:20.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.768+0000 7f6ea8c94640 1 -- 172.21.15.60:0/3927489337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea4153a00 con 0x7f6ea41f6d20 2023-11-18T20:44:20.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.769+0000 7f6ea8c94640 1 -- 172.21.15.60:0/3927489337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea41534d0 con 0x7f6ea41f6d20 2023-11-18T20:44:20.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.769+0000 7f6e92ffd640 1 -- 172.21.15.60:0/3927489337 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e98032070 con 0x7f6ea41f6d20 2023-11-18T20:44:20.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.770+0000 7f6e92ffd640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e8406ae70 0x7f6e8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:20.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.770+0000 7f6e92ffd640 1 -- 172.21.15.60:0/3927489337 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f6e980aab20 con 0x7f6ea41f6d20 2023-11-18T20:44:20.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.770+0000 7f6ea259c640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e8406ae70 0x7f6e8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:20.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.770+0000 7f6ea259c640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e8406ae70 0x7f6e8406d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c000c00 tx=0x7f6e9c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:20.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.774+0000 7f6e92ffd640 1 -- 172.21.15.60:0/3927489337 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e98075ca0 con 0x7f6ea41f6d20 2023-11-18T20:44:20.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.987+0000 7f6ea8c94640 1 -- 172.21.15.60:0/3927489337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ea4069400 con 0x7f6ea41f6d20 2023-11-18T20:44:20.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.987+0000 7f6e92ffd640 1 -- 172.21.15.60:0/3927489337 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f6e980792c0 con 0x7f6ea41f6d20 2023-11-18T20:44:20.989 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:20.989 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4814,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3885363174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3885363174},{"type":"v1","addr":"172.21.15.100:6837","nonce":3885363174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1102236552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1102236552},{"type":"v1","addr":"172.21.15.60:6835","nonce":1102236552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2680685354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2680685354},{"type":"v1","addr":"172.21.15.100:6836","nonce":2680685354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:43:33.146258+0000","modified":"2023-11-18T20:43:41.162003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5065":{"gid":5065,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3762312847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3762312847},{"type":"v1","addr":"172.21.15.60:6834","nonce":3762312847}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-18T20:44:20.989 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 14 2023-11-18T20:44:20.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 -- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e8406ae70 msgr2=0x7f6e8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e8406ae70 0x7f6e8406d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c000c00 tx=0x7f6e9c009040 comp rx=0 tx=0).stop 2023-11-18T20:44:20.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 -- 172.21.15.60:0/3927489337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ea41f6d20 msgr2=0x7f6ea4152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:20.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ea41f6d20 0x7f6ea4152bb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6e9802c3f0 tx=0x7f6e9802e070 comp rx=0 tx=0).stop 2023-11-18T20:44:20.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 -- 172.21.15.60:0/3927489337 shutdown_connections 2023-11-18T20:44:20.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6e8406ae70 0x7f6e8406d330 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea41530f0 0x7f6ea4154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ea41f6d20 0x7f6ea4152bb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 --2- 172.21.15.60:0/3927489337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e8c0047c0 0x7f6ea413b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:20.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 -- 172.21.15.60:0/3927489337 >> 172.21.15.60:0/3927489337 conn(0x7f6ea406a6b0 msgr2=0x7f6ea40b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:20.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 -- 172.21.15.60:0/3927489337 shutdown_connections 2023-11-18T20:44:20.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:20.990+0000 7f6ea8c94640 1 -- 172.21.15.60:0/3927489337 wait complete. 2023-11-18T20:44:21.002 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:21.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.107+0000 7f6573a1f640 1 Processor -- start 2023-11-18T20:44:21.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.107+0000 7f6573a1f640 1 -- start start 2023-11-18T20:44:21.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6573a1f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f656c160fb0 0x7f656c161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6573a1f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f656c0d0ed0 0x7f656c0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6573a1f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c161d20 con 0x7f656c06ea50 2023-11-18T20:44:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6573a1f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656c15e760 con 0x7f656c160fb0 2023-11-18T20:44:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6573a1f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656c15d9a0 con 0x7f656c0d0ed0 2023-11-18T20:44:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6570fba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f656c160fb0 0x7f656c161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6570fba640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f656c160fb0 0x7f656c161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34998/0 (socket says 172.21.15.60:34998) 2023-11-18T20:44:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6570fba640 1 -- 172.21.15.60:0/256446140 learned_addr learned my addr 172.21.15.60:0/256446140 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6571fbc640 1 --2- 172.21.15.60:0/256446140 >> v2:172.21.15.100:3300/0 conn(0x7f656c0d0ed0 0x7f656c0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6570fba640 1 -- 172.21.15.60:0/256446140 >> v2:172.21.15.100:3300/0 conn(0x7f656c0d0ed0 msgr2=0x7f656c0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6570fba640 1 --2- 172.21.15.60:0/256446140 >> v2:172.21.15.100:3300/0 conn(0x7f656c0d0ed0 0x7f656c0d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6570fba640 1 -- 172.21.15.60:0/256446140 >> v1:172.21.15.100:6789/0 conn(0x7f656c06ea50 legacy=0x7f656c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:21.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6570fba640 1 -- 172.21.15.60:0/256446140 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c15dc80 con 0x7f656c160fb0 2023-11-18T20:44:21.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.108+0000 7f6571fbc640 1 --2- 172.21.15.60:0/256446140 >> v2:172.21.15.100:3300/0 conn(0x7f656c0d0ed0 0x7f656c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:21.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.109+0000 7f6570fba640 1 --2- 172.21.15.60:0/256446140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 0x7f656c161440 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f656c0b55a0 tx=0x7f655c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9b04a46e365cb server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:21.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.109+0000 7f6563fff640 1 -- 172.21.15.60:0/256446140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c0317e0 con 0x7f656c160fb0 2023-11-18T20:44:21.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.109+0000 7f6563fff640 1 -- 172.21.15.60:0/256446140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c031940 con 0x7f656c160fb0 2023-11-18T20:44:21.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.109+0000 7f6563fff640 1 -- 172.21.15.60:0/256446140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c031cd0 con 0x7f656c160fb0 2023-11-18T20:44:21.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.110+0000 7f6573a1f640 1 -- 172.21.15.60:0/256446140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 msgr2=0x7f656c161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.110+0000 7f6573a1f640 1 --2- 172.21.15.60:0/256446140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 0x7f656c161440 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f656c0b55a0 tx=0x7f655c02b880 comp rx=0 tx=0).stop 2023-11-18T20:44:21.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.110+0000 7f6573a1f640 1 -- 172.21.15.60:0/256446140 shutdown_connections 2023-11-18T20:44:21.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.110+0000 7f6573a1f640 1 --2- 172.21.15.60:0/256446140 >> v2:172.21.15.100:3300/0 conn(0x7f656c0d0ed0 0x7f656c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.110+0000 7f6573a1f640 1 --2- 172.21.15.60:0/256446140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 0x7f656c161440 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.110+0000 7f6573a1f640 1 -- 172.21.15.60:0/256446140 >> 172.21.15.60:0/256446140 conn(0x7f656c06a6b0 msgr2=0x7f656c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:21.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.110+0000 7f6573a1f640 1 -- 172.21.15.60:0/256446140 shutdown_connections 2023-11-18T20:44:21.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.110+0000 7f6573a1f640 1 -- 172.21.15.60:0/256446140 wait complete. 2023-11-18T20:44:21.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.111+0000 7f6573a1f640 1 Processor -- start 2023-11-18T20:44:21.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.111+0000 7f6573a1f640 1 -- start start 2023-11-18T20:44:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6573a1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f656c06ea50 0x7f656c13c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6573a1f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f656c0d0ed0 0x7f656c13cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6573a1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 0x7f656c0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6573a1f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c13df90 con 0x7f656c160fb0 2023-11-18T20:44:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6573a1f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c156f30 con 0x7f656c0d0ed0 2023-11-18T20:44:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6573a1f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c13d380 con 0x7f656c06ea50 2023-11-18T20:44:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6571fbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 0x7f656c0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6571fbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 0x7f656c0b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35014/0 (socket says 172.21.15.60:35014) 2023-11-18T20:44:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6571fbc640 1 -- 172.21.15.60:0/1469564681 learned_addr learned my addr 172.21.15.60:0/1469564681 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6570fba640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f656c0d0ed0 0x7f656c13cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f65717bb640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f656c06ea50 0x7f656c13c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6571fbc640 1 -- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f656c06ea50 msgr2=0x7f656c13c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6571fbc640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f656c06ea50 0x7f656c13c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6571fbc640 1 -- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f656c0d0ed0 msgr2=0x7f656c13cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6571fbc640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f656c0d0ed0 0x7f656c13cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6571fbc640 1 -- 172.21.15.60:0/1469564681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0094d0 con 0x7f656c160fb0 2023-11-18T20:44:21.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6570fba640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f656c0d0ed0 0x7f656c13cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:21.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.112+0000 7f6571fbc640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 0x7f656c0b3d20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f656400ce70 tx=0x7f656400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:21.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.113+0000 7f6561ffb640 1 -- 172.21.15.60:0/1469564681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656400d840 con 0x7f656c160fb0 2023-11-18T20:44:21.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.113+0000 7f6561ffb640 1 -- 172.21.15.60:0/1469564681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656400d9a0 con 0x7f656c160fb0 2023-11-18T20:44:21.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.113+0000 7f6573a1f640 1 -- 172.21.15.60:0/1469564681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c13d660 con 0x7f656c160fb0 2023-11-18T20:44:21.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.113+0000 7f6561ffb640 1 -- 172.21.15.60:0/1469564681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656400dd50 con 0x7f656c160fb0 2023-11-18T20:44:21.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.113+0000 7f6573a1f640 1 -- 172.21.15.60:0/1469564681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c13dc20 con 0x7f656c160fb0 2023-11-18T20:44:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.114+0000 7f6573a1f640 1 -- 172.21.15.60:0/1469564681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c13e140 con 0x7f656c160fb0 2023-11-18T20:44:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.114+0000 7f6561ffb640 1 -- 172.21.15.60:0/1469564681 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f656401e3b0 con 0x7f656c160fb0 2023-11-18T20:44:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.114+0000 7f6561ffb640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f655006ae70 0x7f655006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.115+0000 7f6561ffb640 1 -- 172.21.15.60:0/1469564681 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f656408a190 con 0x7f656c160fb0 2023-11-18T20:44:21.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.115+0000 7f65717bb640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f655006ae70 0x7f655006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.115+0000 7f65717bb640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f655006ae70 0x7f655006d330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6568006fd0 tx=0x7f6568008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:21.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.119+0000 7f6561ffb640 1 -- 172.21.15.60:0/1469564681 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f656c13e140 con 0x7f656c160fb0 2023-11-18T20:44:21.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.300+0000 7f6573a1f640 1 -- 172.21.15.60:0/1469564681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f656c13e140 con 0x7f656c160fb0 2023-11-18T20:44:21.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.301+0000 7f6561ffb640 1 -- 172.21.15.60:0/1469564681 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18336 (secure 0 0 0) 0x7f656c13e140 con 0x7f656c160fb0 2023-11-18T20:44:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:21.303 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:43:42.814654+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:43:31.703619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-18T20:43:32.121610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2023-11-18T20:43:40.650315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:21.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 -- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f655006ae70 msgr2=0x7f655006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f655006ae70 0x7f655006d330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6568006fd0 tx=0x7f6568008040 comp rx=0 tx=0).stop 2023-11-18T20:44:21.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 -- 172.21.15.60:0/1469564681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 msgr2=0x7f656c0b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 0x7f656c0b3d20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f656400ce70 tx=0x7f656400a710 comp rx=0 tx=0).stop 2023-11-18T20:44:21.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 -- 172.21.15.60:0/1469564681 shutdown_connections 2023-11-18T20:44:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f655006ae70 0x7f655006d330 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f656c160fb0 0x7f656c0b3d20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f656c0d0ed0 0x7f656c13cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 --2- 172.21.15.60:0/1469564681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f656c06ea50 0x7f656c13c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 -- 172.21.15.60:0/1469564681 >> 172.21.15.60:0/1469564681 conn(0x7f656c06a6b0 msgr2=0x7f656c06c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 -- 172.21.15.60:0/1469564681 shutdown_connections 2023-11-18T20:44:21.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.304+0000 7f6573a1f640 1 -- 172.21.15.60:0/1469564681 wait complete. 2023-11-18T20:44:21.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-18T20:44:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.423+0000 7f611e856640 1 Processor -- start 2023-11-18T20:44:21.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.423+0000 7f611e856640 1 -- start start 2023-11-18T20:44:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f611e856640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f61181409d0 0x7f61181618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f611e856640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61181406f0 con 0x7f611806abe0 2023-11-18T20:44:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f611e856640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6118143960 con 0x7f6118140310 2023-11-18T20:44:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f611e856640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6118161e20 con 0x7f61181409d0 2023-11-18T20:44:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f61177fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f611806abe0 0x7f611806efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49424/0 (socket says 172.21.15.60:49424) 2023-11-18T20:44:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f61177fe640 1 -- 172.21.15.60:0/720368115 learned_addr learned my addr 172.21.15.60:0/720368115 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232877145 0 0) 0x7f61181406f0 con 0x7f611806abe0 2023-11-18T20:44:21.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6100003520 con 0x7f611806abe0 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f611cdf3640 1 --2- 172.21.15.60:0/720368115 >> v2:172.21.15.100:3300/0 conn(0x7f61181409d0 0x7f61181618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 351708397 0 0) 0x7f6100003520 con 0x7f611806abe0 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 >> v2:172.21.15.100:3300/0 conn(0x7f61181409d0 msgr2=0x7f61181618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f6116ffd640 1 --2- 172.21.15.60:0/720368115 >> v2:172.21.15.100:3300/0 conn(0x7f61181409d0 0x7f61181618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.424+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 >> v1:172.21.15.100:6789/0 conn(0x7f6118140310 legacy=0x7f611806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6118142de0 con 0x7f611806abe0 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f6108001670 con 0x7f611806abe0 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f611cdf3640 1 --2- 172.21.15.60:0/720368115 >> v2:172.21.15.100:3300/0 conn(0x7f61181409d0 0x7f61181618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 >> v1:172.21.15.60:6789/0 conn(0x7f611806abe0 legacy=0x7f611806efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f6116ffd640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6100003970 0x7f6100003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f6116ffd640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61000047c0 0x7f6100004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f6116ffd640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61000052a0 0x7f610000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6118161e20 con 0x7f61000052a0 2023-11-18T20:44:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610000c170 con 0x7f61000047c0 2023-11-18T20:44:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610000c330 con 0x7f6100003970 2023-11-18T20:44:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f611cdf3640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61000052a0 0x7f610000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f6117fff640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6100003970 0x7f6100003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f61177fe640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61000047c0 0x7f6100004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f611cdf3640 1 -- 172.21.15.60:0/720368115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6100003970 msgr2=0x7f6100003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:21.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f611cdf3640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6100003970 0x7f6100003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f611cdf3640 1 -- 172.21.15.60:0/720368115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61000047c0 msgr2=0x7f6100004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f611cdf3640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61000047c0 0x7f6100004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.425+0000 7f611cdf3640 1 -- 172.21.15.60:0/720368115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6100003520 con 0x7f61000052a0 2023-11-18T20:44:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.426+0000 7f611cdf3640 1 -- 172.21.15.60:0/720368115 reap_dead start 2023-11-18T20:44:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.426+0000 7f611cdf3640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61000052a0 0x7f610000b980 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f61100073b0 tx=0x7f61000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10021641b22853e server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.426+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611002cd90 con 0x7f61000052a0 2023-11-18T20:44:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.426+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6110034440 con 0x7f61000052a0 2023-11-18T20:44:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.426+0000 7f6116ffd640 1 -- 172.21.15.60:0/720368115 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61100347d0 con 0x7f61000052a0 2023-11-18T20:44:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.426+0000 7f611e856640 1 -- 172.21.15.60:0/720368115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61000052a0 msgr2=0x7f610000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.426+0000 7f611e856640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61000052a0 0x7f610000b980 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f61100073b0 tx=0x7f61000057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.427+0000 7f611e856640 1 -- 172.21.15.60:0/720368115 shutdown_connections 2023-11-18T20:44:21.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.427+0000 7f611e856640 1 --2- 172.21.15.60:0/720368115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61000052a0 0x7f610000b980 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.427+0000 7f611e856640 1 -- 172.21.15.60:0/720368115 >> 172.21.15.60:0/720368115 conn(0x7f61180b1e50 msgr2=0x7f61180b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:21.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.427+0000 7f611e856640 1 -- 172.21.15.60:0/720368115 shutdown_connections 2023-11-18T20:44:21.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.427+0000 7f611e856640 1 -- 172.21.15.60:0/720368115 wait complete. 2023-11-18T20:44:21.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.428+0000 7f611e856640 1 Processor -- start 2023-11-18T20:44:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.428+0000 7f611e856640 1 -- start start 2023-11-18T20:44:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f611e856640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61000052a0 0x7f611815fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f611e856640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61181343a0 0x7f6118134820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f611e856640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61180b4330 0x7f61180b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f611e856640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61181555e0 con 0x7f61181343a0 2023-11-18T20:44:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f611e856640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6118134d60 con 0x7f61000052a0 2023-11-18T20:44:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f611e856640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6118154960 con 0x7f61180b4330 2023-11-18T20:44:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f61177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61181343a0 0x7f6118134820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f61177fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61181343a0 0x7f6118134820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35032/0 (socket says 172.21.15.60:35032) 2023-11-18T20:44:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f61177fe640 1 -- 172.21.15.60:0/2314022149 learned_addr learned my addr 172.21.15.60:0/2314022149 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f6117fff640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61000052a0 0x7f611815fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f611cdf3640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61180b4330 0x7f61180b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f61177fe640 1 -- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61180b4330 msgr2=0x7f61180b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f61177fe640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61180b4330 0x7f61180b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f61177fe640 1 -- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61000052a0 msgr2=0x7f611815fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f61177fe640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61000052a0 0x7f611815fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f61177fe640 1 -- 172.21.15.60:0/2314022149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6118154c40 con 0x7f61181343a0 2023-11-18T20:44:21.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f6117fff640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61000052a0 0x7f611815fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f611cdf3640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61180b4330 0x7f61180b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f61177fe640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61181343a0 0x7f6118134820 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f61080072c0 tx=0x7f6108007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f6114ff9640 1 -- 172.21.15.60:0/2314022149 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610800f960 con 0x7f61181343a0 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f6114ff9640 1 -- 172.21.15.60:0/2314022149 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610800fac0 con 0x7f61181343a0 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f611e856640 1 -- 172.21.15.60:0/2314022149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6118154f20 con 0x7f61181343a0 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f6114ff9640 1 -- 172.21.15.60:0/2314022149 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610800fe70 con 0x7f61181343a0 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.429+0000 7f611e856640 1 -- 172.21.15.60:0/2314022149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61180b4cf0 con 0x7f61181343a0 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.430+0000 7f611e856640 1 -- 172.21.15.60:0/2314022149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6118155720 con 0x7f61181343a0 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.431+0000 7f6114ff9640 1 -- 172.21.15.60:0/2314022149 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f610801d460 con 0x7f61181343a0 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.431+0000 7f6114ff9640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f610406ae40 0x7f610406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.437 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.431+0000 7f6114ff9640 1 -- 172.21.15.60:0/2314022149 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6104+0+0 (secure 0 0 0) 0x7f61080898b0 con 0x7f61181343a0 2023-11-18T20:44:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.431+0000 7f6117fff640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f610406ae40 0x7f610406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.432+0000 7f6117fff640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f610406ae40 0x7f610406d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f610c005580 tx=0x7f610c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:21.438 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.436+0000 7f6114ff9640 1 -- 172.21.15.60:0/2314022149 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6118155720 con 0x7f61181343a0 2023-11-18T20:44:21.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.648+0000 7f611e856640 1 -- 172.21.15.60:0/2314022149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6118155720 con 0x7f61181343a0 2023-11-18T20:44:21.649 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-11-18T20:44:21.648+0000 7fe66ea9a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-18T20:44:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.650+0000 7f6114ff9640 1 -- 172.21.15.60:0/2314022149 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6118155720 con 0x7f61181343a0 2023-11-18T20:44:21.651 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-18T20:44:21.653 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.652+0000 7f611e856640 1 -- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f610406ae40 msgr2=0x7f610406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.652+0000 7f611e856640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f610406ae40 0x7f610406d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f610c005580 tx=0x7f610c009040 comp rx=0 tx=0).stop 2023-11-18T20:44:21.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.652+0000 7f611e856640 1 -- 172.21.15.60:0/2314022149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61181343a0 msgr2=0x7f6118134820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.652+0000 7f611e856640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61181343a0 0x7f6118134820 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f61080072c0 tx=0x7f6108007780 comp rx=0 tx=0).stop 2023-11-18T20:44:21.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.652+0000 7f611e856640 1 -- 172.21.15.60:0/2314022149 shutdown_connections 2023-11-18T20:44:21.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.652+0000 7f611e856640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f610406ae40 0x7f610406d300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.652+0000 7f611e856640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61180b4330 0x7f61180b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.652+0000 7f611e856640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61181343a0 0x7f6118134820 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.652+0000 7f611e856640 1 --2- 172.21.15.60:0/2314022149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61000052a0 0x7f611815fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.652+0000 7f611e856640 1 -- 172.21.15.60:0/2314022149 >> 172.21.15.60:0/2314022149 conn(0x7f61180b1e50 msgr2=0x7f611813f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:21.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.653+0000 7f611e856640 1 -- 172.21.15.60:0/2314022149 shutdown_connections 2023-11-18T20:44:21.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.653+0000 7f611e856640 1 -- 172.21.15.60:0/2314022149 wait complete. 2023-11-18T20:44:21.656 INFO:tasks.ceph.mds.c.smithi060.stderr: -3454> 2023-11-18T20:43:38.766+0000 7f7b99893640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-18T20:44:21.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-18T20:44:21.685 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-11-18T20:44:21.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.773+0000 7f1404c76640 1 Processor -- start 2023-11-18T20:44:21.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.773+0000 7f1404c76640 1 -- start start 2023-11-18T20:44:21.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.774+0000 7f1404c76640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000b4370 0x7f14000b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.774+0000 7f1404c76640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1400071770 con 0x7f14000ceb70 2023-11-18T20:44:21.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.774+0000 7f1404c76640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14000ccb60 con 0x7f1400144980 2023-11-18T20:44:21.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.774+0000 7f1404c76640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140015ced0 con 0x7f14000b4370 2023-11-18T20:44:21.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.774+0000 7f13fdd9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f14000ceb70 0x7f14000cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49434/0 (socket says 172.21.15.60:49434) 2023-11-18T20:44:21.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.774+0000 7f13fdd9b640 1 -- 172.21.15.60:0/1161799693 learned_addr learned my addr 172.21.15.60:0/1161799693 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:21.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.774+0000 7f13fed9d640 1 --2- 172.21.15.60:0/1161799693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000b4370 0x7f14000b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.774+0000 7f13fd59a640 1 -- 172.21.15.60:0/1161799693 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582441774 0 0) 0x7f1400071770 con 0x7f14000ceb70 2023-11-18T20:44:21.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.774+0000 7f13fd59a640 1 -- 172.21.15.60:0/1161799693 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13ec003520 con 0x7f14000ceb70 2023-11-18T20:44:21.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.774+0000 7f13fd59a640 1 -- 172.21.15.60:0/1161799693 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780271753 0 0) 0x7f14000ccb60 con 0x7f1400144980 2023-11-18T20:44:21.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.775+0000 7f13fd59a640 1 -- 172.21.15.60:0/1161799693 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1400071770 con 0x7f1400144980 2023-11-18T20:44:21.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.775+0000 7f13fd59a640 1 -- 172.21.15.60:0/1161799693 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1205413944 0 0) 0x7f13ec003520 con 0x7f14000ceb70 2023-11-18T20:44:21.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.775+0000 7f13fed9d640 1 -- 172.21.15.60:0/1161799693 >> v1:172.21.15.100:6789/0 conn(0x7f1400144980 legacy=0x7f140015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.775+0000 7f13fed9d640 1 -- 172.21.15.60:0/1161799693 >> v1:172.21.15.60:6789/0 conn(0x7f14000ceb70 legacy=0x7f14000cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.775+0000 7f13fed9d640 1 -- 172.21.15.60:0/1161799693 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14000b4ce0 con 0x7f14000b4370 2023-11-18T20:44:21.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.775+0000 7f13fed9d640 1 --2- 172.21.15.60:0/1161799693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000b4370 0x7f14000b47a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f14000b4e70 tx=0x7f13f002b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7f4d843b774bdaa server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:21.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.776+0000 7f13fd59a640 1 -- 172.21.15.60:0/1161799693 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8001670 con 0x7f14000b4370 2023-11-18T20:44:21.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.776+0000 7f13fd59a640 1 -- 172.21.15.60:0/1161799693 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f00314d0 con 0x7f14000b4370 2023-11-18T20:44:21.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.776+0000 7f13fd59a640 1 -- 172.21.15.60:0/1161799693 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0031860 con 0x7f14000b4370 2023-11-18T20:44:21.779 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.776+0000 7f1404c76640 1 -- 172.21.15.60:0/1161799693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000b4370 msgr2=0x7f14000b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.776+0000 7f1404c76640 1 --2- 172.21.15.60:0/1161799693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000b4370 0x7f14000b47a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f14000b4e70 tx=0x7f13f002b7f0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.776+0000 7f1404c76640 1 -- 172.21.15.60:0/1161799693 shutdown_connections 2023-11-18T20:44:21.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.776+0000 7f1404c76640 1 --2- 172.21.15.60:0/1161799693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000b4370 0x7f14000b47a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.776+0000 7f1404c76640 1 -- 172.21.15.60:0/1161799693 >> 172.21.15.60:0/1161799693 conn(0x7f140006a4d0 msgr2=0x7f140006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:21.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.776+0000 7f1404c76640 1 -- 172.21.15.60:0/1161799693 shutdown_connections 2023-11-18T20:44:21.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.776+0000 7f1404c76640 1 -- 172.21.15.60:0/1161799693 wait complete. 2023-11-18T20:44:21.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.777+0000 7f1404c76640 1 Processor -- start 2023-11-18T20:44:21.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.778+0000 7f1404c76640 1 -- start start 2023-11-18T20:44:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.778+0000 7f1404c76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14000b4370 0x7f14000b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.778+0000 7f1404c76640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000ceb70 0x7f140013afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.778+0000 7f1404c76640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1400144980 0x7f14001f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.778+0000 7f1404c76640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b3430 con 0x7f1400144980 2023-11-18T20:44:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.778+0000 7f1404c76640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b2820 con 0x7f14000b4370 2023-11-18T20:44:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.778+0000 7f1404c76640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b2b00 con 0x7f14000ceb70 2023-11-18T20:44:21.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.778+0000 7f13fed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1400144980 0x7f14001f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.778+0000 7f13fed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1400144980 0x7f14001f6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35056/0 (socket says 172.21.15.60:35056) 2023-11-18T20:44:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.778+0000 7f13fed9d640 1 -- 172.21.15.60:0/3912121991 learned_addr learned my addr 172.21.15.60:0/3912121991 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13fdd9b640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000ceb70 0x7f140013afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13fe59c640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14000b4370 0x7f14000b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13fed9d640 1 -- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000ceb70 msgr2=0x7f140013afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13fed9d640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000ceb70 0x7f140013afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13fed9d640 1 -- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14000b4370 msgr2=0x7f14000b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13fed9d640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14000b4370 0x7f14000b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13fed9d640 1 -- 172.21.15.60:0/3912121991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0009400 con 0x7f1400144980 2023-11-18T20:44:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13fdd9b640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000ceb70 0x7f140013afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13fe59c640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14000b4370 0x7f14000b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13fed9d640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1400144980 0x7f14001f6250 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f13f0037040 tx=0x7f13f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:21.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13f6ffd640 1 -- 172.21.15.60:0/3912121991 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0004230 con 0x7f1400144980 2023-11-18T20:44:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13f6ffd640 1 -- 172.21.15.60:0/3912121991 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f0004390 con 0x7f1400144980 2023-11-18T20:44:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f1404c76640 1 -- 172.21.15.60:0/3912121991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14000b2de0 con 0x7f1400144980 2023-11-18T20:44:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f13f6ffd640 1 -- 172.21.15.60:0/3912121991 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00348b0 con 0x7f1400144980 2023-11-18T20:44:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.779+0000 7f1404c76640 1 -- 172.21.15.60:0/3912121991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14001f6790 con 0x7f1400144980 2023-11-18T20:44:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.780+0000 7f1404c76640 1 -- 172.21.15.60:0/3912121991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14001f6950 con 0x7f1400144980 2023-11-18T20:44:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.780+0000 7f13f6ffd640 1 -- 172.21.15.60:0/3912121991 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f13f003b040 con 0x7f1400144980 2023-11-18T20:44:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.781+0000 7f13f6ffd640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f13e006ae40 0x7f13e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:21.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.781+0000 7f13f6ffd640 1 -- 172.21.15.60:0/3912121991 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6190+0+0 (secure 0 0 0) 0x7f13f00aafa0 con 0x7f1400144980 2023-11-18T20:44:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.781+0000 7f13fe59c640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f13e006ae40 0x7f13e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.782+0000 7f13fe59c640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f13e006ae40 0x7f13e006d300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13f8007fd0 tx=0x7f13f8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:21.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:21.785+0000 7f13f6ffd640 1 -- 172.21.15.60:0/3912121991 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13f00dde90 con 0x7f1400144980 2023-11-18T20:44:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.002+0000 7f1404c76640 1 -- 172.21.15.60:0/3912121991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1400069400 con 0x7f1400144980 2023-11-18T20:44:22.652 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.651+0000 7f13f6ffd640 1 -- 172.21.15.60:0/3912121991 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f13f0075f00 con 0x7f1400144980 2023-11-18T20:44:22.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 -- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f13e006ae40 msgr2=0x7f13e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:22.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f13e006ae40 0x7f13e006d300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13f8007fd0 tx=0x7f13f8009040 comp rx=0 tx=0).stop 2023-11-18T20:44:22.655 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 -- 172.21.15.60:0/3912121991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1400144980 msgr2=0x7f14001f6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1400144980 0x7f14001f6250 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f13f0037040 tx=0x7f13f00046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 -- 172.21.15.60:0/3912121991 shutdown_connections 2023-11-18T20:44:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f13e006ae40 0x7f13e006d300 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1400144980 0x7f14001f6250 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14000ceb70 0x7f140013afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 --2- 172.21.15.60:0/3912121991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14000b4370 0x7f14000b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 -- 172.21.15.60:0/3912121991 >> 172.21.15.60:0/3912121991 conn(0x7f140006a4d0 msgr2=0x7f140006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 -- 172.21.15.60:0/3912121991 shutdown_connections 2023-11-18T20:44:22.656 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.654+0000 7f1404c76640 1 -- 172.21.15.60:0/3912121991 wait complete. 2023-11-18T20:44:22.669 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-18T20:44:22.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.826+0000 7fc8bcc90640 1 Processor -- start 2023-11-18T20:44:22.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.826+0000 7fc8bcc90640 1 -- start start 2023-11-18T20:44:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.827+0000 7fc8bcc90640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc8b80d3900 0x7fc8b80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.827+0000 7fc8bcc90640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b80b2b10 con 0x7fc8b815d330 2023-11-18T20:44:22.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.827+0000 7fc8bcc90640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b80d1590 con 0x7fc8b80d2910 2023-11-18T20:44:22.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.827+0000 7fc8bcc90640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b80d1870 con 0x7fc8b80d3900 2023-11-18T20:44:22.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.827+0000 7fc8b659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc8b815d330 0x7fc8b80b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49442/0 (socket says 172.21.15.60:49442) 2023-11-18T20:44:22.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.827+0000 7fc8b659c640 1 -- 172.21.15.60:0/1368450644 learned_addr learned my addr 172.21.15.60:0/1368450644 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:22.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.827+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912943895 0 0) 0x7fc8b80b2b10 con 0x7fc8b815d330 2023-11-18T20:44:22.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.827+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8a0003520 con 0x7fc8b815d330 2023-11-18T20:44:22.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.827+0000 7fc8b6d9d640 1 --2- 172.21.15.60:0/1368450644 >> v2:172.21.15.100:3300/0 conn(0x7fc8b80d3900 0x7fc8b80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:22.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1138693600 0 0) 0x7fc8a0003520 con 0x7fc8b815d330 2023-11-18T20:44:22.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 >> v2:172.21.15.100:3300/0 conn(0x7fc8b80d3900 msgr2=0x7fc8b80b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:22.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 --2- 172.21.15.60:0/1368450644 >> v2:172.21.15.100:3300/0 conn(0x7fc8b80d3900 0x7fc8b80b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 >> v1:172.21.15.100:6789/0 conn(0x7fc8b80d2910 legacy=0x7fc8b80d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b80d1b50 con 0x7fc8b815d330 2023-11-18T20:44:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fc8a8001670 con 0x7fc8b815d330 2023-11-18T20:44:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 >> v1:172.21.15.60:6789/0 conn(0x7fc8b815d330 legacy=0x7fc8b80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:22.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a0003970 0x7fc8a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:22.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a00047c0 0x7fc8a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:22.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8a0005180 0x7fc8a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:22.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80d1870 con 0x7fc8a00047c0 2023-11-18T20:44:22.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a000c120 con 0x7fc8a0005180 2023-11-18T20:44:22.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a000c2e0 con 0x7fc8a0003970 2023-11-18T20:44:22.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b5d9b640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a00047c0 0x7fc8a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:22.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b659c640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a0003970 0x7fc8a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:22.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b6d9d640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8a0005180 0x7fc8a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:22.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b5d9b640 1 -- 172.21.15.60:0/1368450644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a0003970 msgr2=0x7fc8a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b5d9b640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a0003970 0x7fc8a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b5d9b640 1 -- 172.21.15.60:0/1368450644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8a0005180 msgr2=0x7fc8a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b5d9b640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8a0005180 0x7fc8a000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b5d9b640 1 -- 172.21.15.60:0/1368450644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a0003520 con 0x7fc8a00047c0 2023-11-18T20:44:22.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.828+0000 7fc8b6d9d640 1 -- 172.21.15.60:0/1368450644 reap_dead start 2023-11-18T20:44:22.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.829+0000 7fc8b5d9b640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a00047c0 0x7fc8a0004c40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac001750 tx=0x7fc8a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c70c869ef62c1b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:22.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.829+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac0052d0 con 0x7fc8a00047c0 2023-11-18T20:44:22.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.829+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac02e050 con 0x7fc8a00047c0 2023-11-18T20:44:22.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.829+0000 7fc8b559a640 1 -- 172.21.15.60:0/1368450644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac003830 con 0x7fc8a00047c0 2023-11-18T20:44:22.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.830+0000 7fc8bcc90640 1 -- 172.21.15.60:0/1368450644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a00047c0 msgr2=0x7fc8a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:22.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.830+0000 7fc8bcc90640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a00047c0 0x7fc8a0004c40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac001750 tx=0x7fc8a00057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.830+0000 7fc8bcc90640 1 -- 172.21.15.60:0/1368450644 shutdown_connections 2023-11-18T20:44:22.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.830+0000 7fc8bcc90640 1 --2- 172.21.15.60:0/1368450644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a00047c0 0x7fc8a0004c40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.830+0000 7fc8bcc90640 1 -- 172.21.15.60:0/1368450644 >> 172.21.15.60:0/1368450644 conn(0x7fc8b8072020 msgr2=0x7fc8b8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:22.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.830+0000 7fc8bcc90640 1 -- 172.21.15.60:0/1368450644 shutdown_connections 2023-11-18T20:44:22.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.830+0000 7fc8bcc90640 1 -- 172.21.15.60:0/1368450644 wait complete. 2023-11-18T20:44:22.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.831+0000 7fc8bcc90640 1 Processor -- start 2023-11-18T20:44:22.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.831+0000 7fc8bcc90640 1 -- start start 2023-11-18T20:44:22.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8bcc90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a00047c0 0x7fc8b80c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:22.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8bcc90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b80c7bf0 0x7fc8b8156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:22.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8bcc90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b8156f40 0x7fc8b8143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:22.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8bcc90640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b81573c0 con 0x7fc8b8156f40 2023-11-18T20:44:22.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8bcc90640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80c80d0 con 0x7fc8b80c7bf0 2023-11-18T20:44:22.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8bcc90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80c83b0 con 0x7fc8a00047c0 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b8156f40 0x7fc8b8143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b8156f40 0x7fc8b8143620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35072/0 (socket says 172.21.15.60:35072) 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b6d9d640 1 -- 172.21.15.60:0/2698120088 learned_addr learned my addr 172.21.15.60:0/2698120088 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b5d9b640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b80c7bf0 0x7fc8b8156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b659c640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a00047c0 0x7fc8b80c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b6d9d640 1 -- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a00047c0 msgr2=0x7fc8b80c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b6d9d640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a00047c0 0x7fc8b80c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b6d9d640 1 -- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b80c7bf0 msgr2=0x7fc8b8156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b6d9d640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b80c7bf0 0x7fc8b8156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b6d9d640 1 -- 172.21.15.60:0/2698120088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b80c8690 con 0x7fc8b8156f40 2023-11-18T20:44:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.832+0000 7fc8b5d9b640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b80c7bf0 0x7fc8b8156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:22.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.833+0000 7fc8b6d9d640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b8156f40 0x7fc8b8143620 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80d1510 tx=0x7fc8b000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:22.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.833+0000 7fc8a6ffd640 1 -- 172.21.15.60:0/2698120088 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000e040 con 0x7fc8b8156f40 2023-11-18T20:44:22.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.833+0000 7fc8a6ffd640 1 -- 172.21.15.60:0/2698120088 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b000f040 con 0x7fc8b8156f40 2023-11-18T20:44:22.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.833+0000 7fc8bcc90640 1 -- 172.21.15.60:0/2698120088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b80c8970 con 0x7fc8b8156f40 2023-11-18T20:44:22.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.833+0000 7fc8a6ffd640 1 -- 172.21.15.60:0/2698120088 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0015430 con 0x7fc8b8156f40 2023-11-18T20:44:22.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.833+0000 7fc8bcc90640 1 -- 172.21.15.60:0/2698120088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b8143b60 con 0x7fc8b8156f40 2023-11-18T20:44:22.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.834+0000 7fc8bcc90640 1 -- 172.21.15.60:0/2698120088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b80c8e90 con 0x7fc8b8156f40 2023-11-18T20:44:22.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.834+0000 7fc8a6ffd640 1 -- 172.21.15.60:0/2698120088 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc8b0015590 con 0x7fc8b8156f40 2023-11-18T20:44:22.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.835+0000 7fc8a6ffd640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc89806ae70 0x7fc89806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:22.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.835+0000 7fc8a6ffd640 1 -- 172.21.15.60:0/2698120088 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6190+0+0 (secure 0 0 0) 0x7fc8b0089ff0 con 0x7fc8b8156f40 2023-11-18T20:44:22.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.835+0000 7fc8b659c640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc89806ae70 0x7fc89806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:22.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.836+0000 7fc8b659c640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc89806ae70 0x7fc89806d330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8007c40 tx=0x7fc8a8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:22.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:22.839+0000 7fc8a6ffd640 1 -- 172.21.15.60:0/2698120088 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8b80c8e90 con 0x7fc8b8156f40 2023-11-18T20:44:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.016+0000 7fc8bcc90640 1 -- 172.21.15.60:0/2698120088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8b80c8e90 con 0x7fc8b8156f40 2023-11-18T20:44:23.654 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.653+0000 7fc8a6ffd640 1 -- 172.21.15.60:0/2698120088 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc8b80c8e90 con 0x7fc8b8156f40 2023-11-18T20:44:23.654 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-11-18T20:44:23.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 -- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc89806ae70 msgr2=0x7fc89806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:23.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc89806ae70 0x7fc89806d330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8007c40 tx=0x7fc8a8009040 comp rx=0 tx=0).stop 2023-11-18T20:44:23.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 -- 172.21.15.60:0/2698120088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b8156f40 msgr2=0x7fc8b8143620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:23.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b8156f40 0x7fc8b8143620 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80d1510 tx=0x7fc8b000f580 comp rx=0 tx=0).stop 2023-11-18T20:44:23.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 -- 172.21.15.60:0/2698120088 shutdown_connections 2023-11-18T20:44:23.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc89806ae70 0x7fc89806d330 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:23.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8b8156f40 0x7fc8b8143620 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:23.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8b80c7bf0 0x7fc8b8156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:23.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 --2- 172.21.15.60:0/2698120088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a00047c0 0x7fc8b80c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:23.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 -- 172.21.15.60:0/2698120088 >> 172.21.15.60:0/2698120088 conn(0x7fc8b8072020 msgr2=0x7fc8b815de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:23.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 -- 172.21.15.60:0/2698120088 shutdown_connections 2023-11-18T20:44:23.658 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.656+0000 7fc8bcc90640 1 -- 172.21.15.60:0/2698120088 wait complete. 2023-11-18T20:44:23.673 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-18T20:44:23.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.786+0000 7f58fcb61640 1 Processor -- start 2023-11-18T20:44:23.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.786+0000 7f58fcb61640 1 -- start start 2023-11-18T20:44:23.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.786+0000 7f58fcb61640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58f806ea50 0x7f58f806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:23.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58fcb61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58f8160fb0 0x7f58f8161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:23.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58fcb61640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f8161d20 con 0x7f58f80b3ff0 2023-11-18T20:44:23.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58fcb61640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f80d1ee0 con 0x7f58f806ea50 2023-11-18T20:44:23.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58fcb61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f806f370 con 0x7f58f8160fb0 2023-11-18T20:44:23.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58f659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58f806ea50 0x7f58f806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:23.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58f659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58f806ea50 0x7f58f806ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35084/0 (socket says 172.21.15.60:35084) 2023-11-18T20:44:23.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58f659c640 1 -- 172.21.15.60:0/3780339839 learned_addr learned my addr 172.21.15.60:0/3780339839 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:23.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58f659c640 1 -- 172.21.15.60:0/3780339839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58f8160fb0 msgr2=0x7f58f8161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:23.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58f659c640 1 --2- 172.21.15.60:0/3780339839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58f8160fb0 0x7f58f8161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:23.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58f659c640 1 -- 172.21.15.60:0/3780339839 >> v1:172.21.15.100:6789/0 conn(0x7f58f80b3ff0 legacy=0x7f58f80b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:23.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.787+0000 7f58f659c640 1 -- 172.21.15.60:0/3780339839 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f815e760 con 0x7f58f806ea50 2023-11-18T20:44:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.788+0000 7f58f659c640 1 --2- 172.21.15.60:0/3780339839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f806ea50 0x7f58f806ee30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009870 tx=0x7f58e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee07977030b3dcab server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.788+0000 7f58f5d9b640 1 -- 172.21.15.60:0/3780339839 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8031640 con 0x7f58f806ea50 2023-11-18T20:44:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.788+0000 7f58f5d9b640 1 -- 172.21.15.60:0/3780339839 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e80317a0 con 0x7f58f806ea50 2023-11-18T20:44:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.788+0000 7f58f5d9b640 1 -- 172.21.15.60:0/3780339839 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8031b30 con 0x7f58f806ea50 2023-11-18T20:44:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.788+0000 7f58fcb61640 1 -- 172.21.15.60:0/3780339839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f806ea50 msgr2=0x7f58f806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.788+0000 7f58fcb61640 1 --2- 172.21.15.60:0/3780339839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f806ea50 0x7f58f806ee30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009870 tx=0x7f58e802b8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.788+0000 7f58fcb61640 1 -- 172.21.15.60:0/3780339839 shutdown_connections 2023-11-18T20:44:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.788+0000 7f58fcb61640 1 --2- 172.21.15.60:0/3780339839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58f8160fb0 0x7f58f8161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.788+0000 7f58fcb61640 1 --2- 172.21.15.60:0/3780339839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f806ea50 0x7f58f806ee30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.789+0000 7f58fcb61640 1 -- 172.21.15.60:0/3780339839 >> 172.21.15.60:0/3780339839 conn(0x7f58f806a6b0 msgr2=0x7f58f806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.789+0000 7f58fcb61640 1 -- 172.21.15.60:0/3780339839 shutdown_connections 2023-11-18T20:44:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.789+0000 7f58fcb61640 1 -- 172.21.15.60:0/3780339839 wait complete. 2023-11-18T20:44:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.790+0000 7f58fcb61640 1 Processor -- start 2023-11-18T20:44:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.790+0000 7f58fcb61640 1 -- start start 2023-11-18T20:44:23.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58fcb61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58f806ea50 0x7f58f813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58fcb61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f80b3ff0 0x7f58f81584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58fcb61640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58f8160fb0 0x7f58f81589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58fcb61640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81e7d00 con 0x7f58f80b3ff0 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58fcb61640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f813f930 con 0x7f58f8160fb0 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58fcb61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81f3490 con 0x7f58f806ea50 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f80b3ff0 0x7f58f81584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f80b3ff0 0x7f58f81584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35100/0 (socket says 172.21.15.60:35100) 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58effff640 1 -- 172.21.15.60:0/68491316 learned_addr learned my addr 172.21.15.60:0/68491316 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58f659c640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58f806ea50 0x7f58f813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58effff640 1 -- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58f806ea50 msgr2=0x7f58f813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58effff640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58f806ea50 0x7f58f813f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58effff640 1 -- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58f8160fb0 msgr2=0x7f58f81589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58effff640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58f8160fb0 0x7f58f81589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58effff640 1 -- 172.21.15.60:0/68491316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80094d0 con 0x7f58f80b3ff0 2023-11-18T20:44:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58f659c640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58f806ea50 0x7f58f813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58effff640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f80b3ff0 0x7f58f81584a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b4a0 tx=0x7f58e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58eeffd640 1 -- 172.21.15.60:0/68491316 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0010610 con 0x7f58f80b3ff0 2023-11-18T20:44:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58eeffd640 1 -- 172.21.15.60:0/68491316 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e0010770 con 0x7f58f80b3ff0 2023-11-18T20:44:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58fcb61640 1 -- 172.21.15.60:0/68491316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f81e7080 con 0x7f58f80b3ff0 2023-11-18T20:44:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58eeffd640 1 -- 172.21.15.60:0/68491316 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0010b20 con 0x7f58f80b3ff0 2023-11-18T20:44:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.791+0000 7f58fcb61640 1 -- 172.21.15.60:0/68491316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81e7640 con 0x7f58f80b3ff0 2023-11-18T20:44:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.792+0000 7f58fcb61640 1 -- 172.21.15.60:0/68491316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f81e7e40 con 0x7f58f80b3ff0 2023-11-18T20:44:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.793+0000 7f58eeffd640 1 -- 172.21.15.60:0/68491316 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58e0010c80 con 0x7f58f80b3ff0 2023-11-18T20:44:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.793+0000 7f58eeffd640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58d806ae40 0x7f58d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.794+0000 7f58eeffd640 1 -- 172.21.15.60:0/68491316 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5752+0+0 (secure 0 0 0) 0x7f58e0089230 con 0x7f58f80b3ff0 2023-11-18T20:44:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.794+0000 7f58f659c640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58d806ae40 0x7f58d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:23.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.794+0000 7f58f659c640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58d806ae40 0x7f58d806d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f58e8036000 tx=0x7f58e803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:23.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.798+0000 7f58eeffd640 1 -- 172.21.15.60:0/68491316 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58f81e7e40 con 0x7f58f80b3ff0 2023-11-18T20:44:23.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:23.973+0000 7f58fcb61640 1 -- 172.21.15.60:0/68491316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f58f81e7e40 con 0x7f58f80b3ff0 2023-11-18T20:44:24.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.658+0000 7f58eeffd640 1 -- 172.21.15.60:0/68491316 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7f58f81e7e40 con 0x7f58f80b3ff0 2023-11-18T20:44:24.660 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-11-18T20:44:24.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.665+0000 7f58fcb61640 1 -- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58d806ae40 msgr2=0x7f58d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:24.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.665+0000 7f58fcb61640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58d806ae40 0x7f58d806d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f58e8036000 tx=0x7f58e803c040 comp rx=0 tx=0).stop 2023-11-18T20:44:24.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.666+0000 7f58fcb61640 1 -- 172.21.15.60:0/68491316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f80b3ff0 msgr2=0x7f58f81584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:24.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.666+0000 7f58fcb61640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f80b3ff0 0x7f58f81584a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b4a0 tx=0x7f58e000b960 comp rx=0 tx=0).stop 2023-11-18T20:44:24.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.666+0000 7f58fcb61640 1 -- 172.21.15.60:0/68491316 shutdown_connections 2023-11-18T20:44:24.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.666+0000 7f58fcb61640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58d806ae40 0x7f58d806d300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:24.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.666+0000 7f58fcb61640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58f8160fb0 0x7f58f81589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:24.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.666+0000 7f58fcb61640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58f80b3ff0 0x7f58f81584a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:24.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.666+0000 7f58fcb61640 1 --2- 172.21.15.60:0/68491316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58f806ea50 0x7f58f813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:24.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.666+0000 7f58fcb61640 1 -- 172.21.15.60:0/68491316 >> 172.21.15.60:0/68491316 conn(0x7f58f806a6b0 msgr2=0x7f58f80b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:24.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.667+0000 7f58fcb61640 1 -- 172.21.15.60:0/68491316 shutdown_connections 2023-11-18T20:44:24.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.667+0000 7f58fcb61640 1 -- 172.21.15.60:0/68491316 wait complete. 2023-11-18T20:44:24.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2023-11-18T20:44:24.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.798+0000 7f68b6826640 1 Processor -- start 2023-11-18T20:44:24.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.798+0000 7f68b6826640 1 -- start start 2023-11-18T20:44:24.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.799+0000 7f68b6826640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 0x7f68b00aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:24.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.799+0000 7f68b6826640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68b00b4460 0x7f68b006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:24.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.799+0000 7f68b6826640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b00ada20 con 0x7f68b00b2b10 2023-11-18T20:44:24.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.799+0000 7f68b6826640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b00ae6a0 con 0x7f68b00b4460 2023-11-18T20:44:24.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.799+0000 7f68b6826640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b00add00 con 0x7f68b015d140 2023-11-18T20:44:24.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68affff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 0x7f68b00aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:24.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68af7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68b00b4460 0x7f68b006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:24.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68af7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68b00b4460 0x7f68b006b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58512/0 (socket says 172.21.15.60:58512) 2023-11-18T20:44:24.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68affff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 0x7f68b00aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:34586/0 (socket says 172.21.15.60:34586) 2023-11-18T20:44:24.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68af7fe640 1 -- 172.21.15.60:0/2980481755 learned_addr learned my addr 172.21.15.60:0/2980481755 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:24.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68af7fe640 1 -- 172.21.15.60:0/2980481755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 msgr2=0x7f68b00aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:24.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68af7fe640 1 --2- 172.21.15.60:0/2980481755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 0x7f68b00aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:24.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68af7fe640 1 -- 172.21.15.60:0/2980481755 >> v1:172.21.15.100:6789/0 conn(0x7f68b00b2b10 legacy=0x7f68b006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:24.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68af7fe640 1 -- 172.21.15.60:0/2980481755 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b00adfe0 con 0x7f68b00b4460 2023-11-18T20:44:24.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68aeffd640 1 -- 172.21.15.60:0/2980481755 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687870111 0 0) 0x7f68b00ada20 con 0x7f68b00b2b10 2023-11-18T20:44:24.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.800+0000 7f68affff640 1 --2- 172.21.15.60:0/2980481755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 0x7f68b00aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:24.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.801+0000 7f68af7fe640 1 --2- 172.21.15.60:0/2980481755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b00b4460 0x7f68b006b4e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009870 tx=0x7f68a002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40db7ae70f5704e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:24.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.801+0000 7f68aeffd640 1 -- 172.21.15.60:0/2980481755 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0031640 con 0x7f68b00b4460 2023-11-18T20:44:24.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.801+0000 7f68aeffd640 1 -- 172.21.15.60:0/2980481755 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a00317a0 con 0x7f68b00b4460 2023-11-18T20:44:24.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.801+0000 7f68aeffd640 1 -- 172.21.15.60:0/2980481755 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0031b30 con 0x7f68b00b4460 2023-11-18T20:44:24.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.801+0000 7f68b6826640 1 -- 172.21.15.60:0/2980481755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b00b4460 msgr2=0x7f68b006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:24.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.801+0000 7f68b6826640 1 --2- 172.21.15.60:0/2980481755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b00b4460 0x7f68b006b4e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009870 tx=0x7f68a002b8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:24.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.802+0000 7f68b6826640 1 -- 172.21.15.60:0/2980481755 shutdown_connections 2023-11-18T20:44:24.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.802+0000 7f68b6826640 1 --2- 172.21.15.60:0/2980481755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b00b4460 0x7f68b006b4e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:24.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.802+0000 7f68b6826640 1 --2- 172.21.15.60:0/2980481755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 0x7f68b00aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:24.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.802+0000 7f68b6826640 1 -- 172.21.15.60:0/2980481755 >> 172.21.15.60:0/2980481755 conn(0x7f68b0070e70 msgr2=0x7f68b0071280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:24.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.802+0000 7f68b6826640 1 -- 172.21.15.60:0/2980481755 shutdown_connections 2023-11-18T20:44:24.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.802+0000 7f68b6826640 1 -- 172.21.15.60:0/2980481755 wait complete. 2023-11-18T20:44:24.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.803+0000 7f68b6826640 1 Processor -- start 2023-11-18T20:44:24.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.803+0000 7f68b6826640 1 -- start start 2023-11-18T20:44:24.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68b6826640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68b00b2b10 0x7f68b0157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68b6826640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b00b4460 0x7f68b01f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68b6826640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 0x7f68b014e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68b6826640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01f6220 con 0x7f68b00b2b10 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68b6826640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01f64c0 con 0x7f68b00b4460 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68b6826640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01f67a0 con 0x7f68b015d140 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68b00b2b10 0x7f68b0157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68b00b2b10 0x7f68b0157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35110/0 (socket says 172.21.15.60:35110) 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68affff640 1 -- 172.21.15.60:0/535928499 learned_addr learned my addr 172.21.15.60:0/535928499 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68b4dc3640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 0x7f68b014e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68af7fe640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b00b4460 0x7f68b01f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:24.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68affff640 1 -- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 msgr2=0x7f68b014e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68affff640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 0x7f68b014e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68affff640 1 -- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b00b4460 msgr2=0x7f68b01f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68affff640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b00b4460 0x7f68b01f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68affff640 1 -- 172.21.15.60:0/535928499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4009400 con 0x7f68b00b2b10 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.804+0000 7f68affff640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68b00b2b10 0x7f68b0157470 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f68a400ebe0 tx=0x7f68a400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.805+0000 7f68acff9640 1 -- 172.21.15.60:0/535928499 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4009760 con 0x7f68b00b2b10 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.805+0000 7f68acff9640 1 -- 172.21.15.60:0/535928499 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a400cc30 con 0x7f68b00b2b10 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.805+0000 7f68b6826640 1 -- 172.21.15.60:0/535928499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a00094d0 con 0x7f68b00b2b10 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.805+0000 7f68acff9640 1 -- 172.21.15.60:0/535928499 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a400eee0 con 0x7f68b00b2b10 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.805+0000 7f68b6826640 1 -- 172.21.15.60:0/535928499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b0159170 con 0x7f68b00b2b10 2023-11-18T20:44:24.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.805+0000 7f68b6826640 1 -- 172.21.15.60:0/535928499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b0069400 con 0x7f68b00b2b10 2023-11-18T20:44:24.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.806+0000 7f68acff9640 1 -- 172.21.15.60:0/535928499 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68a4013470 con 0x7f68b00b2b10 2023-11-18T20:44:24.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.806+0000 7f68acff9640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f689c06ae40 0x7f689c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:24.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.807+0000 7f68acff9640 1 -- 172.21.15.60:0/535928499 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5338+0+0 (secure 0 0 0) 0x7f68a40897a0 con 0x7f68b00b2b10 2023-11-18T20:44:24.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.807+0000 7f68af7fe640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f689c06ae40 0x7f689c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:24.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.807+0000 7f68af7fe640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f689c06ae40 0x7f689c06d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f68a0036000 tx=0x7f68a0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:24.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.811+0000 7f68acff9640 1 -- 172.21.15.60:0/535928499 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68a40549b0 con 0x7f68b00b2b10 2023-11-18T20:44:24.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:24.990+0000 7f68b6826640 1 -- 172.21.15.60:0/535928499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f68b006c010 con 0x7f68b00b2b10 2023-11-18T20:44:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.663+0000 7f68acff9640 1 -- 172.21.15.60:0/535928499 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v41) v1 ==== 160+0+0 (secure 0 0 0) 0x7f68a4057fd0 con 0x7f68b00b2b10 2023-11-18T20:44:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'data_new' removed 2023-11-18T20:44:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.666+0000 7f68b6826640 1 -- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f689c06ae40 msgr2=0x7f689c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.666+0000 7f68b6826640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f689c06ae40 0x7f689c06d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f68a0036000 tx=0x7f68a0008030 comp rx=0 tx=0).stop 2023-11-18T20:44:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.666+0000 7f68b6826640 1 -- 172.21.15.60:0/535928499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68b00b2b10 msgr2=0x7f68b0157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.666+0000 7f68b6826640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68b00b2b10 0x7f68b0157470 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f68a400ebe0 tx=0x7f68a400c520 comp rx=0 tx=0).stop 2023-11-18T20:44:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.667+0000 7f68b6826640 1 -- 172.21.15.60:0/535928499 shutdown_connections 2023-11-18T20:44:25.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.667+0000 7f68b6826640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f689c06ae40 0x7f689c06d300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.667+0000 7f68b6826640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68b015d140 0x7f68b014e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.667+0000 7f68b6826640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68b00b4460 0x7f68b01f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.667+0000 7f68b6826640 1 --2- 172.21.15.60:0/535928499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68b00b2b10 0x7f68b0157470 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.667+0000 7f68b6826640 1 -- 172.21.15.60:0/535928499 >> 172.21.15.60:0/535928499 conn(0x7f68b0070e70 msgr2=0x7f68b00affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.667+0000 7f68b6826640 1 -- 172.21.15.60:0/535928499 shutdown_connections 2023-11-18T20:44:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.667+0000 7f68b6826640 1 -- 172.21.15.60:0/535928499 wait complete. 2023-11-18T20:44:25.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-18T20:44:25.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.797+0000 7f006c8f2640 1 Processor -- start 2023-11-18T20:44:25.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.797+0000 7f006c8f2640 1 -- start start 2023-11-18T20:44:25.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.798+0000 7f006c8f2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00680d31c0 0x7f00680d35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:25.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.798+0000 7f006c8f2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00680d19b0 0x7f00680cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:25.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.798+0000 7f006c8f2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f00680d1df0 0x7f00680ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:25.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.798+0000 7f006c8f2640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00680d2e80 con 0x7f00680d31c0 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.798+0000 7f006c8f2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00680d28a0 con 0x7f00680d1df0 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.798+0000 7f006c8f2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00680d3ae0 con 0x7f00680d19b0 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.798+0000 7f006659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00680d31c0 0x7f00680d35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.798+0000 7f006659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00680d31c0 0x7f00680d35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35124/0 (socket says 172.21.15.60:35124) 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.798+0000 7f006659c640 1 -- 172.21.15.60:0/1737047191 learned_addr learned my addr 172.21.15.60:0/1737047191 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.799+0000 7f0065d9b640 1 --2- 172.21.15.60:0/1737047191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00680d19b0 0x7f00680cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.799+0000 7f0066d9d640 1 --2- 172.21.15.60:0/1737047191 >> v2:172.21.15.100:3300/0 conn(0x7f00680d1df0 0x7f00680ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.799+0000 7f006659c640 1 -- 172.21.15.60:0/1737047191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00680d19b0 msgr2=0x7f00680cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.799+0000 7f006659c640 1 --2- 172.21.15.60:0/1737047191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00680d19b0 0x7f00680cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.799+0000 7f006659c640 1 -- 172.21.15.60:0/1737047191 >> v2:172.21.15.100:3300/0 conn(0x7f00680d1df0 msgr2=0x7f00680ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.799+0000 7f006659c640 1 --2- 172.21.15.60:0/1737047191 >> v2:172.21.15.100:3300/0 conn(0x7f00680d1df0 0x7f00680ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.799+0000 7f006659c640 1 -- 172.21.15.60:0/1737047191 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680cdc50 con 0x7f00680d31c0 2023-11-18T20:44:25.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.799+0000 7f0066d9d640 1 --2- 172.21.15.60:0/1737047191 >> v2:172.21.15.100:3300/0 conn(0x7f00680d1df0 0x7f00680ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:25.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006659c640 1 --2- 172.21.15.60:0/1737047191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d31c0 0x7f00680d35a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f00680cd860 tx=0x7f006002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824f33dbb270a128 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:25.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006559a640 1 -- 172.21.15.60:0/1737047191 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006002ca70 con 0x7f00680d31c0 2023-11-18T20:44:25.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006559a640 1 -- 172.21.15.60:0/1737047191 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006002cbd0 con 0x7f00680d31c0 2023-11-18T20:44:25.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006559a640 1 -- 172.21.15.60:0/1737047191 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600343f0 con 0x7f00680d31c0 2023-11-18T20:44:25.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006c8f2640 1 -- 172.21.15.60:0/1737047191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d31c0 msgr2=0x7f00680d35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:25.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006c8f2640 1 --2- 172.21.15.60:0/1737047191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d31c0 0x7f00680d35a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f00680cd860 tx=0x7f006002bed0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006c8f2640 1 -- 172.21.15.60:0/1737047191 shutdown_connections 2023-11-18T20:44:25.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006c8f2640 1 --2- 172.21.15.60:0/1737047191 >> v2:172.21.15.100:3300/0 conn(0x7f00680d1df0 0x7f00680ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006c8f2640 1 --2- 172.21.15.60:0/1737047191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00680d19b0 0x7f00680cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006c8f2640 1 --2- 172.21.15.60:0/1737047191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d31c0 0x7f00680d35a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006c8f2640 1 -- 172.21.15.60:0/1737047191 >> 172.21.15.60:0/1737047191 conn(0x7f0068072020 msgr2=0x7f0068072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:25.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006c8f2640 1 -- 172.21.15.60:0/1737047191 shutdown_connections 2023-11-18T20:44:25.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.800+0000 7f006c8f2640 1 -- 172.21.15.60:0/1737047191 wait complete. 2023-11-18T20:44:25.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.802+0000 7f006c8f2640 1 Processor -- start 2023-11-18T20:44:25.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.802+0000 7f006c8f2640 1 -- start start 2023-11-18T20:44:25.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.802+0000 7f006c8f2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d19b0 0x7f00680c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:25.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.802+0000 7f006c8f2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00680d1df0 0x7f0068153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.802+0000 7f006c8f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00680d31c0 0x7f00680ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.802+0000 7f006c8f2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00680abb10 con 0x7f00680d19b0 2023-11-18T20:44:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.802+0000 7f006c8f2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00680c9000 con 0x7f00680d31c0 2023-11-18T20:44:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.802+0000 7f006c8f2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0068153cb0 con 0x7f00680d1df0 2023-11-18T20:44:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d19b0 0x7f00680c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d19b0 0x7f00680c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35134/0 (socket says 172.21.15.60:35134) 2023-11-18T20:44:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006659c640 1 -- 172.21.15.60:0/1814748055 learned_addr learned my addr 172.21.15.60:0/1814748055 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f0065d9b640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00680d1df0 0x7f0068153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f0066d9d640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00680d31c0 0x7f00680ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:25.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006659c640 1 -- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00680d1df0 msgr2=0x7f0068153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006659c640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00680d1df0 0x7f0068153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006659c640 1 -- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00680d31c0 msgr2=0x7f00680ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006659c640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00680d31c0 0x7f00680ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006659c640 1 -- 172.21.15.60:0/1814748055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060009b30 con 0x7f00680d19b0 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006659c640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d19b0 0x7f00680c9490 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0060000f80 tx=0x7f0060004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f005effd640 1 -- 172.21.15.60:0/1814748055 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060009510 con 0x7f00680d19b0 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f005effd640 1 -- 172.21.15.60:0/1814748055 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006003e450 con 0x7f00680d19b0 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006c8f2640 1 -- 172.21.15.60:0/1814748055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00680c83f0 con 0x7f00680d19b0 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f005effd640 1 -- 172.21.15.60:0/1814748055 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006003e800 con 0x7f00680d19b0 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.803+0000 7f006c8f2640 1 -- 172.21.15.60:0/1814748055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00680c89b0 con 0x7f00680d19b0 2023-11-18T20:44:25.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.804+0000 7f006c8f2640 1 -- 172.21.15.60:0/1814748055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0068069400 con 0x7f00680d19b0 2023-11-18T20:44:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.805+0000 7f005effd640 1 -- 172.21.15.60:0/1814748055 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f006003e960 con 0x7f00680d19b0 2023-11-18T20:44:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.805+0000 7f005effd640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f004806ae70 0x7f004806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.805+0000 7f005effd640 1 -- 172.21.15.60:0/1814748055 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f00600abfa0 con 0x7f00680d19b0 2023-11-18T20:44:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.806+0000 7f0065d9b640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f004806ae70 0x7f004806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:25.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.806+0000 7f0065d9b640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f004806ae70 0x7f004806d330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f00680c6360 tx=0x7f0050008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:25.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.810+0000 7f005effd640 1 -- 172.21.15.60:0/1814748055 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00600773f0 con 0x7f00680d19b0 2023-11-18T20:44:25.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.988+0000 7f006c8f2640 1 -- 172.21.15.60:0/1814748055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f006815c6b0 con 0x7f00680d19b0 2023-11-18T20:44:25.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.991+0000 7f005effd640 1 -- 172.21.15.60:0/1814748055 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f006007aa10 con 0x7f00680d19b0 2023-11-18T20:44:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 -- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f004806ae70 msgr2=0x7f004806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f004806ae70 0x7f004806d330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f00680c6360 tx=0x7f0050008040 comp rx=0 tx=0).stop 2023-11-18T20:44:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 -- 172.21.15.60:0/1814748055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d19b0 msgr2=0x7f00680c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d19b0 0x7f00680c9490 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0060000f80 tx=0x7f0060004a40 comp rx=0 tx=0).stop 2023-11-18T20:44:25.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 -- 172.21.15.60:0/1814748055 shutdown_connections 2023-11-18T20:44:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f004806ae70 0x7f004806d330 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00680d31c0 0x7f00680ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00680d1df0 0x7f0068153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 --2- 172.21.15.60:0/1814748055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00680d19b0 0x7f00680c9490 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 -- 172.21.15.60:0/1814748055 >> 172.21.15.60:0/1814748055 conn(0x7f0068072020 msgr2=0x7f00680cb100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 -- 172.21.15.60:0/1814748055 shutdown_connections 2023-11-18T20:44:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:25.994+0000 7f006c8f2640 1 -- 172.21.15.60:0/1814748055 wait complete. 2023-11-18T20:44:26.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-11-18T20:44:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.111+0000 7f558ef7c640 1 Processor -- start 2023-11-18T20:44:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.111+0000 7f558ef7c640 1 -- start start 2023-11-18T20:44:26.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558ef7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5588140310 0x7f55881618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558ef7c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f558806abe0 0x7f558806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558ef7c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55881406f0 con 0x7f55881409d0 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558ef7c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5588161e20 con 0x7f558806abe0 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558ef7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5588143960 con 0x7f5588140310 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f5587fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f55881409d0 0x7f558806a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49456/0 (socket says 172.21.15.60:49456) 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f5587fff640 1 -- 172.21.15.60:0/2580034649 learned_addr learned my addr 172.21.15.60:0/2580034649 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558cd18640 1 --2- 172.21.15.60:0/2580034649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5588140310 0x7f55881618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558d519640 1 --2- 172.21.15.60:0/2580034649 >> v2:172.21.15.100:3300/0 conn(0x7f558806abe0 0x7f558806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f55877fe640 1 -- 172.21.15.60:0/2580034649 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671013489 0 0) 0x7f55881406f0 con 0x7f55881409d0 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f55877fe640 1 -- 172.21.15.60:0/2580034649 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5574003520 con 0x7f55881409d0 2023-11-18T20:44:26.114 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f55877fe640 1 -- 172.21.15.60:0/2580034649 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3106179279 0 0) 0x7f5574003520 con 0x7f55881409d0 2023-11-18T20:44:26.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558cd18640 1 -- 172.21.15.60:0/2580034649 >> v2:172.21.15.100:3300/0 conn(0x7f558806abe0 msgr2=0x7f558806efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:26.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558cd18640 1 --2- 172.21.15.60:0/2580034649 >> v2:172.21.15.100:3300/0 conn(0x7f558806abe0 0x7f558806efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558cd18640 1 -- 172.21.15.60:0/2580034649 >> v1:172.21.15.60:6789/0 conn(0x7f55881409d0 legacy=0x7f558806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.115 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.112+0000 7f558cd18640 1 -- 172.21.15.60:0/2580034649 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588142de0 con 0x7f5588140310 2023-11-18T20:44:26.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.113+0000 7f558cd18640 1 --2- 172.21.15.60:0/2580034649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5588140310 0x7f55881618e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5588161f60 tx=0x7f557802b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1cd5113d5d91d4b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:26.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.113+0000 7f55877fe640 1 -- 172.21.15.60:0/2580034649 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570001670 con 0x7f5588140310 2023-11-18T20:44:26.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.113+0000 7f55877fe640 1 -- 172.21.15.60:0/2580034649 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557802e050 con 0x7f5588140310 2023-11-18T20:44:26.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.113+0000 7f55877fe640 1 -- 172.21.15.60:0/2580034649 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578030710 con 0x7f5588140310 2023-11-18T20:44:26.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.114+0000 7f558ef7c640 1 -- 172.21.15.60:0/2580034649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5588140310 msgr2=0x7f55881618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.114+0000 7f558ef7c640 1 --2- 172.21.15.60:0/2580034649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5588140310 0x7f55881618e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5588161f60 tx=0x7f557802b6f0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.114+0000 7f558ef7c640 1 -- 172.21.15.60:0/2580034649 shutdown_connections 2023-11-18T20:44:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.114+0000 7f558ef7c640 1 --2- 172.21.15.60:0/2580034649 >> v2:172.21.15.100:3300/0 conn(0x7f558806abe0 0x7f558806efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.114+0000 7f558ef7c640 1 --2- 172.21.15.60:0/2580034649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5588140310 0x7f55881618e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.114+0000 7f558ef7c640 1 -- 172.21.15.60:0/2580034649 >> 172.21.15.60:0/2580034649 conn(0x7f55880b1e50 msgr2=0x7f55880b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.114+0000 7f558ef7c640 1 -- 172.21.15.60:0/2580034649 shutdown_connections 2023-11-18T20:44:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.114+0000 7f558ef7c640 1 -- 172.21.15.60:0/2580034649 wait complete. 2023-11-18T20:44:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.115+0000 7f558ef7c640 1 Processor -- start 2023-11-18T20:44:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.115+0000 7f558ef7c640 1 -- start start 2023-11-18T20:44:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f558ef7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558806abe0 0x7f558813f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f558ef7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5588140310 0x7f5588143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f558ef7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55881409d0 0x7f5588141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f558ef7c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558813fea0 con 0x7f5588140310 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f558ef7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55881f3300 con 0x7f55881409d0 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f558ef7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55881f35e0 con 0x7f558806abe0 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f5587fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5588140310 0x7f5588143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f5587fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5588140310 0x7f5588143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35140/0 (socket says 172.21.15.60:35140) 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f5587fff640 1 -- 172.21.15.60:0/3599056413 learned_addr learned my addr 172.21.15.60:0/3599056413 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f558cd18640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558806abe0 0x7f558813f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f558d519640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55881409d0 0x7f5588141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.116+0000 7f5587fff640 1 -- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558806abe0 msgr2=0x7f558813f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f5587fff640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558806abe0 0x7f558813f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f5587fff640 1 -- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55881409d0 msgr2=0x7f5588141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f5587fff640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55881409d0 0x7f5588141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f5587fff640 1 -- 172.21.15.60:0/3599056413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578009400 con 0x7f5588140310 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f558cd18640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558806abe0 0x7f558813f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f558d519640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55881409d0 0x7f5588141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f5587fff640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5588140310 0x7f5588143fc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f55700017a0 tx=0x7f557000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f55857fa640 1 -- 172.21.15.60:0/3599056413 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570005ad0 con 0x7f5588140310 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f55857fa640 1 -- 172.21.15.60:0/3599056413 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5570005c30 con 0x7f5588140310 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f55857fa640 1 -- 172.21.15.60:0/3599056413 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700072b0 con 0x7f5588140310 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f558ef7c640 1 -- 172.21.15.60:0/3599056413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55881f38c0 con 0x7f5588140310 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.117+0000 7f558ef7c640 1 -- 172.21.15.60:0/3599056413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5588144500 con 0x7f5588140310 2023-11-18T20:44:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.118+0000 7f558ef7c640 1 -- 172.21.15.60:0/3599056413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558815dd10 con 0x7f5588140310 2023-11-18T20:44:26.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.118+0000 7f55857fa640 1 -- 172.21.15.60:0/3599056413 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5570007410 con 0x7f5588140310 2023-11-18T20:44:26.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.119+0000 7f55857fa640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f556806ae70 0x7f556806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.122 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.119+0000 7f55857fa640 1 -- 172.21.15.60:0/3599056413 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5570088fe0 con 0x7f5588140310 2023-11-18T20:44:26.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.119+0000 7f558cd18640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f556806ae70 0x7f556806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.120+0000 7f558cd18640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f556806ae70 0x7f556806d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5578034000 tx=0x7f5578008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:26.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.123+0000 7f55857fa640 1 -- 172.21.15.60:0/3599056413 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55700543c0 con 0x7f5588140310 2023-11-18T20:44:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.366+0000 7f558ef7c640 1 -- 172.21.15.60:0/3599056413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f5588069400 con 0x7f5588140310 2023-11-18T20:44:26.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.665+0000 7f55857fa640 1 -- 172.21.15.60:0/3599056413 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v121) v1 ==== 147+0+0 (secure 0 0 0) 0x7f55700579e0 con 0x7f5588140310 2023-11-18T20:44:26.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 -- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f556806ae70 msgr2=0x7f556806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f556806ae70 0x7f556806d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5578034000 tx=0x7f5578008030 comp rx=0 tx=0).stop 2023-11-18T20:44:26.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 -- 172.21.15.60:0/3599056413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5588140310 msgr2=0x7f5588143fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5588140310 0x7f5588143fc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f55700017a0 tx=0x7f557000cbe0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 -- 172.21.15.60:0/3599056413 shutdown_connections 2023-11-18T20:44:26.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f556806ae70 0x7f556806d330 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55881409d0 0x7f5588141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5588140310 0x7f5588143fc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 --2- 172.21.15.60:0/3599056413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f558806abe0 0x7f558813f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 -- 172.21.15.60:0/3599056413 >> 172.21.15.60:0/3599056413 conn(0x7f55880b1e50 msgr2=0x7f558815f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:26.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 -- 172.21.15.60:0/3599056413 shutdown_connections 2023-11-18T20:44:26.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.667+0000 7f558ef7c640 1 -- 172.21.15.60:0/3599056413 wait complete. 2023-11-18T20:44:26.680 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2023-11-18T20:44:26.680 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-11-18T20:44:26.680 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:26.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.786+0000 7f2ad89ce640 1 Processor -- start 2023-11-18T20:44:26.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.786+0000 7f2ad89ce640 1 -- start start 2023-11-18T20:44:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad89ce640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2ad4140310 0x7f2ad41618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad89ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2ad406abe0 0x7f2ad406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad89ce640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad41406f0 con 0x7f2ad41409d0 2023-11-18T20:44:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad89ce640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad4161e20 con 0x7f2ad4140310 2023-11-18T20:44:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad89ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad4143960 con 0x7f2ad406abe0 2023-11-18T20:44:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2ad4140310 0x7f2ad41618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad1d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2ad41409d0 0x7f2ad406a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49468/0 (socket says 172.21.15.60:49468) 2023-11-18T20:44:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad1d9b640 1 -- 172.21.15.60:0/3876566927 learned_addr learned my addr 172.21.15.60:0/3876566927 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad259c640 1 -- 172.21.15.60:0/3876566927 >> v2:172.21.15.100:3300/0 conn(0x7f2ad406abe0 msgr2=0x7f2ad406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad259c640 1 --2- 172.21.15.60:0/3876566927 >> v2:172.21.15.100:3300/0 conn(0x7f2ad406abe0 0x7f2ad406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.787+0000 7f2ad259c640 1 -- 172.21.15.60:0/3876566927 >> v1:172.21.15.60:6789/0 conn(0x7f2ad41409d0 legacy=0x7f2ad406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.788+0000 7f2ad259c640 1 -- 172.21.15.60:0/3876566927 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad4142de0 con 0x7f2ad4140310 2023-11-18T20:44:26.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.788+0000 7f2ad159a640 1 -- 172.21.15.60:0/3876566927 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465299668 0 0) 0x7f2ad41406f0 con 0x7f2ad41409d0 2023-11-18T20:44:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.788+0000 7f2ad259c640 1 --2- 172.21.15.60:0/3876566927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad4140310 0x7f2ad41618e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4154930 tx=0x7f2acc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b9afd6d6a99b2d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.788+0000 7f2ad159a640 1 -- 172.21.15.60:0/3876566927 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc02e070 con 0x7f2ad4140310 2023-11-18T20:44:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.788+0000 7f2ad159a640 1 -- 172.21.15.60:0/3876566927 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2acc0315a0 con 0x7f2ad4140310 2023-11-18T20:44:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.788+0000 7f2ad159a640 1 -- 172.21.15.60:0/3876566927 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc031930 con 0x7f2ad4140310 2023-11-18T20:44:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.789+0000 7f2ad89ce640 1 -- 172.21.15.60:0/3876566927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad4140310 msgr2=0x7f2ad41618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.789+0000 7f2ad89ce640 1 --2- 172.21.15.60:0/3876566927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad4140310 0x7f2ad41618e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4154930 tx=0x7f2acc02b840 comp rx=0 tx=0).stop 2023-11-18T20:44:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.789+0000 7f2ad89ce640 1 -- 172.21.15.60:0/3876566927 shutdown_connections 2023-11-18T20:44:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.789+0000 7f2ad89ce640 1 --2- 172.21.15.60:0/3876566927 >> v2:172.21.15.100:3300/0 conn(0x7f2ad406abe0 0x7f2ad406efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.789+0000 7f2ad89ce640 1 --2- 172.21.15.60:0/3876566927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad4140310 0x7f2ad41618e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.789+0000 7f2ad89ce640 1 -- 172.21.15.60:0/3876566927 >> 172.21.15.60:0/3876566927 conn(0x7f2ad40b1e50 msgr2=0x7f2ad40b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.789+0000 7f2ad89ce640 1 -- 172.21.15.60:0/3876566927 shutdown_connections 2023-11-18T20:44:26.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.789+0000 7f2ad89ce640 1 -- 172.21.15.60:0/3876566927 wait complete. 2023-11-18T20:44:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.790+0000 7f2ad89ce640 1 Processor -- start 2023-11-18T20:44:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.790+0000 7f2ad89ce640 1 -- start start 2023-11-18T20:44:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.791+0000 7f2ad89ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad406abe0 0x7f2ad4143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.791+0000 7f2ad89ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad4140310 0x7f2ad4145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.791+0000 7f2ad89ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad41409d0 0x7f2ad40af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.791+0000 7f2ad89ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad4144d20 con 0x7f2ad41409d0 2023-11-18T20:44:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.791+0000 7f2ad89ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad41460c0 con 0x7f2ad406abe0 2023-11-18T20:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.791+0000 7f2ad89ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad40afb60 con 0x7f2ad4140310 2023-11-18T20:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.791+0000 7f2ad2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad41409d0 0x7f2ad40af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.791+0000 7f2ad2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad41409d0 0x7f2ad40af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35144/0 (socket says 172.21.15.60:35144) 2023-11-18T20:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.791+0000 7f2ad2d9d640 1 -- 172.21.15.60:0/1855476476 learned_addr learned my addr 172.21.15.60:0/1855476476 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.791+0000 7f2ad1d9b640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad4140310 0x7f2ad4145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2ad259c640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad406abe0 0x7f2ad4143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2ad2d9d640 1 -- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad4140310 msgr2=0x7f2ad4145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2ad2d9d640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad4140310 0x7f2ad4145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2ad2d9d640 1 -- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad406abe0 msgr2=0x7f2ad4143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2ad2d9d640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad406abe0 0x7f2ad4143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2ad2d9d640 1 -- 172.21.15.60:0/1855476476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc009400 con 0x7f2ad41409d0 2023-11-18T20:44:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2ad2d9d640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad41409d0 0x7f2ad40af620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400b970 tx=0x7f2ac400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2acaffd640 1 -- 172.21.15.60:0/1855476476 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400cb40 con 0x7f2ad41409d0 2023-11-18T20:44:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2acaffd640 1 -- 172.21.15.60:0/1855476476 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac400ccc0 con 0x7f2ad41409d0 2023-11-18T20:44:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2ad89ce640 1 -- 172.21.15.60:0/1855476476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad4144110 con 0x7f2ad41409d0 2023-11-18T20:44:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2acaffd640 1 -- 172.21.15.60:0/1855476476 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4014680 con 0x7f2ad41409d0 2023-11-18T20:44:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.792+0000 7f2ad89ce640 1 -- 172.21.15.60:0/1855476476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad41446d0 con 0x7f2ad41409d0 2023-11-18T20:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.793+0000 7f2ad89ce640 1 -- 172.21.15.60:0/1855476476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad4161030 con 0x7f2ad41409d0 2023-11-18T20:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.794+0000 7f2acaffd640 1 -- 172.21.15.60:0/1855476476 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ac401d3b0 con 0x7f2ad41409d0 2023-11-18T20:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.794+0000 7f2acaffd640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2ab406ae40 0x7f2ab406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.794+0000 7f2acaffd640 1 -- 172.21.15.60:0/1855476476 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2ac4089210 con 0x7f2ad41409d0 2023-11-18T20:44:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.794+0000 7f2ad259c640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2ab406ae40 0x7f2ab406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.795+0000 7f2ad259c640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2ab406ae40 0x7f2ab406d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2acc036000 tx=0x7f2acc03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.798+0000 7f2acaffd640 1 -- 172.21.15.60:0/1855476476 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ac4054670 con 0x7f2ad41409d0 2023-11-18T20:44:26.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.974+0000 7f2ad89ce640 1 -- 172.21.15.60:0/1855476476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ad4069400 con 0x7f2ad41409d0 2023-11-18T20:44:26.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.975+0000 7f2acaffd640 1 -- 172.21.15.60:0/1855476476 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12974 (secure 0 0 0) 0x7f2ac4087e60 con 0x7f2ad41409d0 2023-11-18T20:44:26.976 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:26.977 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":41,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:25.659843+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6834/3762312847":"2023-11-19T20:44:21.649310+0000","172.21.15.60:6832/3762312847":"2023-11-19T20:44:21.649310+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:26.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 -- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2ab406ae40 msgr2=0x7f2ab406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2ab406ae40 0x7f2ab406d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2acc036000 tx=0x7f2acc03d040 comp rx=0 tx=0).stop 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 -- 172.21.15.60:0/1855476476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad41409d0 msgr2=0x7f2ad40af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad41409d0 0x7f2ad40af620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400b970 tx=0x7f2ac400be30 comp rx=0 tx=0).stop 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 -- 172.21.15.60:0/1855476476 shutdown_connections 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2ab406ae40 0x7f2ab406d300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad41409d0 0x7f2ad40af620 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad4140310 0x7f2ad4145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 --2- 172.21.15.60:0/1855476476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad406abe0 0x7f2ad4143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 -- 172.21.15.60:0/1855476476 >> 172.21.15.60:0/1855476476 conn(0x7f2ad40b1e50 msgr2=0x7f2ad4141bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 -- 172.21.15.60:0/1855476476 shutdown_connections 2023-11-18T20:44:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:26.977+0000 7f2ad89ce640 1 -- 172.21.15.60:0/1855476476 wait complete. 2023-11-18T20:44:26.989 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:44:26.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:44:27.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.097+0000 7f0b279c0640 1 Processor -- start 2023-11-18T20:44:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.097+0000 7f0b279c0640 1 -- start start 2023-11-18T20:44:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.098+0000 7f0b279c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0b20160f70 0x7f0b200cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.098+0000 7f0b279c0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0b200ce480 0x7f0b200d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.098+0000 7f0b279c0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b201614e0 con 0x7f0b2015d2d0 2023-11-18T20:44:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.098+0000 7f0b279c0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b201617c0 con 0x7f0b200ce480 2023-11-18T20:44:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.098+0000 7f0b279c0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b20161aa0 con 0x7f0b20160f70 2023-11-18T20:44:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.098+0000 7f0b25f5d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0b200ce480 0x7f0b200d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.098+0000 7f0b25f5d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0b200ce480 0x7f0b200d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35146/0 (socket says 172.21.15.60:35146) 2023-11-18T20:44:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.098+0000 7f0b25f5d640 1 -- 172.21.15.60:0/2418624063 learned_addr learned my addr 172.21.15.60:0/2418624063 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.098+0000 7f0b24f5b640 1 --2- 172.21.15.60:0/2418624063 >> v2:172.21.15.100:3300/0 conn(0x7f0b20160f70 0x7f0b200cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.099+0000 7f0b25f5d640 1 -- 172.21.15.60:0/2418624063 >> v2:172.21.15.100:3300/0 conn(0x7f0b20160f70 msgr2=0x7f0b200cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.099+0000 7f0b25f5d640 1 --2- 172.21.15.60:0/2418624063 >> v2:172.21.15.100:3300/0 conn(0x7f0b20160f70 0x7f0b200cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.099+0000 7f0b25f5d640 1 -- 172.21.15.60:0/2418624063 >> v1:172.21.15.100:6789/0 conn(0x7f0b2015d2d0 legacy=0x7f0b200b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.099+0000 7f0b25f5d640 1 -- 172.21.15.60:0/2418624063 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b20161d80 con 0x7f0b200ce480 2023-11-18T20:44:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.099+0000 7f0b25f5d640 1 --2- 172.21.15.60:0/2418624063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b200ce480 0x7f0b200d1af0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0b200cdb80 tx=0x7f0b1802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9300fadf7eab4570 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.099+0000 7f0b1ffff640 1 -- 172.21.15.60:0/2418624063 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18030860 con 0x7f0b200ce480 2023-11-18T20:44:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.100+0000 7f0b1ffff640 1 -- 172.21.15.60:0/2418624063 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b180309c0 con 0x7f0b200ce480 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.100+0000 7f0b1ffff640 1 -- 172.21.15.60:0/2418624063 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18030d50 con 0x7f0b200ce480 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.100+0000 7f0b279c0640 1 -- 172.21.15.60:0/2418624063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b200ce480 msgr2=0x7f0b200d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.100+0000 7f0b279c0640 1 --2- 172.21.15.60:0/2418624063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b200ce480 0x7f0b200d1af0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0b200cdb80 tx=0x7f0b1802b9a0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.100+0000 7f0b279c0640 1 -- 172.21.15.60:0/2418624063 shutdown_connections 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.100+0000 7f0b279c0640 1 --2- 172.21.15.60:0/2418624063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b200ce480 0x7f0b200d1af0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.100+0000 7f0b279c0640 1 --2- 172.21.15.60:0/2418624063 >> v2:172.21.15.100:3300/0 conn(0x7f0b20160f70 0x7f0b200cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.100+0000 7f0b279c0640 1 -- 172.21.15.60:0/2418624063 >> 172.21.15.60:0/2418624063 conn(0x7f0b20072230 msgr2=0x7f0b200cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.100+0000 7f0b279c0640 1 -- 172.21.15.60:0/2418624063 shutdown_connections 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.100+0000 7f0b279c0640 1 -- 172.21.15.60:0/2418624063 wait complete. 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.101+0000 7f0b279c0640 1 Processor -- start 2023-11-18T20:44:27.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.101+0000 7f0b279c0640 1 -- start start 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b279c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b200ce480 0x7f0b2014e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b279c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2015d2d0 0x7f0b200b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b279c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b20160f70 0x7f0b200b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b279c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200cd510 con 0x7f0b20160f70 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b279c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2014ec20 con 0x7f0b2015d2d0 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b279c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200c9550 con 0x7f0b200ce480 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b25f5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b20160f70 0x7f0b200b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b25f5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b20160f70 0x7f0b200b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35158/0 (socket says 172.21.15.60:35158) 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b25f5d640 1 -- 172.21.15.60:0/172049527 learned_addr learned my addr 172.21.15.60:0/172049527 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b25f5d640 1 -- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b200ce480 msgr2=0x7f0b2014e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.106 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b2575c640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b200ce480 0x7f0b2014e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.102+0000 7f0b24f5b640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2015d2d0 0x7f0b200b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b25f5d640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b200ce480 0x7f0b2014e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b25f5d640 1 -- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2015d2d0 msgr2=0x7f0b200b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b25f5d640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2015d2d0 0x7f0b200b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b25f5d640 1 -- 172.21.15.60:0/172049527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b180094d0 con 0x7f0b20160f70 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b24f5b640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2015d2d0 0x7f0b200b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b2575c640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b200ce480 0x7f0b2014e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b25f5d640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b20160f70 0x7f0b200b4b80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0b180090b0 tx=0x7f0b18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b1dffb640 1 -- 172.21.15.60:0/172049527 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18039c90 con 0x7f0b20160f70 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b1dffb640 1 -- 172.21.15.60:0/172049527 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b18039df0 con 0x7f0b20160f70 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b279c0640 1 -- 172.21.15.60:0/172049527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b200c9830 con 0x7f0b20160f70 2023-11-18T20:44:27.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b1dffb640 1 -- 172.21.15.60:0/172049527 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18043640 con 0x7f0b20160f70 2023-11-18T20:44:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.103+0000 7f0b279c0640 1 -- 172.21.15.60:0/172049527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b200c77e0 con 0x7f0b20160f70 2023-11-18T20:44:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.104+0000 7f0b279c0640 1 -- 172.21.15.60:0/172049527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b200c9f90 con 0x7f0b20160f70 2023-11-18T20:44:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.105+0000 7f0b1dffb640 1 -- 172.21.15.60:0/172049527 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b180437a0 con 0x7f0b20160f70 2023-11-18T20:44:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.105+0000 7f0b1dffb640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b0c06ae40 0x7f0b0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.105+0000 7f0b1dffb640 1 -- 172.21.15.60:0/172049527 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0b1803e070 con 0x7f0b20160f70 2023-11-18T20:44:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.105+0000 7f0b2575c640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b0c06ae40 0x7f0b0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.106+0000 7f0b2575c640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b0c06ae40 0x7f0b0c06d300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0b100012e0 tx=0x7f0b10009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:27.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.109+0000 7f0b1dffb640 1 -- 172.21.15.60:0/172049527 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b18077820 con 0x7f0b20160f70 2023-11-18T20:44:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.283+0000 7f0b279c0640 1 -- 172.21.15.60:0/172049527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b200c79d0 con 0x7f0b20160f70 2023-11-18T20:44:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.283+0000 7f0b1dffb640 1 -- 172.21.15.60:0/172049527 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b18034020 con 0x7f0b20160f70 2023-11-18T20:44:27.285 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:44:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 -- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b0c06ae40 msgr2=0x7f0b0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b0c06ae40 0x7f0b0c06d300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0b100012e0 tx=0x7f0b10009040 comp rx=0 tx=0).stop 2023-11-18T20:44:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 -- 172.21.15.60:0/172049527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b20160f70 msgr2=0x7f0b200b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b20160f70 0x7f0b200b4b80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0b180090b0 tx=0x7f0b18004730 comp rx=0 tx=0).stop 2023-11-18T20:44:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 -- 172.21.15.60:0/172049527 shutdown_connections 2023-11-18T20:44:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f0b0c06ae40 0x7f0b0c06d300 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b20160f70 0x7f0b200b4b80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2015d2d0 0x7f0b200b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 --2- 172.21.15.60:0/172049527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b200ce480 0x7f0b2014e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 -- 172.21.15.60:0/172049527 >> 172.21.15.60:0/172049527 conn(0x7f0b20072230 msgr2=0x7f0b200b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:27.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 -- 172.21.15.60:0/172049527 shutdown_connections 2023-11-18T20:44:27.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.285+0000 7f0b279c0640 1 -- 172.21.15.60:0/172049527 wait complete. 2023-11-18T20:44:27.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-11-18T20:44:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.402+0000 7f160c901640 1 Processor -- start 2023-11-18T20:44:27.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.402+0000 7f160c901640 1 -- start start 2023-11-18T20:44:27.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.403+0000 7f160c901640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f160815d330 0x7f16080b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.403+0000 7f160c901640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16080d3900 0x7f16080b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.403+0000 7f160c901640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16080b2b10 con 0x7f16080d2910 2023-11-18T20:44:27.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.403+0000 7f160c901640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16080d1590 con 0x7f160815d330 2023-11-18T20:44:27.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.403+0000 7f160c901640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16080d1870 con 0x7f16080d3900 2023-11-18T20:44:27.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.403+0000 7f160659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f160815d330 0x7f16080b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.403+0000 7f160659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f160815d330 0x7f16080b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35164/0 (socket says 172.21.15.60:35164) 2023-11-18T20:44:27.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.403+0000 7f160659c640 1 -- 172.21.15.60:0/1530925452 learned_addr learned my addr 172.21.15.60:0/1530925452 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:27.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.404+0000 7f1606d9d640 1 --2- 172.21.15.60:0/1530925452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16080d3900 0x7f16080b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.404+0000 7f160659c640 1 -- 172.21.15.60:0/1530925452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16080d3900 msgr2=0x7f16080b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:27.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.404+0000 7f160659c640 1 --2- 172.21.15.60:0/1530925452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16080d3900 0x7f16080b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.404+0000 7f160659c640 1 -- 172.21.15.60:0/1530925452 >> v1:172.21.15.100:6789/0 conn(0x7f16080d2910 legacy=0x7f16080d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:27.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.404+0000 7f160659c640 1 -- 172.21.15.60:0/1530925452 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16080d1b50 con 0x7f160815d330 2023-11-18T20:44:27.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.404+0000 7f160659c640 1 --2- 172.21.15.60:0/1530925452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f160815d330 0x7f16080b8b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009e50 tx=0x7f15f802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1741ab8d53f79c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160559a640 1 -- 172.21.15.60:0/1530925452 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f802ca70 con 0x7f160815d330 2023-11-18T20:44:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160559a640 1 -- 172.21.15.60:0/1530925452 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f802cbd0 con 0x7f160815d330 2023-11-18T20:44:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160559a640 1 -- 172.21.15.60:0/1530925452 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f80343f0 con 0x7f160815d330 2023-11-18T20:44:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160c901640 1 -- 172.21.15.60:0/1530925452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f160815d330 msgr2=0x7f16080b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160c901640 1 --2- 172.21.15.60:0/1530925452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f160815d330 0x7f16080b8b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009e50 tx=0x7f15f802bed0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160c901640 1 -- 172.21.15.60:0/1530925452 shutdown_connections 2023-11-18T20:44:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160c901640 1 --2- 172.21.15.60:0/1530925452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16080d3900 0x7f16080b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160c901640 1 --2- 172.21.15.60:0/1530925452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f160815d330 0x7f16080b8b80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160c901640 1 -- 172.21.15.60:0/1530925452 >> 172.21.15.60:0/1530925452 conn(0x7f1608072020 msgr2=0x7f1608072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160c901640 1 -- 172.21.15.60:0/1530925452 shutdown_connections 2023-11-18T20:44:27.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.405+0000 7f160c901640 1 -- 172.21.15.60:0/1530925452 wait complete. 2023-11-18T20:44:27.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160c901640 1 Processor -- start 2023-11-18T20:44:27.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160c901640 1 -- start start 2023-11-18T20:44:27.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160c901640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16080d2910 0x7f1608149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160c901640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16080d3900 0x7f16080ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160c901640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f160815d330 0x7f16080ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160c901640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16080c90f0 con 0x7f16080d2910 2023-11-18T20:44:27.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160c901640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16080c93d0 con 0x7f16080d3900 2023-11-18T20:44:27.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160c901640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16080c96b0 con 0x7f160815d330 2023-11-18T20:44:27.411 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16080d2910 0x7f1608149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16080d2910 0x7f1608149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35174/0 (socket says 172.21.15.60:35174) 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.407+0000 7f160659c640 1 -- 172.21.15.60:0/4088435704 learned_addr learned my addr 172.21.15.60:0/4088435704 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f1606d9d640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f160815d330 0x7f16080ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f1605d9b640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16080d3900 0x7f16080ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f160659c640 1 -- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f160815d330 msgr2=0x7f16080ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f160659c640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f160815d330 0x7f16080ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f160659c640 1 -- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16080d3900 msgr2=0x7f16080ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f160659c640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16080d3900 0x7f16080ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f160659c640 1 -- 172.21.15.60:0/4088435704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f8009b30 con 0x7f16080d2910 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f1605d9b640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16080d3900 0x7f16080ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f1606d9d640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f160815d330 0x7f16080ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:27.412 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f160659c640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16080d2910 0x7f1608149210 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f15f8000f80 tx=0x7f15f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f15feffd640 1 -- 172.21.15.60:0/4088435704 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8009510 con 0x7f16080d2910 2023-11-18T20:44:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f15feffd640 1 -- 172.21.15.60:0/4088435704 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f803e450 con 0x7f16080d2910 2023-11-18T20:44:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f160c901640 1 -- 172.21.15.60:0/4088435704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16080c9990 con 0x7f16080d2910 2023-11-18T20:44:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f15feffd640 1 -- 172.21.15.60:0/4088435704 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f803e800 con 0x7f16080d2910 2023-11-18T20:44:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.408+0000 7f160c901640 1 -- 172.21.15.60:0/4088435704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1608149750 con 0x7f16080d2910 2023-11-18T20:44:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.409+0000 7f160c901640 1 -- 172.21.15.60:0/4088435704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1608069400 con 0x7f16080d2910 2023-11-18T20:44:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.410+0000 7f15feffd640 1 -- 172.21.15.60:0/4088435704 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15f803e960 con 0x7f16080d2910 2023-11-18T20:44:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.410+0000 7f15feffd640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15e806ae40 0x7f15e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.413 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.410+0000 7f15feffd640 1 -- 172.21.15.60:0/4088435704 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f15f80ac140 con 0x7f16080d2910 2023-11-18T20:44:27.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.410+0000 7f1605d9b640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15e806ae40 0x7f15e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.414 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.411+0000 7f1605d9b640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15e806ae40 0x7f15e806d300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1608160b30 tx=0x7f15f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:27.415 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.414+0000 7f15feffd640 1 -- 172.21.15.60:0/4088435704 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15f8077610 con 0x7f16080d2910 2023-11-18T20:44:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.656+0000 7f160c901640 1 -- 172.21.15.60:0/4088435704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7f16080b23e0 con 0x7f16080d2910 2023-11-18T20:44:27.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.669+0000 7f15feffd640 1 -- 172.21.15.60:0/4088435704 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v122) v1 ==== 137+0+0 (secure 0 0 0) 0x7f15f807ac30 con 0x7f16080d2910 2023-11-18T20:44:27.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 -- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15e806ae40 msgr2=0x7f15e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15e806ae40 0x7f15e806d300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1608160b30 tx=0x7f15f0009040 comp rx=0 tx=0).stop 2023-11-18T20:44:27.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 -- 172.21.15.60:0/4088435704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16080d2910 msgr2=0x7f1608149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16080d2910 0x7f1608149210 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f15f8000f80 tx=0x7f15f8004a40 comp rx=0 tx=0).stop 2023-11-18T20:44:27.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 -- 172.21.15.60:0/4088435704 shutdown_connections 2023-11-18T20:44:27.673 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f15e806ae40 0x7f15e806d300 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f160815d330 0x7f16080ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16080d3900 0x7f16080ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 --2- 172.21.15.60:0/4088435704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16080d2910 0x7f1608149210 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 -- 172.21.15.60:0/4088435704 >> 172.21.15.60:0/4088435704 conn(0x7f1608072020 msgr2=0x7f1608072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:27.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 -- 172.21.15.60:0/4088435704 shutdown_connections 2023-11-18T20:44:27.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.671+0000 7f160c901640 1 -- 172.21.15.60:0/4088435704 wait complete. 2023-11-18T20:44:27.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-18T20:44:27.792 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.791+0000 7f6fb1044640 1 Processor -- start 2023-11-18T20:44:27.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.791+0000 7f6fb1044640 1 -- start start 2023-11-18T20:44:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fb1044640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac15c2f0 0x7f6fac06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fb1044640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac15ba40 con 0x7f6fac140f10 2023-11-18T20:44:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fb1044640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac06f4a0 con 0x7f6fac06f040 2023-11-18T20:44:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fb1044640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac15c6d0 con 0x7f6fac15c2f0 2023-11-18T20:44:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fab59e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6fac140f10 0x7f6fac13f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49484/0 (socket says 172.21.15.60:49484) 2023-11-18T20:44:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fab59e640 1 -- 172.21.15.60:0/1508517052 learned_addr learned my addr 172.21.15.60:0/1508517052 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6faad9d640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac15c2f0 0x7f6fac06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908640657 0 0) 0x7f6fac15ba40 con 0x7f6fac140f10 2023-11-18T20:44:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f98003520 con 0x7f6fac140f10 2023-11-18T20:44:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2698853845 0 0) 0x7f6f98003520 con 0x7f6fac140f10 2023-11-18T20:44:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac15c2f0 msgr2=0x7f6fac06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fa9d9b640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac15c2f0 0x7f6fac06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 >> v1:172.21.15.100:6789/0 conn(0x7f6fac06f040 legacy=0x7f6fac140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.792+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac1416e0 con 0x7f6fac140f10 2023-11-18T20:44:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f6f9c0019f0 con 0x7f6fac140f10 2023-11-18T20:44:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 >> v1:172.21.15.60:6789/0 conn(0x7f6fac140f10 legacy=0x7f6fac13f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6fa9d9b640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 0x7f6f98003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6fa9d9b640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac15c2f0 0x7f6f980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6fa9d9b640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98004d00 0x7f6f98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9800ba30 con 0x7f6f980039a0 2023-11-18T20:44:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9800bba0 con 0x7f6f98004d00 2023-11-18T20:44:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9800bd60 con 0x7f6fac15c2f0 2023-11-18T20:44:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6faad9d640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 0x7f6f98003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6faa59c640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac15c2f0 0x7f6f980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6fab59e640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98004d00 0x7f6f98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6faad9d640 1 -- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac15c2f0 msgr2=0x7f6f980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6faad9d640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac15c2f0 0x7f6f980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6faad9d640 1 -- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98004d00 msgr2=0x7f6f98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6faad9d640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98004d00 0x7f6f98005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6faad9d640 1 -- 172.21.15.60:0/1508517052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f98003520 con 0x7f6f980039a0 2023-11-18T20:44:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.793+0000 7f6fab59e640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f98004d00 0x7f6f98005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6faad9d640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 0x7f6f98003dc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009a90 tx=0x7f6f980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f3955951d9c8d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa402c680 con 0x7f6f980039a0 2023-11-18T20:44:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa402c7e0 con 0x7f6f980039a0 2023-11-18T20:44:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6fa9d9b640 1 -- 172.21.15.60:0/1508517052 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa402cb70 con 0x7f6f980039a0 2023-11-18T20:44:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6fb1044640 1 -- 172.21.15.60:0/1508517052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 msgr2=0x7f6f98003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6fb1044640 1 --2- 172.21.15.60:0/1508517052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 0x7f6f98003dc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009a90 tx=0x7f6f980057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6fab59e640 1 -- 172.21.15.60:0/1508517052 reap_dead start 2023-11-18T20:44:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6fb1044640 1 -- 172.21.15.60:0/1508517052 shutdown_connections 2023-11-18T20:44:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6fb1044640 1 -- 172.21.15.60:0/1508517052 >> 172.21.15.60:0/1508517052 conn(0x7f6fac06a6b0 msgr2=0x7f6fac06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6fb1044640 1 -- 172.21.15.60:0/1508517052 shutdown_connections 2023-11-18T20:44:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.794+0000 7f6fb1044640 1 -- 172.21.15.60:0/1508517052 wait complete. 2023-11-18T20:44:27.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.796+0000 7f6fb1044640 1 Processor -- start 2023-11-18T20:44:27.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.796+0000 7f6fb1044640 1 -- start start 2023-11-18T20:44:27.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fb1044640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 0x7f6fac13bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fb1044640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac153e10 0x7f6fac06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fb1044640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac06d320 0x7f6fac13c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fb1044640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac06c4d0 con 0x7f6f980039a0 2023-11-18T20:44:27.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fb1044640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac06b5e0 con 0x7f6fac06d320 2023-11-18T20:44:27.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fb1044640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac06b8c0 con 0x7f6fac153e10 2023-11-18T20:44:27.807 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 0x7f6fac13bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 0x7f6fac13bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35188/0 (socket says 172.21.15.60:35188) 2023-11-18T20:44:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faad9d640 1 -- 172.21.15.60:0/3392014335 learned_addr learned my addr 172.21.15.60:0/3392014335 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faa59c640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac153e10 0x7f6fac06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fab59e640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac06d320 0x7f6fac13c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faad9d640 1 -- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac153e10 msgr2=0x7f6fac06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faad9d640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac153e10 0x7f6fac06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faad9d640 1 -- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac06d320 msgr2=0x7f6fac13c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faad9d640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac06d320 0x7f6fac13c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faad9d640 1 -- 172.21.15.60:0/3392014335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac06bba0 con 0x7f6f980039a0 2023-11-18T20:44:27.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faa59c640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac153e10 0x7f6fac06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:27.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6faad9d640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 0x7f6fac13bf10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6fa402c220 tx=0x7f6fa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:27.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fa37fe640 1 -- 172.21.15.60:0/3392014335 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4003810 con 0x7f6f980039a0 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fa37fe640 1 -- 172.21.15.60:0/3392014335 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4003970 con 0x7f6f980039a0 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fb1044640 1 -- 172.21.15.60:0/3392014335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac06be80 con 0x7f6f980039a0 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fa37fe640 1 -- 172.21.15.60:0/3392014335 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4003ce0 con 0x7f6f980039a0 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.797+0000 7f6fb1044640 1 -- 172.21.15.60:0/3392014335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac154290 con 0x7f6f980039a0 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.798+0000 7f6fb1044640 1 -- 172.21.15.60:0/3392014335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac1f2b80 con 0x7f6f980039a0 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.799+0000 7f6fa37fe640 1 -- 172.21.15.60:0/3392014335 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6fa4034440 con 0x7f6f980039a0 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.799+0000 7f6fa37fe640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6f8c06ae70 0x7f6f8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.799+0000 7f6fa37fe640 1 -- 172.21.15.60:0/3392014335 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6fa40ab0a0 con 0x7f6f980039a0 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.800+0000 7f6faa59c640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6f8c06ae70 0x7f6f8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.800+0000 7f6faa59c640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6f8c06ae70 0x7f6f8c06d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6fac15fc80 tx=0x7f6f94009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:27.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.804+0000 7f6fa37fe640 1 -- 172.21.15.60:0/3392014335 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fa4076590 con 0x7f6f980039a0 2023-11-18T20:44:27.980 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.979+0000 7f6fb1044640 1 -- 172.21.15.60:0/3392014335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6fac06c4d0 con 0x7f6f980039a0 2023-11-18T20:44:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.982+0000 7f6fa37fe640 1 -- 172.21.15.60:0/3392014335 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6fa40044e0 con 0x7f6f980039a0 2023-11-18T20:44:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 -- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6f8c06ae70 msgr2=0x7f6f8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6f8c06ae70 0x7f6f8c06d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6fac15fc80 tx=0x7f6f94009040 comp rx=0 tx=0).stop 2023-11-18T20:44:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 -- 172.21.15.60:0/3392014335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 msgr2=0x7f6fac13bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 0x7f6fac13bf10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6fa402c220 tx=0x7f6fa4004730 comp rx=0 tx=0).stop 2023-11-18T20:44:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 -- 172.21.15.60:0/3392014335 shutdown_connections 2023-11-18T20:44:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6f8c06ae70 0x7f6f8c06d330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac06d320 0x7f6fac13c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fac153e10 0x7f6fac06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 --2- 172.21.15.60:0/3392014335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f980039a0 0x7f6fac13bf10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:27.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 -- 172.21.15.60:0/3392014335 >> 172.21.15.60:0/3392014335 conn(0x7f6fac06a6b0 msgr2=0x7f6fac0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:27.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 -- 172.21.15.60:0/3392014335 shutdown_connections 2023-11-18T20:44:27.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:27.984+0000 7f6fb1044640 1 -- 172.21.15.60:0/3392014335 wait complete. 2023-11-18T20:44:27.996 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-18T20:44:27.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-18T20:44:28.045 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-18T20:44:28.087 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-18T20:44:28.088 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:28.088 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:28.104 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=344 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249488k,nr_inodes=812372,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-18T20:44:28.104 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:44:28.104 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:28.146 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056339,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=480 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-18T20:44:28.146 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:28.146 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:28.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=344 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249488k,nr_inodes=812372,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-18T20:44:28.162 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:44:28.162 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:28.203 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056339,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=480 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-18T20:44:28.203 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-18T20:44:28.204 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:44:28.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.308+0000 7f82f92e0640 1 Processor -- start 2023-11-18T20:44:28.310 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.308+0000 7f82f92e0640 1 -- start start 2023-11-18T20:44:28.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.309+0000 7f82f92e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82f415d330 0x7f82f40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.309+0000 7f82f92e0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f40b2b10 con 0x7f82f40d2910 2023-11-18T20:44:28.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.309+0000 7f82f92e0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f40d1590 con 0x7f82f40d3900 2023-11-18T20:44:28.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.309+0000 7f82f92e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f40d1870 con 0x7f82f415d330 2023-11-18T20:44:28.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.309+0000 7f82f2ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f82f415d330 0x7f82f40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.309+0000 7f82f27fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f82f40d2910 0x7f82f40d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49500/0 (socket says 172.21.15.60:49500) 2023-11-18T20:44:28.313 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.309+0000 7f82f27fc640 1 -- 172.21.15.60:0/1631182871 learned_addr learned my addr 172.21.15.60:0/1631182871 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:28.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.310+0000 7f82f2ffd640 1 -- 172.21.15.60:0/1631182871 >> v1:172.21.15.100:6789/0 conn(0x7f82f40d3900 legacy=0x7f82f40b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:28.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.310+0000 7f82f2ffd640 1 -- 172.21.15.60:0/1631182871 >> v1:172.21.15.60:6789/0 conn(0x7f82f40d2910 legacy=0x7f82f40d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:28.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.310+0000 7f82f1ffb640 1 -- 172.21.15.60:0/1631182871 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679226307 0 0) 0x7f82f40b2b10 con 0x7f82f40d2910 2023-11-18T20:44:28.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.310+0000 7f82f2ffd640 1 -- 172.21.15.60:0/1631182871 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40d1b50 con 0x7f82f415d330 2023-11-18T20:44:28.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.310+0000 7f82f2ffd640 1 --2- 172.21.15.60:0/1631182871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 0x7f82f40b8b80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009d90 tx=0x7f82e402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101094b0a0e19ea6 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:28.314 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.310+0000 7f82f1ffb640 1 -- 172.21.15.60:0/1631182871 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e402cbd0 con 0x7f82f415d330 2023-11-18T20:44:28.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.310+0000 7f82f1ffb640 1 -- 172.21.15.60:0/1631182871 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e402cd30 con 0x7f82f415d330 2023-11-18T20:44:28.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.310+0000 7f82f1ffb640 1 -- 172.21.15.60:0/1631182871 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e402cea0 con 0x7f82f415d330 2023-11-18T20:44:28.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.311+0000 7f82f92e0640 1 -- 172.21.15.60:0/1631182871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 msgr2=0x7f82f40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:28.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.311+0000 7f82f92e0640 1 --2- 172.21.15.60:0/1631182871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 0x7f82f40b8b80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009d90 tx=0x7f82e402be20 comp rx=0 tx=0).stop 2023-11-18T20:44:28.315 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.311+0000 7f82f92e0640 1 -- 172.21.15.60:0/1631182871 shutdown_connections 2023-11-18T20:44:28.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.311+0000 7f82f92e0640 1 --2- 172.21.15.60:0/1631182871 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 0x7f82f40b8b80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.311+0000 7f82f92e0640 1 -- 172.21.15.60:0/1631182871 >> 172.21.15.60:0/1631182871 conn(0x7f82f4072020 msgr2=0x7f82f4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:28.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.311+0000 7f82f92e0640 1 -- 172.21.15.60:0/1631182871 shutdown_connections 2023-11-18T20:44:28.316 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.311+0000 7f82f92e0640 1 -- 172.21.15.60:0/1631182871 wait complete. 2023-11-18T20:44:28.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.312+0000 7f82f92e0640 1 Processor -- start 2023-11-18T20:44:28.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.312+0000 7f82f92e0640 1 -- start start 2023-11-18T20:44:28.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f92e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82f40d2910 0x7f82f4160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f92e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f40d3900 0x7f82f40c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f92e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 0x7f82f40b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f92e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c7af0 con 0x7f82f415d330 2023-11-18T20:44:28.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f92e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c8dc0 con 0x7f82f40d3900 2023-11-18T20:44:28.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f92e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c7d20 con 0x7f82f40d2910 2023-11-18T20:44:28.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 0x7f82f40b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 0x7f82f40b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35212/0 (socket says 172.21.15.60:35212) 2023-11-18T20:44:28.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f37fe640 1 -- 172.21.15.60:0/2625298367 learned_addr learned my addr 172.21.15.60:0/2625298367 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:28.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f27fc640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f40d3900 0x7f82f40c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.313+0000 7f82f2ffd640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82f40d2910 0x7f82f4160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82f37fe640 1 -- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82f40d2910 msgr2=0x7f82f4160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:28.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82f37fe640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82f40d2910 0x7f82f4160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82f37fe640 1 -- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f40d3900 msgr2=0x7f82f40c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:28.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82f37fe640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f40d3900 0x7f82f40c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82f37fe640 1 -- 172.21.15.60:0/2625298367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e4009a70 con 0x7f82f415d330 2023-11-18T20:44:28.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82f2ffd640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82f40d2910 0x7f82f4160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:28.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82f27fc640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f40d3900 0x7f82f40c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:28.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82f37fe640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 0x7f82f40b6f00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c360 tx=0x7f82ec00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:28.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82eb7fe640 1 -- 172.21.15.60:0/2625298367 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0035f0 con 0x7f82f415d330 2023-11-18T20:44:28.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82eb7fe640 1 -- 172.21.15.60:0/2625298367 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec003750 con 0x7f82f415d330 2023-11-18T20:44:28.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82f92e0640 1 -- 172.21.15.60:0/2625298367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f40c8000 con 0x7f82f415d330 2023-11-18T20:44:28.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82eb7fe640 1 -- 172.21.15.60:0/2625298367 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec003b40 con 0x7f82f415d330 2023-11-18T20:44:28.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.314+0000 7f82f92e0640 1 -- 172.21.15.60:0/2625298367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f40c85c0 con 0x7f82f415d330 2023-11-18T20:44:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.315+0000 7f82f92e0640 1 -- 172.21.15.60:0/2625298367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f4069400 con 0x7f82f415d330 2023-11-18T20:44:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.315+0000 7f82eb7fe640 1 -- 172.21.15.60:0/2625298367 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f82ec003ca0 con 0x7f82f415d330 2023-11-18T20:44:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.316+0000 7f82eb7fe640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f82d406ae40 0x7f82d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.316+0000 7f82eb7fe640 1 -- 172.21.15.60:0/2625298367 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f82ec0888c0 con 0x7f82f415d330 2023-11-18T20:44:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.316+0000 7f82f2ffd640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f82d406ae40 0x7f82d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.317+0000 7f82f2ffd640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f82d406ae40 0x7f82d406d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f82e4036000 tx=0x7f82e4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.320+0000 7f82eb7fe640 1 -- 172.21.15.60:0/2625298367 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82ec053c70 con 0x7f82f415d330 2023-11-18T20:44:28.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.534+0000 7f82f92e0640 1 -- 172.21.15.60:0/2625298367 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82f415ffb0 con 0x7f82f415d330 2023-11-18T20:44:28.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.534+0000 7f82eb7fe640 1 -- 172.21.15.60:0/2625298367 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+3704 (secure 0 0 0) 0x7f82ec017020 con 0x7f82f415d330 2023-11-18T20:44:28.536 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:28.536 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4814,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3885363174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3885363174},{"type":"v1","addr":"172.21.15.100:6837","nonce":3885363174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1102236552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1102236552},{"type":"v1","addr":"172.21.15.60:6835","nonce":1102236552}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2680685354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2680685354},{"type":"v1","addr":"172.21.15.100:6836","nonce":2680685354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2082396069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2082396069},{"type":"v1","addr":"172.21.15.60:6834","nonce":2082396069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":16}],"filesystems":[]} 2023-11-18T20:44:28.536 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2023-11-18T20:44:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 -- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f82d406ae40 msgr2=0x7f82d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f82d406ae40 0x7f82d406d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f82e4036000 tx=0x7f82e4030000 comp rx=0 tx=0).stop 2023-11-18T20:44:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 -- 172.21.15.60:0/2625298367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 msgr2=0x7f82f40b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 0x7f82f40b6f00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c360 tx=0x7f82ec00c820 comp rx=0 tx=0).stop 2023-11-18T20:44:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 -- 172.21.15.60:0/2625298367 shutdown_connections 2023-11-18T20:44:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f82d406ae40 0x7f82d406d300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82f415d330 0x7f82f40b6f00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82f40d3900 0x7f82f40c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 --2- 172.21.15.60:0/2625298367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82f40d2910 0x7f82f4160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 -- 172.21.15.60:0/2625298367 >> 172.21.15.60:0/2625298367 conn(0x7f82f4072020 msgr2=0x7f82f4072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:28.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 -- 172.21.15.60:0/2625298367 shutdown_connections 2023-11-18T20:44:28.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.536+0000 7f82f92e0640 1 -- 172.21.15.60:0/2625298367 wait complete. 2023-11-18T20:44:28.548 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-18T20:44:28.549 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-18T20:44:28.549 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-18T20:44:28.549 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:44:28.549 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-18T20:44:28.549 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-18T20:44:28.549 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-18T20:44:28.549 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:44:28.550 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-18T20:44:28.550 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-18T20:44:28.550 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-18T20:44:28.550 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:44:28.550 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-18T20:44:28.550 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-18T20:44:28.550 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-18T20:44:28.551 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:44:28.558 INFO:tasks.ceph.mds.a:Stopped 2023-11-18T20:44:28.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-18T20:44:28.559 INFO:tasks.ceph.mds.c:Stopped 2023-11-18T20:44:28.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-18T20:44:28.565 INFO:tasks.ceph.mds.b:Stopped 2023-11-18T20:44:28.566 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-18T20:44:28.567 INFO:tasks.ceph.mds.d:Stopped 2023-11-18T20:44:28.567 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-18T20:44:28.600 INFO:tasks.ceph.mds.a:Started 2023-11-18T20:44:28.601 DEBUG:teuthology.parallel:result is None 2023-11-18T20:44:28.601 INFO:tasks.ceph.mds.c:Started 2023-11-18T20:44:28.602 DEBUG:teuthology.parallel:result is None 2023-11-18T20:44:28.609 INFO:tasks.ceph.mds.b:Started 2023-11-18T20:44:28.609 DEBUG:teuthology.parallel:result is None 2023-11-18T20:44:28.609 INFO:tasks.ceph.mds.d:Started 2023-11-18T20:44:28.610 DEBUG:teuthology.parallel:result is None 2023-11-18T20:44:28.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-18T20:44:28.682 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2023-11-18T20:44:28.683 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-11-18T20:44:28.692 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2023-11-18T20:44:28.693 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2023-11-18T20:44:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.721+0000 7f379ff41640 1 Processor -- start 2023-11-18T20:44:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.721+0000 7f379ff41640 1 -- start start 2023-11-18T20:44:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.722+0000 7f379ff41640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f379815d330 0x7f37980b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.722+0000 7f379ff41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37980d3900 0x7f37980b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.722+0000 7f379ff41640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37980b2b10 con 0x7f37980d2910 2023-11-18T20:44:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.722+0000 7f379ff41640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37980d1590 con 0x7f379815d330 2023-11-18T20:44:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.722+0000 7f379ff41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37980d1870 con 0x7f37980d3900 2023-11-18T20:44:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.723+0000 7f379dcdd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f379815d330 0x7f37980b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.723+0000 7f379e4de640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37980d3900 0x7f37980b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.723+0000 7f379d4dc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f37980d2910 0x7f37980d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:35818/0 (socket says 172.21.15.60:35818) 2023-11-18T20:44:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.723+0000 7f379d4dc640 1 -- 172.21.15.60:0/686430093 learned_addr learned my addr 172.21.15.60:0/686430093 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.723+0000 7f379e4de640 1 -- 172.21.15.60:0/686430093 >> v2:172.21.15.100:3300/0 conn(0x7f379815d330 msgr2=0x7f37980b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.723+0000 7f379e4de640 1 --2- 172.21.15.60:0/686430093 >> v2:172.21.15.100:3300/0 conn(0x7f379815d330 0x7f37980b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.723+0000 7f379e4de640 1 -- 172.21.15.60:0/686430093 >> v1:172.21.15.100:6789/0 conn(0x7f37980d2910 legacy=0x7f37980d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.723+0000 7f379e4de640 1 -- 172.21.15.60:0/686430093 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37980d1b50 con 0x7f37980d3900 2023-11-18T20:44:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.724+0000 7f379e4de640 1 --2- 172.21.15.60:0/686430093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37980d3900 0x7f37980b6390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3788009d90 tx=0x7f378802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771ec6862e7df8d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.724+0000 7f379ccdb640 1 -- 172.21.15.60:0/686430093 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378802cbc0 con 0x7f37980d3900 2023-11-18T20:44:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.724+0000 7f379ccdb640 1 -- 172.21.15.60:0/686430093 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378802cd20 con 0x7f37980d3900 2023-11-18T20:44:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.724+0000 7f379ccdb640 1 -- 172.21.15.60:0/686430093 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788034660 con 0x7f37980d3900 2023-11-18T20:44:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.724+0000 7f379ff41640 1 -- 172.21.15.60:0/686430093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37980d3900 msgr2=0x7f37980b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.724+0000 7f379ff41640 1 --2- 172.21.15.60:0/686430093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37980d3900 0x7f37980b6390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3788009d90 tx=0x7f378802be10 comp rx=0 tx=0).stop 2023-11-18T20:44:28.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.724+0000 7f379ff41640 1 -- 172.21.15.60:0/686430093 shutdown_connections 2023-11-18T20:44:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.724+0000 7f379ff41640 1 --2- 172.21.15.60:0/686430093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37980d3900 0x7f37980b6390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.724+0000 7f379ff41640 1 --2- 172.21.15.60:0/686430093 >> v2:172.21.15.100:3300/0 conn(0x7f379815d330 0x7f37980b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.724+0000 7f379ff41640 1 -- 172.21.15.60:0/686430093 >> 172.21.15.60:0/686430093 conn(0x7f3798072020 msgr2=0x7f3798072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.725+0000 7f379ff41640 1 -- 172.21.15.60:0/686430093 shutdown_connections 2023-11-18T20:44:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.725+0000 7f379ff41640 1 -- 172.21.15.60:0/686430093 wait complete. 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.726+0000 7f379ff41640 1 Processor -- start 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.726+0000 7f379ff41640 1 -- start start 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.726+0000 7f379ff41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37980d2910 0x7f3798151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.726+0000 7f379ff41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37980d3900 0x7f379815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.726+0000 7f379ff41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f379815d330 0x7f37980c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.726+0000 7f379ff41640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3798151580 con 0x7f37980d2910 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.726+0000 7f379ff41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379815f410 con 0x7f37980d3900 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.726+0000 7f379ff41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379815e370 con 0x7f379815d330 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379dcdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37980d2910 0x7f3798151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379e4de640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f379815d330 0x7f37980c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379dcdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37980d2910 0x7f3798151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35236/0 (socket says 172.21.15.60:35236) 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379dcdd640 1 -- 172.21.15.60:0/273986653 learned_addr learned my addr 172.21.15.60:0/273986653 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379d4dc640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37980d3900 0x7f379815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379d4dc640 1 -- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f379815d330 msgr2=0x7f37980c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379d4dc640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f379815d330 0x7f37980c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379d4dc640 1 -- 172.21.15.60:0/273986653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37980d2910 msgr2=0x7f3798151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379d4dc640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37980d2910 0x7f3798151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379d4dc640 1 -- 172.21.15.60:0/273986653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3788009a70 con 0x7f37980d3900 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379e4de640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f379815d330 0x7f37980c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379dcdd640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37980d2910 0x7f3798151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379d4dc640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37980d3900 0x7f379815d920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f378c00c440 tx=0x7f378c00c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f37967fc640 1 -- 172.21.15.60:0/273986653 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c005ce0 con 0x7f37980d3900 2023-11-18T20:44:28.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f37967fc640 1 -- 172.21.15.60:0/273986653 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c005e40 con 0x7f37980d3900 2023-11-18T20:44:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379ff41640 1 -- 172.21.15.60:0/273986653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379815e650 con 0x7f37980d3900 2023-11-18T20:44:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f37967fc640 1 -- 172.21.15.60:0/273986653 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c0072d0 con 0x7f37980d3900 2023-11-18T20:44:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.727+0000 7f379ff41640 1 -- 172.21.15.60:0/273986653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379815ec10 con 0x7f37980d3900 2023-11-18T20:44:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.728+0000 7f379ff41640 1 -- 172.21.15.60:0/273986653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3798069400 con 0x7f37980d3900 2023-11-18T20:44:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.729+0000 7f37967fc640 1 -- 172.21.15.60:0/273986653 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f378c0074f0 con 0x7f37980d3900 2023-11-18T20:44:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.729+0000 7f37967fc640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f378406ae40 0x7f378406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.729+0000 7f37967fc640 1 -- 172.21.15.60:0/273986653 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f378c0887b0 con 0x7f37980d3900 2023-11-18T20:44:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.729+0000 7f379dcdd640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f378406ae40 0x7f378406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.730+0000 7f379dcdd640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f378406ae40 0x7f378406d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f37980c89a0 tx=0x7f3790009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.734+0000 7f37967fc640 1 -- 172.21.15.60:0/273986653 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f378c053af0 con 0x7f37980d3900 2023-11-18T20:44:28.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:28.924+0000 7f379ff41640 1 -- 172.21.15.60:0/273986653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f379806a4d0 con 0x7f37980d3900 2023-11-18T20:44:29.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.685+0000 7f37967fc640 1 -- 172.21.15.60:0/273986653 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) v1 ==== 97+0+0 (secure 0 0 0) 0x7f378c057110 con 0x7f37980d3900 2023-11-18T20:44:29.687 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2023-11-18T20:44:29.690 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 -- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f378406ae40 msgr2=0x7f378406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f378406ae40 0x7f378406d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f37980c89a0 tx=0x7f3790009190 comp rx=0 tx=0).stop 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 -- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37980d3900 msgr2=0x7f379815d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37980d3900 0x7f379815d920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f378c00c440 tx=0x7f378c00c900 comp rx=0 tx=0).stop 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 -- 172.21.15.60:0/273986653 shutdown_connections 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f378406ae40 0x7f378406d300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f379815d330 0x7f37980c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37980d3900 0x7f379815d920 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 --2- 172.21.15.60:0/273986653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37980d2910 0x7f3798151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 -- 172.21.15.60:0/273986653 >> 172.21.15.60:0/273986653 conn(0x7f3798072020 msgr2=0x7f37980d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 -- 172.21.15.60:0/273986653 shutdown_connections 2023-11-18T20:44:29.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.689+0000 7f379ff41640 1 -- 172.21.15.60:0/273986653 wait complete. 2023-11-18T20:44:29.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-18T20:44:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.816+0000 7f0363d33640 1 Processor -- start 2023-11-18T20:44:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.817+0000 7f0363d33640 1 -- start start 2023-11-18T20:44:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.817+0000 7f0363d33640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c0ad640 0x7f035c06d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.817+0000 7f0363d33640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035c0adae0 con 0x7f035c06bbf0 2023-11-18T20:44:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.817+0000 7f0363d33640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035c0addc0 con 0x7f035c15d1d0 2023-11-18T20:44:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.817+0000 7f0363d33640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0ae0a0 con 0x7f035c0ad640 2023-11-18T20:44:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f03622d0640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f035c06bbf0 0x7f035c06d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49508/0 (socket says 172.21.15.60:49508) 2023-11-18T20:44:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f03622d0640 1 -- 172.21.15.60:0/4061690722 learned_addr learned my addr 172.21.15.60:0/4061690722 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:29.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f03612ce640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c0ad640 0x7f035c06d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819809602 0 0) 0x7f035c0adae0 con 0x7f035c06bbf0 2023-11-18T20:44:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0348003520 con 0x7f035c06bbf0 2023-11-18T20:44:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 105686866 0 0) 0x7f0348003520 con 0x7f035c06bbf0 2023-11-18T20:44:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c0ad640 msgr2=0x7f035c06d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c0ad640 0x7f035c06d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 >> v1:172.21.15.100:6789/0 conn(0x7f035c15d1d0 legacy=0x7f035c06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c0ae380 con 0x7f035c06bbf0 2023-11-18T20:44:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f03540019f0 con 0x7f035c06bbf0 2023-11-18T20:44:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 >> v1:172.21.15.60:6789/0 conn(0x7f035c06bbf0 legacy=0x7f035c06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c0ad640 0x7f03480039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0348003ee0 0x7f03480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348004d00 0x7f0348005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0ae0a0 con 0x7f0348003ee0 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034800b9e0 con 0x7f0348004d00 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034800bba0 con 0x7f035c0ad640 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f03612ce640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0348003ee0 0x7f03480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f0361acf640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c0ad640 0x7f03480039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.818+0000 7f03622d0640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348004d00 0x7f0348005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.819+0000 7f03612ce640 1 -- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c0ad640 msgr2=0x7f03480039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.819+0000 7f03612ce640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c0ad640 0x7f03480039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.819+0000 7f03612ce640 1 -- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348004d00 msgr2=0x7f0348005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.819+0000 7f03612ce640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348004d00 0x7f0348005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.819+0000 7f03612ce640 1 -- 172.21.15.60:0/4061690722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0348003520 con 0x7f0348003ee0 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.819+0000 7f03622d0640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348004d00 0x7f0348005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.819+0000 7f03612ce640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0348003ee0 0x7f03480047c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f034c009ac0 tx=0x7f03480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440c6b42867b551f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.820+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c02c600 con 0x7f0348003ee0 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.820+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c02c760 con 0x7f0348003ee0 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.820+0000 7f0360acd640 1 -- 172.21.15.60:0/4061690722 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c02caf0 con 0x7f0348003ee0 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.820+0000 7f0363d33640 1 -- 172.21.15.60:0/4061690722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0348003ee0 msgr2=0x7f03480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.820+0000 7f0363d33640 1 --2- 172.21.15.60:0/4061690722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0348003ee0 0x7f03480047c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f034c009ac0 tx=0x7f03480057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.820+0000 7f03622d0640 1 -- 172.21.15.60:0/4061690722 reap_dead start 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.820+0000 7f0363d33640 1 -- 172.21.15.60:0/4061690722 shutdown_connections 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.820+0000 7f0363d33640 1 -- 172.21.15.60:0/4061690722 >> 172.21.15.60:0/4061690722 conn(0x7f035c070e70 msgr2=0x7f035c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.820+0000 7f0363d33640 1 -- 172.21.15.60:0/4061690722 shutdown_connections 2023-11-18T20:44:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.820+0000 7f0363d33640 1 -- 172.21.15.60:0/4061690722 wait complete. 2023-11-18T20:44:29.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.821+0000 7f0363d33640 1 Processor -- start 2023-11-18T20:44:29.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.822+0000 7f0363d33640 1 -- start start 2023-11-18T20:44:29.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.822+0000 7f0363d33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348003ee0 0x7f035c15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:29.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.822+0000 7f0363d33640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c15e720 0x7f035c0ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:29.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.822+0000 7f0363d33640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f035c15acb0 0x7f035c15b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.822+0000 7f0363d33640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c15b650 con 0x7f035c15acb0 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.822+0000 7f0363d33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c15de80 con 0x7f0348003ee0 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.822+0000 7f0363d33640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c1f6150 con 0x7f035c15e720 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.822+0000 7f03622d0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f035c15acb0 0x7f035c15b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.822+0000 7f03622d0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f035c15acb0 0x7f035c15b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35248/0 (socket says 172.21.15.60:35248) 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.822+0000 7f03622d0640 1 -- 172.21.15.60:0/782005921 learned_addr learned my addr 172.21.15.60:0/782005921 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f0361acf640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348003ee0 0x7f035c15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f03612ce640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c15e720 0x7f035c0ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f03622d0640 1 -- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c15e720 msgr2=0x7f035c0ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f03622d0640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c15e720 0x7f035c0ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f03622d0640 1 -- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348003ee0 msgr2=0x7f035c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:29.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f03622d0640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348003ee0 0x7f035c15d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f03622d0640 1 -- 172.21.15.60:0/782005921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c1f6430 con 0x7f035c15acb0 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f0361acf640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348003ee0 0x7f035c15d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f03622d0640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f035c15acb0 0x7f035c15b110 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f035c06f440 tx=0x7f035400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f03527fc640 1 -- 172.21.15.60:0/782005921 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0354007ae0 con 0x7f035c15acb0 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f03527fc640 1 -- 172.21.15.60:0/782005921 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035400eda0 con 0x7f035c15acb0 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f03527fc640 1 -- 172.21.15.60:0/782005921 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0354013660 con 0x7f035c15acb0 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f0363d33640 1 -- 172.21.15.60:0/782005921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035c1f6710 con 0x7f035c15acb0 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.823+0000 7f0363d33640 1 -- 172.21.15.60:0/782005921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c1f6cd0 con 0x7f035c15acb0 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.824+0000 7f0363d33640 1 -- 172.21.15.60:0/782005921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035c1f2bd0 con 0x7f035c15acb0 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.825+0000 7f03527fc640 1 -- 172.21.15.60:0/782005921 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f03540137c0 con 0x7f035c15acb0 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.825+0000 7f03527fc640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f034006ae40 0x7f034006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.825+0000 7f03527fc640 1 -- 172.21.15.60:0/782005921 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f03540897a0 con 0x7f035c15acb0 2023-11-18T20:44:29.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.825+0000 7f0361acf640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f034006ae40 0x7f034006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:29.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.826+0000 7f0361acf640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f034006ae40 0x7f034006d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0358003700 tx=0x7f0358009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:29.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:29.830+0000 7f03527fc640 1 -- 172.21.15.60:0/782005921 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f035c1f2bd0 con 0x7f035c15acb0 2023-11-18T20:44:30.071 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.070+0000 7f0363d33640 1 -- 172.21.15.60:0/782005921 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f035c1f2bd0 con 0x7f035c15acb0 2023-11-18T20:44:30.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.070+0000 7f03527fc640 1 -- 172.21.15.60:0/782005921 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f035c1f2bd0 con 0x7f035c15acb0 2023-11-18T20:44:30.072 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:30.072 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:44:30.072 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2023-11-18T20:44:30.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.073 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2023-11-18T20:44:30.073 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllC9UnFhAACm0q2kkeuVd7s/HDJ4HxsQ==", 2023-11-18T20:44:30.073 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.073 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:44:30.073 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-11-18T20:44:30.073 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-11-18T20:44:30.074 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.074 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.074 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2023-11-18T20:44:30.074 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVlluYuEIxAAQk5LaZvlSTcwf9UNsO5a9Q==", 2023-11-18T20:44:30.074 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.074 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:44:30.075 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-11-18T20:44:30.075 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-11-18T20:44:30.075 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.075 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.075 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2023-11-18T20:44:30.075 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllaDI6HhAARs+hK3PNr6Pw7YXSHrks1g==", 2023-11-18T20:44:30.076 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.076 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:44:30.076 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-11-18T20:44:30.076 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-11-18T20:44:30.076 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.076 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.077 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2023-11-18T20:44:30.077 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllFO7dKhAAXNdgA3EauG4YWKi99qLLRg==", 2023-11-18T20:44:30.077 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.077 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:44:30.077 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-11-18T20:44:30.077 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-11-18T20:44:30.077 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.077 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.078 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2023-11-18T20:44:30.078 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCHIVlli+6+GBAAF9u74MMa50Abf4XBRswFpQ==", 2023-11-18T20:44:30.078 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.078 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:44:30.078 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCKIVlln59wFxAAiqokbWoD901iRvF5ESqEbw==", 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCNIVll3XquFhAA1zESpR80pRgZBgAC9U3aDw==", 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2023-11-18T20:44:30.080 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCQIVllMbZDFRAAVEOBFdVCedzteYM1w/enOA==", 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCWIVllnl4LFxAAbiNZODS0LQROAz7MfPIo8A==", 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:44:30.081 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:44:30.082 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.082 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.082 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2023-11-18T20:44:30.082 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCZIVllTPeyFRAAB/IGUS8d7KCVaEqyKA6e3g==", 2023-11-18T20:44:30.082 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.082 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCcIVllMIJ/FBAA7Wn5sn/6BBl+68DOSpAR5A==", 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:44:30.083 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCfIVllIAfgEhAAcOZ3Z5ZmYzIzqao5jmGb7A==", 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA==", 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllyxjzNhAAYt+5RXeMr8fnEZeiWX3b/w==", 2023-11-18T20:44:30.085 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.086 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:44:30.086 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-11-18T20:44:30.086 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-11-18T20:44:30.086 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-11-18T20:44:30.086 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.086 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2023-11-18T20:44:30.086 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVlltRRZMxAALjGvfOl1LnaZj/6RaATH4w==", 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCGIVllI90eARAAOKYZObsxr760sxu2J+xy0Q==", 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2023-11-18T20:44:30.087 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2023-11-18T20:44:30.088 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2023-11-18T20:44:30.088 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2023-11-18T20:44:30.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.088 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCEIVll68DQMhAAnNh3Zk4de+9NNtg+zs4Hmg==", 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVlljlFBEhAAQDo8Xz2m+fCZpazdjrBOfQ==", 2023-11-18T20:44:30.089 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQCFIVllkJ9gDRAAiqXLdSkKHV4SjIPBOA85YA==", 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2023-11-18T20:44:30.090 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2023-11-18T20:44:30.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2023-11-18T20:44:30.091 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.091 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:30.091 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-11-18T20:44:30.091 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:44:30.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 -- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f034006ae40 msgr2=0x7f034006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f034006ae40 0x7f034006d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0358003700 tx=0x7f0358009040 comp rx=0 tx=0).stop 2023-11-18T20:44:30.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 -- 172.21.15.60:0/782005921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f035c15acb0 msgr2=0x7f035c15b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f035c15acb0 0x7f035c15b110 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f035c06f440 tx=0x7f035400e530 comp rx=0 tx=0).stop 2023-11-18T20:44:30.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 -- 172.21.15.60:0/782005921 shutdown_connections 2023-11-18T20:44:30.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f034006ae40 0x7f034006d300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f035c15acb0 0x7f035c15b110 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f035c15e720 0x7f035c0ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 --2- 172.21.15.60:0/782005921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0348003ee0 0x7f035c15d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.094 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 -- 172.21.15.60:0/782005921 >> 172.21.15.60:0/782005921 conn(0x7f035c070e70 msgr2=0x7f035c1534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:30.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 -- 172.21.15.60:0/782005921 shutdown_connections 2023-11-18T20:44:30.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.072+0000 7f0363d33640 1 -- 172.21.15.60:0/782005921 wait complete. 2023-11-18T20:44:30.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:30.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.202+0000 7f76ef2da640 1 Processor -- start 2023-11-18T20:44:30.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.202+0000 7f76ef2da640 1 -- start start 2023-11-18T20:44:30.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76ef2da640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f76e8144980 0x7f76e815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.206 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76ef2da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e80ceb70 0x7f76e80cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76ef2da640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e8071770 con 0x7f76e80b4370 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76ef2da640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e80ccdc0 con 0x7f76e8144980 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76ef2da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80cd0a0 con 0x7f76e80ceb70 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76ed877640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f76e80b4370 0x7f76e8071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49524/0 (socket says 172.21.15.60:49524) 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76ed877640 1 -- 172.21.15.60:0/17040114 learned_addr learned my addr 172.21.15.60:0/17040114 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412428098 0 0) 0x7f76e8071770 con 0x7f76e80b4370 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76ec875640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e80ceb70 0x7f76e80cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76d4003520 con 0x7f76e80b4370 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76ed076640 1 --2- 172.21.15.60:0/17040114 >> v2:172.21.15.100:3300/0 conn(0x7f76e8144980 0x7f76e815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2361508861 0 0) 0x7f76d4003520 con 0x7f76e80b4370 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e80ceb70 msgr2=0x7f76e80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76dffff640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e80ceb70 0x7f76e80cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 >> v2:172.21.15.100:3300/0 conn(0x7f76e8144980 msgr2=0x7f76e815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76dffff640 1 --2- 172.21.15.60:0/17040114 >> v2:172.21.15.100:3300/0 conn(0x7f76e8144980 0x7f76e815c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e80cd380 con 0x7f76e80b4370 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f76e00019f0 con 0x7f76e80b4370 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.203+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 >> v1:172.21.15.60:6789/0 conn(0x7f76e80b4370 legacy=0x7f76e8071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76dffff640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d4003970 0x7f76d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76dffff640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d40047c0 0x7f76d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76dffff640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e80ceb70 0x7f76d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80ccdc0 con 0x7f76d4003970 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e80cd0a0 con 0x7f76d40047c0 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d400b980 con 0x7f76e80ceb70 2023-11-18T20:44:30.208 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76ed076640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d4003970 0x7f76d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76ed877640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e80ceb70 0x7f76d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76ec875640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d40047c0 0x7f76d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76ed076640 1 -- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e80ceb70 msgr2=0x7f76d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76ed076640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e80ceb70 0x7f76d4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76ed076640 1 -- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d40047c0 msgr2=0x7f76d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76ed076640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d40047c0 0x7f76d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76ed076640 1 -- 172.21.15.60:0/17040114 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4003520 con 0x7f76d4003970 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.204+0000 7f76ed877640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e80ceb70 0x7f76d4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76ed076640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d4003970 0x7f76d4003d90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009ac0 tx=0x7f76d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981229a3c0915cf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e402c710 con 0x7f76d4003970 2023-11-18T20:44:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e402c870 con 0x7f76d4003970 2023-11-18T20:44:30.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76dffff640 1 -- 172.21.15.60:0/17040114 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e402cc00 con 0x7f76d4003970 2023-11-18T20:44:30.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76ef2da640 1 -- 172.21.15.60:0/17040114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d4003970 msgr2=0x7f76d4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76ef2da640 1 --2- 172.21.15.60:0/17040114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76d4003970 0x7f76d4003d90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009ac0 tx=0x7f76d40057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76ed877640 1 -- 172.21.15.60:0/17040114 reap_dead start 2023-11-18T20:44:30.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76ef2da640 1 -- 172.21.15.60:0/17040114 shutdown_connections 2023-11-18T20:44:30.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76ef2da640 1 -- 172.21.15.60:0/17040114 >> 172.21.15.60:0/17040114 conn(0x7f76e806a4d0 msgr2=0x7f76e806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:30.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76ef2da640 1 -- 172.21.15.60:0/17040114 shutdown_connections 2023-11-18T20:44:30.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.205+0000 7f76ef2da640 1 -- 172.21.15.60:0/17040114 wait complete. 2023-11-18T20:44:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.207+0000 7f76ef2da640 1 Processor -- start 2023-11-18T20:44:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.207+0000 7f76ef2da640 1 -- start start 2023-11-18T20:44:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.207+0000 7f76ef2da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d4003970 0x7f76e815c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.207+0000 7f76ef2da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e813db20 0x7f76e81f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.207+0000 7f76ef2da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e815a700 0x7f76e815d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.207+0000 7f76ef2da640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e81f4120 con 0x7f76e815a700 2023-11-18T20:44:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.207+0000 7f76ef2da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e815bc80 con 0x7f76d4003970 2023-11-18T20:44:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.207+0000 7f76ef2da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e815b070 con 0x7f76e813db20 2023-11-18T20:44:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed877640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e815a700 0x7f76e815d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed877640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e815a700 0x7f76e815d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35260/0 (socket says 172.21.15.60:35260) 2023-11-18T20:44:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed877640 1 -- 172.21.15.60:0/2433734746 learned_addr learned my addr 172.21.15.60:0/2433734746 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed076640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d4003970 0x7f76e815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ec875640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e813db20 0x7f76e81f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed877640 1 -- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e813db20 msgr2=0x7f76e81f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed877640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e813db20 0x7f76e81f3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed877640 1 -- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d4003970 msgr2=0x7f76e815c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed877640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d4003970 0x7f76e815c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed877640 1 -- 172.21.15.60:0/2433734746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e815b350 con 0x7f76e815a700 2023-11-18T20:44:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed076640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d4003970 0x7f76e815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ed877640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e815a700 0x7f76e815d2b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f76e806e5c0 tx=0x7f76e000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ddffb640 1 -- 172.21.15.60:0/2433734746 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0007a80 con 0x7f76e815a700 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ddffb640 1 -- 172.21.15.60:0/2433734746 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e000eda0 con 0x7f76e815a700 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ddffb640 1 -- 172.21.15.60:0/2433734746 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0013660 con 0x7f76e815a700 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.208+0000 7f76ef2da640 1 -- 172.21.15.60:0/2433734746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e815b630 con 0x7f76e815a700 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.209+0000 7f76ef2da640 1 -- 172.21.15.60:0/2433734746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e815ab90 con 0x7f76e815a700 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.209+0000 7f76ef2da640 1 -- 172.21.15.60:0/2433734746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e8069400 con 0x7f76e815a700 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.210+0000 7f76ddffb640 1 -- 172.21.15.60:0/2433734746 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76e00137c0 con 0x7f76e815a700 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.210+0000 7f76ddffb640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76cc06ae70 0x7f76cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.210+0000 7f76ddffb640 1 -- 172.21.15.60:0/2433734746 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f76e0089720 con 0x7f76e815a700 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.210+0000 7f76ed076640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76cc06ae70 0x7f76cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.215 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.211+0000 7f76ed076640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76cc06ae70 0x7f76cc06d330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f76e4036000 tx=0x7f76e4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:30.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.214+0000 7f76ddffb640 1 -- 172.21.15.60:0/2433734746 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76e0054bb0 con 0x7f76e815a700 2023-11-18T20:44:30.392 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.391+0000 7f76ef2da640 1 -- 172.21.15.60:0/2433734746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76e815bdc0 con 0x7f76e815a700 2023-11-18T20:44:30.393 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.392+0000 7f76ddffb640 1 -- 172.21.15.60:0/2433734746 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+12845 (secure 0 0 0) 0x7f76e00581d0 con 0x7f76e815a700 2023-11-18T20:44:30.393 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:30.393 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":45,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:29.690105+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.393+0000 7f76ef2da640 1 -- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76cc06ae70 msgr2=0x7f76cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.393+0000 7f76ef2da640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76cc06ae70 0x7f76cc06d330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f76e4036000 tx=0x7f76e4030000 comp rx=0 tx=0).stop 2023-11-18T20:44:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.393+0000 7f76ef2da640 1 -- 172.21.15.60:0/2433734746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e815a700 msgr2=0x7f76e815d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.393+0000 7f76ef2da640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e815a700 0x7f76e815d2b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f76e806e5c0 tx=0x7f76e000e530 comp rx=0 tx=0).stop 2023-11-18T20:44:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.393+0000 7f76ef2da640 1 -- 172.21.15.60:0/2433734746 shutdown_connections 2023-11-18T20:44:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.393+0000 7f76ef2da640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f76cc06ae70 0x7f76cc06d330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.393+0000 7f76ef2da640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76e815a700 0x7f76e815d2b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.393+0000 7f76ef2da640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76e813db20 0x7f76e81f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.393+0000 7f76ef2da640 1 --2- 172.21.15.60:0/2433734746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76d4003970 0x7f76e815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.393+0000 7f76ef2da640 1 -- 172.21.15.60:0/2433734746 >> 172.21.15.60:0/2433734746 conn(0x7f76e806a4d0 msgr2=0x7f76e8155ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:30.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.394+0000 7f76ef2da640 1 -- 172.21.15.60:0/2433734746 shutdown_connections 2023-11-18T20:44:30.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.394+0000 7f76ef2da640 1 -- 172.21.15.60:0/2433734746 wait complete. 2023-11-18T20:44:30.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:44:30.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:44:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.514+0000 7f9ee7461640 1 Processor -- start 2023-11-18T20:44:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.514+0000 7f9ee7461640 1 -- start start 2023-11-18T20:44:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.515+0000 7f9ee7461640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 0x7f9ee006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.515+0000 7f9ee7461640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee00adae0 con 0x7f9ee015d1d0 2023-11-18T20:44:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.515+0000 7f9ee7461640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee00addc0 con 0x7f9ee006bbf0 2023-11-18T20:44:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.515+0000 7f9ee7461640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee00ae0a0 con 0x7f9ee00ad640 2023-11-18T20:44:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.515+0000 7f9ee51fd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9ee015d1d0 0x7f9ee006b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49530/0 (socket says 172.21.15.60:49530) 2023-11-18T20:44:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.515+0000 7f9ee51fd640 1 -- 172.21.15.60:0/1317047467 learned_addr learned my addr 172.21.15.60:0/1317047467 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.515+0000 7f9ee49fc640 1 --2- 172.21.15.60:0/1317047467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 0x7f9ee006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.515+0000 7f9edffff640 1 -- 172.21.15.60:0/1317047467 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062740234 0 0) 0x7f9ee00adae0 con 0x7f9ee015d1d0 2023-11-18T20:44:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.515+0000 7f9edffff640 1 -- 172.21.15.60:0/1317047467 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec8003520 con 0x7f9ee015d1d0 2023-11-18T20:44:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.516+0000 7f9edffff640 1 -- 172.21.15.60:0/1317047467 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511893609 0 0) 0x7f9ee00addc0 con 0x7f9ee006bbf0 2023-11-18T20:44:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.516+0000 7f9edffff640 1 -- 172.21.15.60:0/1317047467 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ee00adae0 con 0x7f9ee006bbf0 2023-11-18T20:44:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.516+0000 7f9edffff640 1 -- 172.21.15.60:0/1317047467 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3941941862 0 0) 0x7f9ec8003520 con 0x7f9ee015d1d0 2023-11-18T20:44:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.516+0000 7f9ee49fc640 1 -- 172.21.15.60:0/1317047467 >> v1:172.21.15.100:6789/0 conn(0x7f9ee006bbf0 legacy=0x7f9ee006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.516+0000 7f9ee49fc640 1 -- 172.21.15.60:0/1317047467 >> v1:172.21.15.60:6789/0 conn(0x7f9ee015d1d0 legacy=0x7f9ee006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.516+0000 7f9ee49fc640 1 -- 172.21.15.60:0/1317047467 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee00ae380 con 0x7f9ee00ad640 2023-11-18T20:44:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9ee49fc640 1 --2- 172.21.15.60:0/1317047467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 0x7f9ee006d0b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009800 tx=0x7f9ed402b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637ba42486bdce57 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9edffff640 1 -- 172.21.15.60:0/1317047467 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8001670 con 0x7f9ee00ad640 2023-11-18T20:44:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9edffff640 1 -- 172.21.15.60:0/1317047467 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed40314d0 con 0x7f9ee00ad640 2023-11-18T20:44:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9edffff640 1 -- 172.21.15.60:0/1317047467 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4031840 con 0x7f9ee00ad640 2023-11-18T20:44:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9ee7461640 1 -- 172.21.15.60:0/1317047467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 msgr2=0x7f9ee006d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9ee7461640 1 --2- 172.21.15.60:0/1317047467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 0x7f9ee006d0b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009800 tx=0x7f9ed402b7f0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9ee7461640 1 -- 172.21.15.60:0/1317047467 shutdown_connections 2023-11-18T20:44:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9ee7461640 1 --2- 172.21.15.60:0/1317047467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 0x7f9ee006d0b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9ee7461640 1 -- 172.21.15.60:0/1317047467 >> 172.21.15.60:0/1317047467 conn(0x7f9ee0070e70 msgr2=0x7f9ee0071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9ee7461640 1 -- 172.21.15.60:0/1317047467 shutdown_connections 2023-11-18T20:44:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.517+0000 7f9ee7461640 1 -- 172.21.15.60:0/1317047467 wait complete. 2023-11-18T20:44:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.518+0000 7f9ee7461640 1 Processor -- start 2023-11-18T20:44:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.519+0000 7f9ee7461640 1 -- start start 2023-11-18T20:44:30.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.519+0000 7f9ee7461640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee006bbf0 0x7f9ee0141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.519+0000 7f9ee7461640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 0x7f9ee00cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.519+0000 7f9ee7461640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ee015d1d0 0x7f9ee00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.519+0000 7f9ee7461640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0157e30 con 0x7f9ee006bbf0 2023-11-18T20:44:30.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.519+0000 7f9ee7461640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0157240 con 0x7f9ee015d1d0 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.519+0000 7f9ee7461640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0157490 con 0x7f9ee00ad640 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.519+0000 7f9ee51fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee006bbf0 0x7f9ee0141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.519+0000 7f9ee51fd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee006bbf0 0x7f9ee0141cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35274/0 (socket says 172.21.15.60:35274) 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.519+0000 7f9ee51fd640 1 -- 172.21.15.60:0/2394563370 learned_addr learned my addr 172.21.15.60:0/2394563370 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee49fc640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 0x7f9ee00cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee59fe640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ee015d1d0 0x7f9ee00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee51fd640 1 -- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 msgr2=0x7f9ee00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee51fd640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 0x7f9ee00cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee51fd640 1 -- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ee015d1d0 msgr2=0x7f9ee00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee51fd640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ee015d1d0 0x7f9ee00ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee51fd640 1 -- 172.21.15.60:0/2394563370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed4009400 con 0x7f9ee006bbf0 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee59fe640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ee015d1d0 0x7f9ee00ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee51fd640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee006bbf0 0x7f9ee0141cb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800c730 tx=0x7f9ed800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9eddffb640 1 -- 172.21.15.60:0/2394563370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8005ad0 con 0x7f9ee006bbf0 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9eddffb640 1 -- 172.21.15.60:0/2394563370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed8005c30 con 0x7f9ee006bbf0 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee7461640 1 -- 172.21.15.60:0/2394563370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee0157770 con 0x7f9ee006bbf0 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9eddffb640 1 -- 172.21.15.60:0/2394563370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8007070 con 0x7f9ee006bbf0 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.520+0000 7f9ee7461640 1 -- 172.21.15.60:0/2394563370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee01421f0 con 0x7f9ee006bbf0 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.521+0000 7f9ee7461640 1 -- 172.21.15.60:0/2394563370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee0157f70 con 0x7f9ee006bbf0 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.522+0000 7f9eddffb640 1 -- 172.21.15.60:0/2394563370 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ed80071d0 con 0x7f9ee006bbf0 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.522+0000 7f9eddffb640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ecc06ae40 0x7f9ecc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.522+0000 7f9eddffb640 1 -- 172.21.15.60:0/2394563370 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9ed8088870 con 0x7f9ee006bbf0 2023-11-18T20:44:30.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.522+0000 7f9ee49fc640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ecc06ae40 0x7f9ecc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.523+0000 7f9ee49fc640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ecc06ae40 0x7f9ecc06d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4036000 tx=0x7f9ed4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.527+0000 7f9eddffb640 1 -- 172.21.15.60:0/2394563370 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ee0157f70 con 0x7f9ee006bbf0 2023-11-18T20:44:30.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.704+0000 7f9ee7461640 1 -- 172.21.15.60:0/2394563370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ee0157f70 con 0x7f9ee006bbf0 2023-11-18T20:44:30.706 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.704+0000 7f9eddffb640 1 -- 172.21.15.60:0/2394563370 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ee0157f70 con 0x7f9ee006bbf0 2023-11-18T20:44:30.707 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:44:30.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 -- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ecc06ae40 msgr2=0x7f9ecc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ecc06ae40 0x7f9ecc06d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4036000 tx=0x7f9ed4008030 comp rx=0 tx=0).stop 2023-11-18T20:44:30.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 -- 172.21.15.60:0/2394563370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee006bbf0 msgr2=0x7f9ee0141cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee006bbf0 0x7f9ee0141cb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800c730 tx=0x7f9ed800cbf0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.708 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 -- 172.21.15.60:0/2394563370 shutdown_connections 2023-11-18T20:44:30.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ecc06ae40 0x7f9ecc06d300 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ee015d1d0 0x7f9ee00ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ee00ad640 0x7f9ee00cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 --2- 172.21.15.60:0/2394563370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ee006bbf0 0x7f9ee0141cb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 -- 172.21.15.60:0/2394563370 >> 172.21.15.60:0/2394563370 conn(0x7f9ee0070e70 msgr2=0x7f9ee006ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:30.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 -- 172.21.15.60:0/2394563370 shutdown_connections 2023-11-18T20:44:30.709 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.706+0000 7f9ee7461640 1 -- 172.21.15.60:0/2394563370 wait complete. 2023-11-18T20:44:30.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-18T20:44:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.823+0000 7f79f80e0640 1 Processor -- start 2023-11-18T20:44:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.824+0000 7f79f80e0640 1 -- start start 2023-11-18T20:44:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.824+0000 7f79f80e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 0x7f79f00d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.824+0000 7f79f80e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79f00d3730 0x7f79f00b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.824+0000 7f79f80e0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f00b2b10 con 0x7f79f015d330 2023-11-18T20:44:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.824+0000 7f79f80e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f00d3b50 con 0x7f79f00d3730 2023-11-18T20:44:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.824+0000 7f79f80e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00d1590 con 0x7f79f00d2910 2023-11-18T20:44:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.825+0000 7f79f667d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79f00d3730 0x7f79f00b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.825+0000 7f79f5e7c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f79f015d330 0x7f79f00b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55364/0 (socket says 172.21.15.60:55364) 2023-11-18T20:44:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.825+0000 7f79f5e7c640 1 -- 172.21.15.60:0/2981438714 learned_addr learned my addr 172.21.15.60:0/2981438714 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.825+0000 7f79f567b640 1 --2- 172.21.15.60:0/2981438714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 0x7f79f00d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.825+0000 7f79f667d640 1 -- 172.21.15.60:0/2981438714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 msgr2=0x7f79f00d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.825+0000 7f79f667d640 1 --2- 172.21.15.60:0/2981438714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 0x7f79f00d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.825+0000 7f79f667d640 1 -- 172.21.15.60:0/2981438714 >> v1:172.21.15.60:6789/0 conn(0x7f79f015d330 legacy=0x7f79f00b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.825+0000 7f79f4e7a640 1 -- 172.21.15.60:0/2981438714 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532208521 0 0) 0x7f79f00b2b10 con 0x7f79f015d330 2023-11-18T20:44:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.825+0000 7f79f667d640 1 -- 172.21.15.60:0/2981438714 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f00d1870 con 0x7f79f00d3730 2023-11-18T20:44:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.825+0000 7f79f567b640 1 --2- 172.21.15.60:0/2981438714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 0x7f79f00d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:30.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.826+0000 7f79f667d640 1 --2- 172.21.15.60:0/2981438714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f00d3730 0x7f79f00b6390 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009d90 tx=0x7f79e802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180f2b5fb7950415 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.826+0000 7f79f4e7a640 1 -- 172.21.15.60:0/2981438714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e802cbc0 con 0x7f79f00d3730 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.826+0000 7f79f4e7a640 1 -- 172.21.15.60:0/2981438714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e802cd20 con 0x7f79f00d3730 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.826+0000 7f79f4e7a640 1 -- 172.21.15.60:0/2981438714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8034660 con 0x7f79f00d3730 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.826+0000 7f79f80e0640 1 -- 172.21.15.60:0/2981438714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f00d3730 msgr2=0x7f79f00b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.826+0000 7f79f80e0640 1 --2- 172.21.15.60:0/2981438714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f00d3730 0x7f79f00b6390 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009d90 tx=0x7f79e802be10 comp rx=0 tx=0).stop 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.826+0000 7f79f80e0640 1 -- 172.21.15.60:0/2981438714 shutdown_connections 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.826+0000 7f79f80e0640 1 --2- 172.21.15.60:0/2981438714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f00d3730 0x7f79f00b6390 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.826+0000 7f79f80e0640 1 --2- 172.21.15.60:0/2981438714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 0x7f79f00d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.826+0000 7f79f80e0640 1 -- 172.21.15.60:0/2981438714 >> 172.21.15.60:0/2981438714 conn(0x7f79f0072020 msgr2=0x7f79f0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.827+0000 7f79f80e0640 1 -- 172.21.15.60:0/2981438714 shutdown_connections 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.827+0000 7f79f80e0640 1 -- 172.21.15.60:0/2981438714 wait complete. 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.828+0000 7f79f80e0640 1 Processor -- start 2023-11-18T20:44:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.828+0000 7f79f80e0640 1 -- start start 2023-11-18T20:44:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.828+0000 7f79f80e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 0x7f79f00c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.828+0000 7f79f80e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f00d3730 0x7f79f00c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.828+0000 7f79f80e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f015d330 0x7f79f01f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.828+0000 7f79f80e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00c7620 con 0x7f79f015d330 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.828+0000 7f79f80e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f0157ff0 con 0x7f79f00d3730 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.828+0000 7f79f80e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00c7840 con 0x7f79f00d2910 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.828+0000 7f79f667d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f015d330 0x7f79f01f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f667d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f015d330 0x7f79f01f2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49788/0 (socket says 172.21.15.60:49788) 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f667d640 1 -- 172.21.15.60:0/2089612906 learned_addr learned my addr 172.21.15.60:0/2089612906 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f5e7c640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 0x7f79f00c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f567b640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f00d3730 0x7f79f00c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f667d640 1 -- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 msgr2=0x7f79f00c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f667d640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 0x7f79f00c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f667d640 1 -- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f00d3730 msgr2=0x7f79f00c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f667d640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f00d3730 0x7f79f00c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f667d640 1 -- 172.21.15.60:0/2089612906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e8009a70 con 0x7f79f015d330 2023-11-18T20:44:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f567b640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f00d3730 0x7f79f00c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f667d640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f015d330 0x7f79f01f2b40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f79e8037040 tx=0x7f79e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79e67fc640 1 -- 172.21.15.60:0/2089612906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8040650 con 0x7f79f015d330 2023-11-18T20:44:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79e67fc640 1 -- 172.21.15.60:0/2089612906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e802ce90 con 0x7f79f015d330 2023-11-18T20:44:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f80e0640 1 -- 172.21.15.60:0/2089612906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f00c7b20 con 0x7f79f015d330 2023-11-18T20:44:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79e67fc640 1 -- 172.21.15.60:0/2089612906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80409f0 con 0x7f79f015d330 2023-11-18T20:44:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.829+0000 7f79f80e0640 1 -- 172.21.15.60:0/2089612906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f00c80e0 con 0x7f79f015d330 2023-11-18T20:44:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.830+0000 7f79f80e0640 1 -- 172.21.15.60:0/2089612906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f0069400 con 0x7f79f015d330 2023-11-18T20:44:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.831+0000 7f79e67fc640 1 -- 172.21.15.60:0/2089612906 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79e8040b50 con 0x7f79f015d330 2023-11-18T20:44:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.831+0000 7f79e67fc640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79d406ae40 0x7f79d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.831+0000 7f79e67fc640 1 -- 172.21.15.60:0/2089612906 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f79e80ac0b0 con 0x7f79f015d330 2023-11-18T20:44:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.831+0000 7f79f5e7c640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79d406ae40 0x7f79d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.832+0000 7f79f5e7c640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79d406ae40 0x7f79d406d300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f79ec000f60 tx=0x7f79ec007b50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:30.835+0000 7f79e67fc640 1 -- 172.21.15.60:0/2089612906 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79e80774b0 con 0x7f79f015d330 2023-11-18T20:44:31.017 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.016+0000 7f79f80e0640 1 -- 172.21.15.60:0/2089612906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f79f00b6010 con 0x7f79f015d330 2023-11-18T20:44:31.018 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.016+0000 7f79e67fc640 1 -- 172.21.15.60:0/2089612906 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) v1 ==== 84+0+65 (secure 0 0 0) 0x7f79f00b6010 con 0x7f79f015d330 2023-11-18T20:44:31.018 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:31.018 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2023-11-18T20:44:31.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2023-11-18T20:44:31.018 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2023-11-18T20:44:31.018 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2023-11-18T20:44:31.018 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:31.018 INFO:teuthology.orchestra.run.smithi060.stdout:] 2023-11-18T20:44:31.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.018+0000 7f79f80e0640 1 -- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79d406ae40 msgr2=0x7f79d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.018+0000 7f79f80e0640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79d406ae40 0x7f79d406d300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f79ec000f60 tx=0x7f79ec007b50 comp rx=0 tx=0).stop 2023-11-18T20:44:31.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.018+0000 7f79f80e0640 1 -- 172.21.15.60:0/2089612906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f015d330 msgr2=0x7f79f01f2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.018+0000 7f79f80e0640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f015d330 0x7f79f01f2b40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f79e8037040 tx=0x7f79e8004a40 comp rx=0 tx=0).stop 2023-11-18T20:44:31.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.018+0000 7f79f80e0640 1 -- 172.21.15.60:0/2089612906 shutdown_connections 2023-11-18T20:44:31.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.018+0000 7f79f80e0640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f79d406ae40 0x7f79d406d300 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.018+0000 7f79f80e0640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79f015d330 0x7f79f01f2b40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.020 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.018+0000 7f79f80e0640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79f00d3730 0x7f79f00c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.018+0000 7f79f80e0640 1 --2- 172.21.15.60:0/2089612906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79f00d2910 0x7f79f00c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.019+0000 7f79f80e0640 1 -- 172.21.15.60:0/2089612906 >> 172.21.15.60:0/2089612906 conn(0x7f79f0072020 msgr2=0x7f79f00b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:31.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.019+0000 7f79f80e0640 1 -- 172.21.15.60:0/2089612906 shutdown_connections 2023-11-18T20:44:31.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.019+0000 7f79f80e0640 1 -- 172.21.15.60:0/2089612906 wait complete. 2023-11-18T20:44:31.030 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-18T20:44:31.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-18T20:44:31.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.140+0000 7f4e794a2640 1 Processor -- start 2023-11-18T20:44:31.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.140+0000 7f4e794a2640 1 -- start start 2023-11-18T20:44:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.140+0000 7f4e794a2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740b4370 0x7f4e740b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.140+0000 7f4e794a2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e74071770 con 0x7f4e74144980 2023-11-18T20:44:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.140+0000 7f4e794a2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e740ccb60 con 0x7f4e740ceb70 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.140+0000 7f4e794a2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7415ced0 con 0x7f4e740b4370 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e72ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4e74144980 0x7f4e7415c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55366/0 (socket says 172.21.15.60:55366) 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e72ffd640 1 -- 172.21.15.60:0/1816213537 learned_addr learned my addr 172.21.15.60:0/1816213537 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e737fe640 1 --2- 172.21.15.60:0/1816213537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740b4370 0x7f4e740b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e71ffb640 1 -- 172.21.15.60:0/1816213537 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101999403 0 0) 0x7f4e74071770 con 0x7f4e74144980 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e71ffb640 1 -- 172.21.15.60:0/1816213537 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e5c003520 con 0x7f4e74144980 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e71ffb640 1 -- 172.21.15.60:0/1816213537 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053711288 0 0) 0x7f4e740ccb60 con 0x7f4e740ceb70 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e71ffb640 1 -- 172.21.15.60:0/1816213537 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e74071770 con 0x7f4e740ceb70 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e737fe640 1 -- 172.21.15.60:0/1816213537 >> v1:172.21.15.100:6789/0 conn(0x7f4e740ceb70 legacy=0x7f4e740cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e737fe640 1 -- 172.21.15.60:0/1816213537 >> v1:172.21.15.60:6789/0 conn(0x7f4e74144980 legacy=0x7f4e7415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e71ffb640 1 -- 172.21.15.60:0/1816213537 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1602188928 0 0) 0x7f4e5c003520 con 0x7f4e74144980 2023-11-18T20:44:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.141+0000 7f4e737fe640 1 -- 172.21.15.60:0/1816213537 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e740b4ce0 con 0x7f4e740b4370 2023-11-18T20:44:31.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.142+0000 7f4e737fe640 1 --2- 172.21.15.60:0/1816213537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740b4370 0x7f4e740b47a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4e740b4e70 tx=0x7f4e6c02bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e327bdcaffb5c84 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:31.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.142+0000 7f4e71ffb640 1 -- 172.21.15.60:0/1816213537 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64001670 con 0x7f4e740b4370 2023-11-18T20:44:31.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.142+0000 7f4e71ffb640 1 -- 172.21.15.60:0/1816213537 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c02c910 con 0x7f4e740b4370 2023-11-18T20:44:31.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.142+0000 7f4e71ffb640 1 -- 172.21.15.60:0/1816213537 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c02cca0 con 0x7f4e740b4370 2023-11-18T20:44:31.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.143+0000 7f4e794a2640 1 -- 172.21.15.60:0/1816213537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740b4370 msgr2=0x7f4e740b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.143+0000 7f4e794a2640 1 --2- 172.21.15.60:0/1816213537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740b4370 0x7f4e740b47a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4e740b4e70 tx=0x7f4e6c02bde0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.143+0000 7f4e794a2640 1 -- 172.21.15.60:0/1816213537 shutdown_connections 2023-11-18T20:44:31.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.143+0000 7f4e794a2640 1 --2- 172.21.15.60:0/1816213537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740b4370 0x7f4e740b47a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.143+0000 7f4e794a2640 1 -- 172.21.15.60:0/1816213537 >> 172.21.15.60:0/1816213537 conn(0x7f4e7406a4d0 msgr2=0x7f4e7406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:31.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.143+0000 7f4e794a2640 1 -- 172.21.15.60:0/1816213537 shutdown_connections 2023-11-18T20:44:31.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.143+0000 7f4e794a2640 1 -- 172.21.15.60:0/1816213537 wait complete. 2023-11-18T20:44:31.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.144+0000 7f4e794a2640 1 Processor -- start 2023-11-18T20:44:31.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.144+0000 7f4e794a2640 1 -- start start 2023-11-18T20:44:31.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e794a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e740b4370 0x7f4e741ee120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e794a2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740ceb70 0x7f4e741f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e794a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e74144980 0x7f4e7413df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e794a2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e741ee660 con 0x7f4e740b4370 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e794a2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7413eb00 con 0x7f4e74144980 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e794a2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e74158a90 con 0x7f4e740ceb70 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e72ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e740b4370 0x7f4e741ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e72ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e740b4370 0x7f4e741ee120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49804/0 (socket says 172.21.15.60:49804) 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e72ffd640 1 -- 172.21.15.60:0/3147751615 learned_addr learned my addr 172.21.15.60:0/3147751615 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e737fe640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e74144980 0x7f4e7413df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e727fc640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740ceb70 0x7f4e741f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e72ffd640 1 -- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740ceb70 msgr2=0x7f4e741f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e72ffd640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740ceb70 0x7f4e741f1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e72ffd640 1 -- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e74144980 msgr2=0x7f4e7413df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e72ffd640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e74144980 0x7f4e7413df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e72ffd640 1 -- 172.21.15.60:0/3147751615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c009a70 con 0x7f4e740b4370 2023-11-18T20:44:31.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e737fe640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e74144980 0x7f4e7413df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:31.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e72ffd640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e740b4370 0x7f4e741ee120 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400c780 tx=0x7f4e6400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:31.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e5b7fe640 1 -- 172.21.15.60:0/3147751615 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64005ad0 con 0x7f4e740b4370 2023-11-18T20:44:31.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e5b7fe640 1 -- 172.21.15.60:0/3147751615 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e64005c50 con 0x7f4e740b4370 2023-11-18T20:44:31.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.145+0000 7f4e794a2640 1 -- 172.21.15.60:0/3147751615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e74155e60 con 0x7f4e740b4370 2023-11-18T20:44:31.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.146+0000 7f4e5b7fe640 1 -- 172.21.15.60:0/3147751615 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64007070 con 0x7f4e740b4370 2023-11-18T20:44:31.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.146+0000 7f4e794a2640 1 -- 172.21.15.60:0/3147751615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e74156420 con 0x7f4e740b4370 2023-11-18T20:44:31.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.147+0000 7f4e794a2640 1 -- 172.21.15.60:0/3147751615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e74069400 con 0x7f4e740b4370 2023-11-18T20:44:31.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.147+0000 7f4e5b7fe640 1 -- 172.21.15.60:0/3147751615 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e640071d0 con 0x7f4e740b4370 2023-11-18T20:44:31.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.147+0000 7f4e5b7fe640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4e6006ae40 0x7f4e6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.147+0000 7f4e5b7fe640 1 -- 172.21.15.60:0/3147751615 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4e64088960 con 0x7f4e740b4370 2023-11-18T20:44:31.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.148+0000 7f4e727fc640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4e6006ae40 0x7f4e6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.148+0000 7f4e727fc640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4e6006ae40 0x7f4e6006d300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4e74143ad0 tx=0x7f4e68009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:31.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.152+0000 7f4e5b7fe640 1 -- 172.21.15.60:0/3147751615 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e64053df0 con 0x7f4e740b4370 2023-11-18T20:44:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.327+0000 7f4e794a2640 1 -- 172.21.15.60:0/3147751615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4e74067600 con 0x7f4e740b4370 2023-11-18T20:44:31.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.683+0000 7f4e5b7fe640 1 -- 172.21.15.60:0/3147751615 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4e64017020 con 0x7f4e740b4370 2023-11-18T20:44:31.685 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2023-11-18T20:44:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 -- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4e6006ae40 msgr2=0x7f4e6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4e6006ae40 0x7f4e6006d300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4e74143ad0 tx=0x7f4e68009040 comp rx=0 tx=0).stop 2023-11-18T20:44:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 -- 172.21.15.60:0/3147751615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e740b4370 msgr2=0x7f4e741ee120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e740b4370 0x7f4e741ee120 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400c780 tx=0x7f4e6400cc40 comp rx=0 tx=0).stop 2023-11-18T20:44:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 -- 172.21.15.60:0/3147751615 shutdown_connections 2023-11-18T20:44:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4e6006ae40 0x7f4e6006d300 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e74144980 0x7f4e7413df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e740ceb70 0x7f4e741f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 --2- 172.21.15.60:0/3147751615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e740b4370 0x7f4e741ee120 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 -- 172.21.15.60:0/3147751615 >> 172.21.15.60:0/3147751615 conn(0x7f4e7406a4d0 msgr2=0x7f4e7406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:31.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 -- 172.21.15.60:0/3147751615 shutdown_connections 2023-11-18T20:44:31.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.686+0000 7f4e794a2640 1 -- 172.21.15.60:0/3147751615 wait complete. 2023-11-18T20:44:31.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-18T20:44:31.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.817+0000 7f913bef2640 1 Processor -- start 2023-11-18T20:44:31.819 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.817+0000 7f913bef2640 1 -- start start 2023-11-18T20:44:31.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.818+0000 7f913bef2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 0x7f913406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.818+0000 7f913bef2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9134143380 0x7f913415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.818+0000 7f913bef2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913406be80 con 0x7f913406ba20 2023-11-18T20:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.818+0000 7f913bef2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91340b2820 con 0x7f9134143380 2023-11-18T20:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.818+0000 7f913bef2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9134140900 con 0x7f9134140520 2023-11-18T20:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.818+0000 7f9139c8e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 0x7f913406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.818+0000 7f913a48f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9134143380 0x7f913415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.818+0000 7f9139c8e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 0x7f913406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:44640/0 (socket says 172.21.15.60:44640) 2023-11-18T20:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.818+0000 7f9139c8e640 1 -- 172.21.15.60:0/899249127 learned_addr learned my addr 172.21.15.60:0/899249127 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.819+0000 7f9139c8e640 1 -- 172.21.15.60:0/899249127 >> v2:172.21.15.100:3300/0 conn(0x7f9134143380 msgr2=0x7f913415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.819+0000 7f9139c8e640 1 --2- 172.21.15.60:0/899249127 >> v2:172.21.15.100:3300/0 conn(0x7f9134143380 0x7f913415d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.819+0000 7f9139c8e640 1 -- 172.21.15.60:0/899249127 >> v1:172.21.15.100:6789/0 conn(0x7f913406ba20 legacy=0x7f9134142c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.819+0000 7f9139c8e640 1 -- 172.21.15.60:0/899249127 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91341439a0 con 0x7f9134140520 2023-11-18T20:44:31.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.819+0000 7f9139c8e640 1 --2- 172.21.15.60:0/899249127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 0x7f913406b4e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f912c009870 tx=0x7f912c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb3fba875ce2a2a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:31.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f9138c8c640 1 -- 172.21.15.60:0/899249127 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c030840 con 0x7f9134140520 2023-11-18T20:44:31.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f9138c8c640 1 -- 172.21.15.60:0/899249127 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c0309a0 con 0x7f9134140520 2023-11-18T20:44:31.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f9138c8c640 1 -- 172.21.15.60:0/899249127 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c030d30 con 0x7f9134140520 2023-11-18T20:44:31.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f913bef2640 1 -- 172.21.15.60:0/899249127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 msgr2=0x7f913406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f913bef2640 1 --2- 172.21.15.60:0/899249127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 0x7f913406b4e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f912c009870 tx=0x7f912c02b980 comp rx=0 tx=0).stop 2023-11-18T20:44:31.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f913bef2640 1 -- 172.21.15.60:0/899249127 shutdown_connections 2023-11-18T20:44:31.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f913bef2640 1 --2- 172.21.15.60:0/899249127 >> v2:172.21.15.100:3300/0 conn(0x7f9134143380 0x7f913415d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f913bef2640 1 --2- 172.21.15.60:0/899249127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 0x7f913406b4e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f913bef2640 1 -- 172.21.15.60:0/899249127 >> 172.21.15.60:0/899249127 conn(0x7f9134070c70 msgr2=0x7f9134071080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:31.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f913bef2640 1 -- 172.21.15.60:0/899249127 shutdown_connections 2023-11-18T20:44:31.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.820+0000 7f913bef2640 1 -- 172.21.15.60:0/899249127 wait complete. 2023-11-18T20:44:31.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.821+0000 7f913bef2640 1 Processor -- start 2023-11-18T20:44:31.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.822+0000 7f913bef2640 1 -- start start 2023-11-18T20:44:31.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.822+0000 7f913bef2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f913406ba20 0x7f9134154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.826 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.822+0000 7f913bef2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 0x7f91341f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.822+0000 7f913bef2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9134143380 0x7f913406d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.822+0000 7f913bef2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9134155480 con 0x7f9134143380 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.822+0000 7f913bef2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9134155d70 con 0x7f913406ba20 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.822+0000 7f913bef2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9134155fc0 con 0x7f9134140520 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.822+0000 7f913a48f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9134143380 0x7f913406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.822+0000 7f913a48f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9134143380 0x7f913406d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49806/0 (socket says 172.21.15.60:49806) 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f913a48f640 1 -- 172.21.15.60:0/1159618527 learned_addr learned my addr 172.21.15.60:0/1159618527 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f913948d640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 0x7f91341f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f9139c8e640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f913406ba20 0x7f9134154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f913a48f640 1 -- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 msgr2=0x7f91341f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f913a48f640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 0x7f91341f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f913a48f640 1 -- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f913406ba20 msgr2=0x7f9134154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f913a48f640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f913406ba20 0x7f9134154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f913a48f640 1 -- 172.21.15.60:0/1159618527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c0094d0 con 0x7f9134143380 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f913a48f640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9134143380 0x7f913406d580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f912400efc0 tx=0x7f912400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f91327fc640 1 -- 172.21.15.60:0/1159618527 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124014480 con 0x7f9134143380 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f91327fc640 1 -- 172.21.15.60:0/1159618527 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912400f040 con 0x7f9134143380 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f91327fc640 1 -- 172.21.15.60:0/1159618527 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124014820 con 0x7f9134143380 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f913bef2640 1 -- 172.21.15.60:0/1159618527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91341562a0 con 0x7f9134143380 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.823+0000 7f913bef2640 1 -- 172.21.15.60:0/1159618527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9134156860 con 0x7f9134143380 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.824+0000 7f913bef2640 1 -- 172.21.15.60:0/1159618527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9134156a80 con 0x7f9134143380 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.825+0000 7f91327fc640 1 -- 172.21.15.60:0/1159618527 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9124014980 con 0x7f9134143380 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.825+0000 7f91327fc640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f912006ae40 0x7f912006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.825+0000 7f91327fc640 1 -- 172.21.15.60:0/1159618527 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f912408a570 con 0x7f9134143380 2023-11-18T20:44:31.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.826+0000 7f9139c8e640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f912006ae40 0x7f912006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:31.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.826+0000 7f9139c8e640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f912006ae40 0x7f912006d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f912c036000 tx=0x7f912c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:31.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:31.829+0000 7f91327fc640 1 -- 172.21.15.60:0/1159618527 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9124055a20 con 0x7f9134143380 2023-11-18T20:44:32.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.005+0000 7f913bef2640 1 -- 172.21.15.60:0/1159618527 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9134069400 con 0x7f9134143380 2023-11-18T20:44:32.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.691+0000 7f91327fc640 1 -- 172.21.15.60:0/1159618527 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9124059040 con 0x7f9134143380 2023-11-18T20:44:32.693 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2023-11-18T20:44:32.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.695+0000 7f913bef2640 1 -- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f912006ae40 msgr2=0x7f912006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:32.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.695+0000 7f913bef2640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f912006ae40 0x7f912006d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f912c036000 tx=0x7f912c008030 comp rx=0 tx=0).stop 2023-11-18T20:44:32.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.695+0000 7f913bef2640 1 -- 172.21.15.60:0/1159618527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9134143380 msgr2=0x7f913406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:32.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.695+0000 7f913bef2640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9134143380 0x7f913406d580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f912400efc0 tx=0x7f912400c490 comp rx=0 tx=0).stop 2023-11-18T20:44:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.695+0000 7f913bef2640 1 -- 172.21.15.60:0/1159618527 shutdown_connections 2023-11-18T20:44:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.695+0000 7f913bef2640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f912006ae40 0x7f912006d300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.695+0000 7f913bef2640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9134143380 0x7f913406d580 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.695+0000 7f913bef2640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134140520 0x7f91341f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.695+0000 7f913bef2640 1 --2- 172.21.15.60:0/1159618527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f913406ba20 0x7f9134154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.695+0000 7f913bef2640 1 -- 172.21.15.60:0/1159618527 >> 172.21.15.60:0/1159618527 conn(0x7f9134070c70 msgr2=0x7f913415b610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.696+0000 7f913bef2640 1 -- 172.21.15.60:0/1159618527 shutdown_connections 2023-11-18T20:44:32.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.696+0000 7f913bef2640 1 -- 172.21.15.60:0/1159618527 wait complete. 2023-11-18T20:44:32.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-18T20:44:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.839+0000 7f9364a44640 1 Processor -- start 2023-11-18T20:44:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.839+0000 7f9364a44640 1 -- start start 2023-11-18T20:44:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f9364a44640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93600ceb70 0x7f93600cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f9364a44640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93600b4370 0x7f93600b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f9364a44640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9360071770 con 0x7f9360144980 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f9364a44640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936015ced0 con 0x7f93600ceb70 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f9364a44640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93600b4ce0 con 0x7f93600b4370 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f9357fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93600ceb70 0x7f93600cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f935e59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9360144980 0x7f936015c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55374/0 (socket says 172.21.15.60:55374) 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f935e59c640 1 -- 172.21.15.60:0/1662062545 learned_addr learned my addr 172.21.15.60:0/1662062545 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f935ed9d640 1 --2- 172.21.15.60:0/1662062545 >> v2:172.21.15.100:3300/0 conn(0x7f93600b4370 0x7f93600b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f9357fff640 1 -- 172.21.15.60:0/1662062545 >> v2:172.21.15.100:3300/0 conn(0x7f93600b4370 msgr2=0x7f93600b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f9357fff640 1 --2- 172.21.15.60:0/1662062545 >> v2:172.21.15.100:3300/0 conn(0x7f93600b4370 0x7f93600b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f9357fff640 1 -- 172.21.15.60:0/1662062545 >> v1:172.21.15.60:6789/0 conn(0x7f9360144980 legacy=0x7f936015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.840+0000 7f9357fff640 1 -- 172.21.15.60:0/1662062545 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600ccdc0 con 0x7f93600ceb70 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.841+0000 7f9357fff640 1 --2- 172.21.15.60:0/1662062545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93600ceb70 0x7f93600cc450 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f93600716e0 tx=0x7f934802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432565b68bdd15ba server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.841+0000 7f935dd9b640 1 -- 172.21.15.60:0/1662062545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348031830 con 0x7f93600ceb70 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.841+0000 7f935dd9b640 1 -- 172.21.15.60:0/1662062545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348031990 con 0x7f93600ceb70 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.841+0000 7f935dd9b640 1 -- 172.21.15.60:0/1662062545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348031d20 con 0x7f93600ceb70 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.842+0000 7f9364a44640 1 -- 172.21.15.60:0/1662062545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93600ceb70 msgr2=0x7f93600cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.842+0000 7f9364a44640 1 --2- 172.21.15.60:0/1662062545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93600ceb70 0x7f93600cc450 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f93600716e0 tx=0x7f934802b8d0 comp rx=0 tx=0).stop 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.842+0000 7f9364a44640 1 -- 172.21.15.60:0/1662062545 shutdown_connections 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.842+0000 7f9364a44640 1 --2- 172.21.15.60:0/1662062545 >> v2:172.21.15.100:3300/0 conn(0x7f93600b4370 0x7f93600b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.842+0000 7f9364a44640 1 --2- 172.21.15.60:0/1662062545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93600ceb70 0x7f93600cc450 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.842+0000 7f9364a44640 1 -- 172.21.15.60:0/1662062545 >> 172.21.15.60:0/1662062545 conn(0x7f936006a4d0 msgr2=0x7f936006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.842+0000 7f9364a44640 1 -- 172.21.15.60:0/1662062545 shutdown_connections 2023-11-18T20:44:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.842+0000 7f9364a44640 1 -- 172.21.15.60:0/1662062545 wait complete. 2023-11-18T20:44:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.843+0000 7f9364a44640 1 Processor -- start 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.843+0000 7f9364a44640 1 -- start start 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f9364a44640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93600b4370 0x7f93601f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f9364a44640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93600ceb70 0x7f936013b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f9364a44640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9360144980 0x7f93600ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f9364a44640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600cb300 con 0x7f9360144980 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f9364a44640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936013dca0 con 0x7f93600b4370 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f9364a44640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93600c8e70 con 0x7f93600ceb70 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f935ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9360144980 0x7f93600ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f935ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9360144980 0x7f93600ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49826/0 (socket says 172.21.15.60:49826) 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f935ed9d640 1 -- 172.21.15.60:0/2817093248 learned_addr learned my addr 172.21.15.60:0/2817093248 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f9357fff640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93600ceb70 0x7f936013b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f935e59c640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93600b4370 0x7f93601f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f935ed9d640 1 -- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93600ceb70 msgr2=0x7f936013b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f935ed9d640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93600ceb70 0x7f936013b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f935ed9d640 1 -- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93600b4370 msgr2=0x7f93601f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f935ed9d640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93600b4370 0x7f93601f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.844+0000 7f935ed9d640 1 -- 172.21.15.60:0/2817093248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9348009400 con 0x7f9360144980 2023-11-18T20:44:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.845+0000 7f935ed9d640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9360144980 0x7f93600ca9f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9358007e40 tx=0x7f935800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.845+0000 7f9356ffd640 1 -- 172.21.15.60:0/2817093248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358014620 con 0x7f9360144980 2023-11-18T20:44:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.845+0000 7f9356ffd640 1 -- 172.21.15.60:0/2817093248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935800ee90 con 0x7f9360144980 2023-11-18T20:44:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.845+0000 7f9364a44640 1 -- 172.21.15.60:0/2817093248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93600c9150 con 0x7f9360144980 2023-11-18T20:44:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.845+0000 7f9356ffd640 1 -- 172.21.15.60:0/2817093248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580149e0 con 0x7f9360144980 2023-11-18T20:44:32.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.845+0000 7f9364a44640 1 -- 172.21.15.60:0/2817093248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93600caf30 con 0x7f9360144980 2023-11-18T20:44:32.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.846+0000 7f9364a44640 1 -- 172.21.15.60:0/2817093248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936013dde0 con 0x7f9360144980 2023-11-18T20:44:32.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.846+0000 7f9356ffd640 1 -- 172.21.15.60:0/2817093248 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9358014b40 con 0x7f9360144980 2023-11-18T20:44:32.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.847+0000 7f9356ffd640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f934006ae40 0x7f934006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:32.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.847+0000 7f9356ffd640 1 -- 172.21.15.60:0/2817093248 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f9358089fd0 con 0x7f9360144980 2023-11-18T20:44:32.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.847+0000 7f935e59c640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f934006ae40 0x7f934006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:32.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.847+0000 7f935e59c640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f934006ae40 0x7f934006d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9350005880 tx=0x7f9350009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:32.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:32.851+0000 7f9356ffd640 1 -- 172.21.15.60:0/2817093248 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f936013dde0 con 0x7f9360144980 2023-11-18T20:44:33.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.068+0000 7f9364a44640 1 -- 172.21.15.60:0/2817093248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f936013dde0 con 0x7f9360144980 2023-11-18T20:44:33.694 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.692+0000 7f9356ffd640 1 -- 172.21.15.60:0/2817093248 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7f936013dde0 con 0x7f9360144980 2023-11-18T20:44:33.694 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 7 and data pool 8 2023-11-18T20:44:33.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 -- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f934006ae40 msgr2=0x7f934006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:33.698 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f934006ae40 0x7f934006d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9350005880 tx=0x7f9350009040 comp rx=0 tx=0).stop 2023-11-18T20:44:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 -- 172.21.15.60:0/2817093248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9360144980 msgr2=0x7f93600ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9360144980 0x7f93600ca9f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9358007e40 tx=0x7f935800e4b0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 -- 172.21.15.60:0/2817093248 shutdown_connections 2023-11-18T20:44:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f934006ae40 0x7f934006d300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9360144980 0x7f93600ca9f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93600ceb70 0x7f936013b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 --2- 172.21.15.60:0/2817093248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93600b4370 0x7f93601f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 -- 172.21.15.60:0/2817093248 >> 172.21.15.60:0/2817093248 conn(0x7f936006a4d0 msgr2=0x7f936015d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 -- 172.21.15.60:0/2817093248 shutdown_connections 2023-11-18T20:44:33.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.697+0000 7f9364a44640 1 -- 172.21.15.60:0/2817093248 wait complete. 2023-11-18T20:44:33.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.828+0000 7f1c65883640 1 Processor -- start 2023-11-18T20:44:33.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.828+0000 7f1c65883640 1 -- start start 2023-11-18T20:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c65883640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1c6006ea50 0x7f1c6006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c65883640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c600d0ed0 0x7f1c600d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c65883640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c60161d20 con 0x7f1c60160fb0 2023-11-18T20:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c65883640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6006f370 con 0x7f1c600d0ed0 2023-11-18T20:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c65883640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6015e760 con 0x7f1c6006ea50 2023-11-18T20:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c5f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c600d0ed0 0x7f1c600d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c5f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1c600d0ed0 0x7f1c600d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49830/0 (socket says 172.21.15.60:49830) 2023-11-18T20:44:33.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c5f7fe640 1 -- 172.21.15.60:0/880202154 learned_addr learned my addr 172.21.15.60:0/880202154 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:33.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c5effd640 1 --2- 172.21.15.60:0/880202154 >> v2:172.21.15.100:3300/0 conn(0x7f1c6006ea50 0x7f1c6006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:33.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c5f7fe640 1 -- 172.21.15.60:0/880202154 >> v2:172.21.15.100:3300/0 conn(0x7f1c6006ea50 msgr2=0x7f1c6006ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:33.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c5f7fe640 1 --2- 172.21.15.60:0/880202154 >> v2:172.21.15.100:3300/0 conn(0x7f1c6006ea50 0x7f1c6006ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c5f7fe640 1 -- 172.21.15.60:0/880202154 >> v1:172.21.15.100:6789/0 conn(0x7f1c60160fb0 legacy=0x7f1c60161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:33.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.829+0000 7f1c5f7fe640 1 -- 172.21.15.60:0/880202154 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6015d9a0 con 0x7f1c600d0ed0 2023-11-18T20:44:33.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.830+0000 7f1c5f7fe640 1 --2- 172.21.15.60:0/880202154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c600d0ed0 0x7f1c600d1320 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1c6006f680 tx=0x7f1c5002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484a89fc0a292798 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:33.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.830+0000 7f1c5dffb640 1 -- 172.21.15.60:0/880202154 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5002ca80 con 0x7f1c600d0ed0 2023-11-18T20:44:33.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.830+0000 7f1c5dffb640 1 -- 172.21.15.60:0/880202154 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5002cbe0 con 0x7f1c600d0ed0 2023-11-18T20:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.830+0000 7f1c5dffb640 1 -- 172.21.15.60:0/880202154 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c500343f0 con 0x7f1c600d0ed0 2023-11-18T20:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.831+0000 7f1c65883640 1 -- 172.21.15.60:0/880202154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c600d0ed0 msgr2=0x7f1c600d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.831+0000 7f1c65883640 1 --2- 172.21.15.60:0/880202154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c600d0ed0 0x7f1c600d1320 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1c6006f680 tx=0x7f1c5002bee0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.831+0000 7f1c65883640 1 -- 172.21.15.60:0/880202154 shutdown_connections 2023-11-18T20:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.831+0000 7f1c65883640 1 --2- 172.21.15.60:0/880202154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c600d0ed0 0x7f1c600d1320 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.831+0000 7f1c65883640 1 --2- 172.21.15.60:0/880202154 >> v2:172.21.15.100:3300/0 conn(0x7f1c6006ea50 0x7f1c6006ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.831+0000 7f1c65883640 1 -- 172.21.15.60:0/880202154 >> 172.21.15.60:0/880202154 conn(0x7f1c6006a6b0 msgr2=0x7f1c6006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.831+0000 7f1c65883640 1 -- 172.21.15.60:0/880202154 shutdown_connections 2023-11-18T20:44:33.834 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.831+0000 7f1c65883640 1 -- 172.21.15.60:0/880202154 wait complete. 2023-11-18T20:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.832+0000 7f1c65883640 1 Processor -- start 2023-11-18T20:44:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.832+0000 7f1c65883640 1 -- start start 2023-11-18T20:44:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c65883640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6006ea50 0x7f1c60156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:33.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c65883640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c600d0ed0 0x7f1c6013e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c65883640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c60160fb0 0x7f1c600b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c65883640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6013f760 con 0x7f1c6006ea50 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c65883640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c601f44c0 con 0x7f1c60160fb0 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c65883640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c601f41e0 con 0x7f1c600d0ed0 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6006ea50 0x7f1c60156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6006ea50 0x7f1c60156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49838/0 (socket says 172.21.15.60:49838) 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5effd640 1 -- 172.21.15.60:0/1366444545 learned_addr learned my addr 172.21.15.60:0/1366444545 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5e7fc640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c600d0ed0 0x7f1c6013e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5f7fe640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c60160fb0 0x7f1c600b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5effd640 1 -- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c600d0ed0 msgr2=0x7f1c6013e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5effd640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c600d0ed0 0x7f1c6013e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5effd640 1 -- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c60160fb0 msgr2=0x7f1c600b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5effd640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c60160fb0 0x7f1c600b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5effd640 1 -- 172.21.15.60:0/1366444545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c50009b30 con 0x7f1c6006ea50 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5f7fe640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c60160fb0 0x7f1c600b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.833+0000 7f1c5e7fc640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c600d0ed0 0x7f1c6013e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.834+0000 7f1c5effd640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6006ea50 0x7f1c60156aa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800d5f0 tx=0x7f1c5800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.834+0000 7f1c57fff640 1 -- 172.21.15.60:0/1366444545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c580125f0 con 0x7f1c6006ea50 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.834+0000 7f1c57fff640 1 -- 172.21.15.60:0/1366444545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c58012750 con 0x7f1c6006ea50 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.834+0000 7f1c65883640 1 -- 172.21.15.60:0/1366444545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6013da70 con 0x7f1c6006ea50 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.834+0000 7f1c57fff640 1 -- 172.21.15.60:0/1366444545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58012b00 con 0x7f1c6006ea50 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.834+0000 7f1c65883640 1 -- 172.21.15.60:0/1366444545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6013e030 con 0x7f1c6006ea50 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.835+0000 7f1c65883640 1 -- 172.21.15.60:0/1366444545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6004ed60 con 0x7f1c6006ea50 2023-11-18T20:44:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.835+0000 7f1c57fff640 1 -- 172.21.15.60:0/1366444545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c58012c60 con 0x7f1c6006ea50 2023-11-18T20:44:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.836+0000 7f1c57fff640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1c4006ae40 0x7f1c4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.836+0000 7f1c57fff640 1 -- 172.21.15.60:0/1366444545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c58089ae0 con 0x7f1c6006ea50 2023-11-18T20:44:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.836+0000 7f1c5e7fc640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1c4006ae40 0x7f1c4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.837+0000 7f1c5e7fc640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1c4006ae40 0x7f1c4006d300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1c601f7bf0 tx=0x7f1c48009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:33.840+0000 7f1c57fff640 1 -- 172.21.15.60:0/1366444545 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c58054b90 con 0x7f1c6006ea50 2023-11-18T20:44:34.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.019+0000 7f1c65883640 1 -- 172.21.15.60:0/1366444545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c60070ac0 con 0x7f1c6006ea50 2023-11-18T20:44:34.022 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.020+0000 7f1c57fff640 1 -- 172.21.15.60:0/1366444545 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16570 (secure 0 0 0) 0x7f1c580581b0 con 0x7f1c6006ea50 2023-11-18T20:44:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:34.023 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:33.069788+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:34.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 -- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1c4006ae40 msgr2=0x7f1c4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1c4006ae40 0x7f1c4006d300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1c601f7bf0 tx=0x7f1c48009040 comp rx=0 tx=0).stop 2023-11-18T20:44:34.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 -- 172.21.15.60:0/1366444545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6006ea50 msgr2=0x7f1c60156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6006ea50 0x7f1c60156aa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800d5f0 tx=0x7f1c5800dab0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 -- 172.21.15.60:0/1366444545 shutdown_connections 2023-11-18T20:44:34.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1c4006ae40 0x7f1c4006d300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c60160fb0 0x7f1c600b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c600d0ed0 0x7f1c6013e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 --2- 172.21.15.60:0/1366444545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c6006ea50 0x7f1c60156aa0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 -- 172.21.15.60:0/1366444545 >> 172.21.15.60:0/1366444545 conn(0x7f1c6006a6b0 msgr2=0x7f1c6006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:34.026 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 -- 172.21.15.60:0/1366444545 shutdown_connections 2023-11-18T20:44:34.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.023+0000 7f1c65883640 1 -- 172.21.15.60:0/1366444545 wait complete. 2023-11-18T20:44:34.035 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:34.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.142+0000 7fc7eba64640 1 Processor -- start 2023-11-18T20:44:34.144 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.143+0000 7fc7eba64640 1 -- start start 2023-11-18T20:44:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.143+0000 7fc7eba64640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc7e40d2910 0x7fc7e40d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.143+0000 7fc7eba64640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e40b2b10 con 0x7fc7e40d3730 2023-11-18T20:44:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.143+0000 7fc7eba64640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e40d1590 con 0x7fc7e415d330 2023-11-18T20:44:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.143+0000 7fc7eba64640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e40d3bc0 con 0x7fc7e40d2910 2023-11-18T20:44:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.144+0000 7fc7ea001640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc7e40d3730 0x7fc7e40b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55378/0 (socket says 172.21.15.60:55378) 2023-11-18T20:44:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.144+0000 7fc7ea001640 1 -- 172.21.15.60:0/1346475829 learned_addr learned my addr 172.21.15.60:0/1346475829 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.144+0000 7fc7e8fff640 1 --2- 172.21.15.60:0/1346475829 >> v2:172.21.15.100:3300/0 conn(0x7fc7e40d2910 0x7fc7e40d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.144+0000 7fc7dbfff640 1 -- 172.21.15.60:0/1346475829 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291951931 0 0) 0x7fc7e40b2b10 con 0x7fc7e40d3730 2023-11-18T20:44:34.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.144+0000 7fc7dbfff640 1 -- 172.21.15.60:0/1346475829 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7d0003520 con 0x7fc7e40d3730 2023-11-18T20:44:34.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.144+0000 7fc7dbfff640 1 -- 172.21.15.60:0/1346475829 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597203840 0 0) 0x7fc7e40d1590 con 0x7fc7e415d330 2023-11-18T20:44:34.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.144+0000 7fc7dbfff640 1 -- 172.21.15.60:0/1346475829 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e40b2b10 con 0x7fc7e415d330 2023-11-18T20:44:34.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.144+0000 7fc7e8fff640 1 -- 172.21.15.60:0/1346475829 >> v1:172.21.15.100:6789/0 conn(0x7fc7e415d330 legacy=0x7fc7e40b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.144+0000 7fc7e8fff640 1 -- 172.21.15.60:0/1346475829 >> v1:172.21.15.60:6789/0 conn(0x7fc7e40d3730 legacy=0x7fc7e40b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.144+0000 7fc7e8fff640 1 -- 172.21.15.60:0/1346475829 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e40d1870 con 0x7fc7e40d2910 2023-11-18T20:44:34.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.145+0000 7fc7e8fff640 1 --2- 172.21.15.60:0/1346475829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e40d2910 0x7fc7e40d31f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40097a0 tx=0x7fc7d402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97816ed38722aa96 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:34.148 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.145+0000 7fc7dbfff640 1 -- 172.21.15.60:0/1346475829 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4031830 con 0x7fc7e40d2910 2023-11-18T20:44:34.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.145+0000 7fc7dbfff640 1 -- 172.21.15.60:0/1346475829 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d4031990 con 0x7fc7e40d2910 2023-11-18T20:44:34.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.145+0000 7fc7dbfff640 1 -- 172.21.15.60:0/1346475829 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4031d00 con 0x7fc7e40d2910 2023-11-18T20:44:34.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.146+0000 7fc7eba64640 1 -- 172.21.15.60:0/1346475829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e40d2910 msgr2=0x7fc7e40d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.146+0000 7fc7eba64640 1 --2- 172.21.15.60:0/1346475829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e40d2910 0x7fc7e40d31f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40097a0 tx=0x7fc7d402b8d0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.146+0000 7fc7eba64640 1 -- 172.21.15.60:0/1346475829 shutdown_connections 2023-11-18T20:44:34.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.146+0000 7fc7eba64640 1 --2- 172.21.15.60:0/1346475829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e40d2910 0x7fc7e40d31f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.146+0000 7fc7eba64640 1 -- 172.21.15.60:0/1346475829 >> 172.21.15.60:0/1346475829 conn(0x7fc7e4072020 msgr2=0x7fc7e4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:34.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.146+0000 7fc7eba64640 1 -- 172.21.15.60:0/1346475829 shutdown_connections 2023-11-18T20:44:34.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.146+0000 7fc7eba64640 1 -- 172.21.15.60:0/1346475829 wait complete. 2023-11-18T20:44:34.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.147+0000 7fc7eba64640 1 Processor -- start 2023-11-18T20:44:34.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.147+0000 7fc7eba64640 1 -- start start 2023-11-18T20:44:34.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7eba64640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e40d2910 0x7fc7e415fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7eba64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e40d3730 0x7fc7e4152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7eba64640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7e415d330 0x7fc7e4157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7eba64640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e4156c70 con 0x7fc7e40d3730 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7eba64640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e4160870 con 0x7fc7e40d2910 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7eba64640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e4160590 con 0x7fc7e415d330 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7e8fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e40d3730 0x7fc7e4152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7e8fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e40d3730 0x7fc7e4152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49846/0 (socket says 172.21.15.60:49846) 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7e8fff640 1 -- 172.21.15.60:0/189319625 learned_addr learned my addr 172.21.15.60:0/189319625 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7e9800640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e40d2910 0x7fc7e415fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.148+0000 7fc7ea001640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7e415d330 0x7fc7e4157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7e8fff640 1 -- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7e415d330 msgr2=0x7fc7e4157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7e8fff640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7e415d330 0x7fc7e4157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7e8fff640 1 -- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e40d2910 msgr2=0x7fc7e415fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7e8fff640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e40d2910 0x7fc7e415fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7e8fff640 1 -- 172.21.15.60:0/189319625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d4009400 con 0x7fc7e40d3730 2023-11-18T20:44:34.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7e8fff640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e40d3730 0x7fc7e4152ab0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000c00 tx=0x7fc7d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:34.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7d9ffb640 1 -- 172.21.15.60:0/189319625 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d403a630 con 0x7fc7e40d3730 2023-11-18T20:44:34.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7d9ffb640 1 -- 172.21.15.60:0/189319625 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d403a7b0 con 0x7fc7e40d3730 2023-11-18T20:44:34.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7eba64640 1 -- 172.21.15.60:0/189319625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e41565b0 con 0x7fc7e40d3730 2023-11-18T20:44:34.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7d9ffb640 1 -- 172.21.15.60:0/189319625 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d403ab60 con 0x7fc7e40d3730 2023-11-18T20:44:34.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.149+0000 7fc7eba64640 1 -- 172.21.15.60:0/189319625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e4156260 con 0x7fc7e40d3730 2023-11-18T20:44:34.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.150+0000 7fc7eba64640 1 -- 172.21.15.60:0/189319625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e4156db0 con 0x7fc7e40d3730 2023-11-18T20:44:34.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.150+0000 7fc7d9ffb640 1 -- 172.21.15.60:0/189319625 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc7d403acc0 con 0x7fc7e40d3730 2023-11-18T20:44:34.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.151+0000 7fc7d9ffb640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7c806ae40 0x7fc7c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.151+0000 7fc7d9ffb640 1 -- 172.21.15.60:0/189319625 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7d40ab120 con 0x7fc7e40d3730 2023-11-18T20:44:34.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.151+0000 7fc7e9800640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7c806ae40 0x7fc7c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.152+0000 7fc7e9800640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7c806ae40 0x7fc7c806d300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0001e00 tx=0x7fc7e0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:34.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.155+0000 7fc7d9ffb640 1 -- 172.21.15.60:0/189319625 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc7e4156db0 con 0x7fc7e40d3730 2023-11-18T20:44:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.331+0000 7fc7eba64640 1 -- 172.21.15.60:0/189319625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7e4156db0 con 0x7fc7e40d3730 2023-11-18T20:44:34.334 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.332+0000 7fc7d9ffb640 1 -- 172.21.15.60:0/189319625 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fc7e4156db0 con 0x7fc7e40d3730 2023-11-18T20:44:34.334 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:34.334 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:34.103787+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:34.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 -- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7c806ae40 msgr2=0x7fc7c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7c806ae40 0x7fc7c806d300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0001e00 tx=0x7fc7e0009040 comp rx=0 tx=0).stop 2023-11-18T20:44:34.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 -- 172.21.15.60:0/189319625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e40d3730 msgr2=0x7fc7e4152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e40d3730 0x7fc7e4152ab0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000c00 tx=0x7fc7d40046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 -- 172.21.15.60:0/189319625 shutdown_connections 2023-11-18T20:44:34.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7c806ae40 0x7fc7c806d300 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7e415d330 0x7fc7e4157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7e40d3730 0x7fc7e4152ab0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 --2- 172.21.15.60:0/189319625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7e40d2910 0x7fc7e415fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 -- 172.21.15.60:0/189319625 >> 172.21.15.60:0/189319625 conn(0x7fc7e4072020 msgr2=0x7fc7e40cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:34.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.334+0000 7fc7eba64640 1 -- 172.21.15.60:0/189319625 shutdown_connections 2023-11-18T20:44:34.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.335+0000 7fc7eba64640 1 -- 172.21.15.60:0/189319625 wait complete. 2023-11-18T20:44:34.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-18T20:44:34.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.456+0000 7fd808573640 1 Processor -- start 2023-11-18T20:44:34.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.456+0000 7fd808573640 1 -- start start 2023-11-18T20:44:34.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.457+0000 7fd808573640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd800144980 0x7fd80015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.457+0000 7fd808573640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8000ceb70 0x7fd8000cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.457+0000 7fd808573640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd800071770 con 0x7fd8000b4370 2023-11-18T20:44:34.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.457+0000 7fd808573640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8000ccdc0 con 0x7fd8000ceb70 2023-11-18T20:44:34.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.457+0000 7fd808573640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8000cd0a0 con 0x7fd800144980 2023-11-18T20:44:34.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.457+0000 7fd805b0e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8000ceb70 0x7fd8000cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.457+0000 7fd805b0e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8000ceb70 0x7fd8000cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49848/0 (socket says 172.21.15.60:49848) 2023-11-18T20:44:34.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.457+0000 7fd805b0e640 1 -- 172.21.15.60:0/3439079748 learned_addr learned my addr 172.21.15.60:0/3439079748 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:34.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.457+0000 7fd80630f640 1 --2- 172.21.15.60:0/3439079748 >> v2:172.21.15.100:3300/0 conn(0x7fd800144980 0x7fd80015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.458+0000 7fd805b0e640 1 -- 172.21.15.60:0/3439079748 >> v2:172.21.15.100:3300/0 conn(0x7fd800144980 msgr2=0x7fd80015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.458+0000 7fd805b0e640 1 --2- 172.21.15.60:0/3439079748 >> v2:172.21.15.100:3300/0 conn(0x7fd800144980 0x7fd80015c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.458+0000 7fd805b0e640 1 -- 172.21.15.60:0/3439079748 >> v1:172.21.15.100:6789/0 conn(0x7fd8000b4370 legacy=0x7fd800071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.458+0000 7fd805b0e640 1 -- 172.21.15.60:0/3439079748 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8000cd380 con 0x7fd8000ceb70 2023-11-18T20:44:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.458+0000 7fd80630f640 1 --2- 172.21.15.60:0/3439079748 >> v2:172.21.15.100:3300/0 conn(0x7fd800144980 0x7fd80015c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.458+0000 7fd805b0e640 1 --2- 172.21.15.60:0/3439079748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000ceb70 0x7fd8000cc450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd8000b4ec0 tx=0x7fd7f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5742ec95f86a8577 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.458+0000 7fd80530d640 1 -- 172.21.15.60:0/3439079748 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4031870 con 0x7fd8000ceb70 2023-11-18T20:44:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.458+0000 7fd80530d640 1 -- 172.21.15.60:0/3439079748 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f40319d0 con 0x7fd8000ceb70 2023-11-18T20:44:34.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.459+0000 7fd80530d640 1 -- 172.21.15.60:0/3439079748 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4031d60 con 0x7fd8000ceb70 2023-11-18T20:44:34.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.459+0000 7fd808573640 1 -- 172.21.15.60:0/3439079748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000ceb70 msgr2=0x7fd8000cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.459+0000 7fd808573640 1 --2- 172.21.15.60:0/3439079748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000ceb70 0x7fd8000cc450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd8000b4ec0 tx=0x7fd7f402b910 comp rx=0 tx=0).stop 2023-11-18T20:44:34.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.459+0000 7fd808573640 1 -- 172.21.15.60:0/3439079748 shutdown_connections 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.459+0000 7fd808573640 1 --2- 172.21.15.60:0/3439079748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000ceb70 0x7fd8000cc450 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.459+0000 7fd808573640 1 --2- 172.21.15.60:0/3439079748 >> v2:172.21.15.100:3300/0 conn(0x7fd800144980 0x7fd80015c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.459+0000 7fd808573640 1 -- 172.21.15.60:0/3439079748 >> 172.21.15.60:0/3439079748 conn(0x7fd80006a4d0 msgr2=0x7fd80006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.459+0000 7fd808573640 1 -- 172.21.15.60:0/3439079748 shutdown_connections 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.459+0000 7fd808573640 1 -- 172.21.15.60:0/3439079748 wait complete. 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.460+0000 7fd808573640 1 Processor -- start 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd808573640 1 -- start start 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd808573640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000b4370 0x7fd800141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd808573640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8000ceb70 0x7fd80013e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd808573640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800144980 0x7fd8001320f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd808573640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80013ec40 con 0x7fd8000b4370 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd808573640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80013e9a0 con 0x7fd800144980 2023-11-18T20:44:34.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd808573640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8000afe20 con 0x7fd8000ceb70 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd80630f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000b4370 0x7fd800141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd80630f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000b4370 0x7fd800141440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49850/0 (socket says 172.21.15.60:49850) 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd80630f640 1 -- 172.21.15.60:0/4240614090 learned_addr learned my addr 172.21.15.60:0/4240614090 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd805b0e640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8000ceb70 0x7fd80013e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.461+0000 7fd806b10640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800144980 0x7fd8001320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd80630f640 1 -- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8000ceb70 msgr2=0x7fd80013e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd80630f640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8000ceb70 0x7fd80013e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd80630f640 1 -- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800144980 msgr2=0x7fd8001320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd80630f640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800144980 0x7fd8001320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd80630f640 1 -- 172.21.15.60:0/4240614090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f40094d0 con 0x7fd8000b4370 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd80630f640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000b4370 0x7fd800141440 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c7f0 tx=0x7fd7f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd7feffd640 1 -- 172.21.15.60:0/4240614090 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8007820 con 0x7fd8000b4370 2023-11-18T20:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd7feffd640 1 -- 172.21.15.60:0/4240614090 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f8007980 con 0x7fd8000b4370 2023-11-18T20:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd808573640 1 -- 172.21.15.60:0/4240614090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8000b0100 con 0x7fd8000b4370 2023-11-18T20:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd7feffd640 1 -- 172.21.15.60:0/4240614090 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8007cf0 con 0x7fd8000b4370 2023-11-18T20:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.462+0000 7fd808573640 1 -- 172.21.15.60:0/4240614090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8000b06c0 con 0x7fd8000b4370 2023-11-18T20:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.463+0000 7fd808573640 1 -- 172.21.15.60:0/4240614090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd800069400 con 0x7fd8000b4370 2023-11-18T20:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.464+0000 7fd7feffd640 1 -- 172.21.15.60:0/4240614090 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd7f80113f0 con 0x7fd8000b4370 2023-11-18T20:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.464+0000 7fd7feffd640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd7ec06ae40 0x7fd7ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.464+0000 7fd7feffd640 1 -- 172.21.15.60:0/4240614090 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7f8089670 con 0x7fd8000b4370 2023-11-18T20:44:34.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.464+0000 7fd805b0e640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd7ec06ae40 0x7fd7ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.469 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.465+0000 7fd805b0e640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd7ec06ae40 0x7fd7ec06d300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4036000 tx=0x7fd7f403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:34.470 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.468+0000 7fd7feffd640 1 -- 172.21.15.60:0/4240614090 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7f80547d0 con 0x7fd8000b4370 2023-11-18T20:44:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.684+0000 7fd808573640 1 -- 172.21.15.60:0/4240614090 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd8000b1b50 con 0x7fd8000b4370 2023-11-18T20:44:34.699 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.698+0000 7fd7feffd640 1 -- 172.21.15.60:0/4240614090 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v20) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd7f8057df0 con 0x7fd8000b4370 2023-11-18T20:44:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 -- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd7ec06ae40 msgr2=0x7fd7ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd7ec06ae40 0x7fd7ec06d300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4036000 tx=0x7fd7f403c040 comp rx=0 tx=0).stop 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 -- 172.21.15.60:0/4240614090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000b4370 msgr2=0x7fd800141440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000b4370 0x7fd800141440 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c7f0 tx=0x7fd7f800ccb0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 -- 172.21.15.60:0/4240614090 shutdown_connections 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd7ec06ae40 0x7fd7ec06d300 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd800144980 0x7fd8001320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8000ceb70 0x7fd80013e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 --2- 172.21.15.60:0/4240614090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8000b4370 0x7fd800141440 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 -- 172.21.15.60:0/4240614090 >> 172.21.15.60:0/4240614090 conn(0x7fd80006a4d0 msgr2=0x7fd8001590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 -- 172.21.15.60:0/4240614090 shutdown_connections 2023-11-18T20:44:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.700+0000 7fd808573640 1 -- 172.21.15.60:0/4240614090 wait complete. 2023-11-18T20:44:34.712 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-18T20:44:34.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-18T20:44:34.821 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.820+0000 7fc337461640 1 Processor -- start 2023-11-18T20:44:34.822 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.821+0000 7fc337461640 1 -- start start 2023-11-18T20:44:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.821+0000 7fc337461640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc33015af90 0x7fc3300b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.821+0000 7fc337461640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc33015b370 0x7fc3300b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.821+0000 7fc337461640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33015f6b0 con 0x7fc330071800 2023-11-18T20:44:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.821+0000 7fc337461640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3300cf140 con 0x7fc33015b370 2023-11-18T20:44:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.821+0000 7fc337461640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33015fbe0 con 0x7fc33015af90 2023-11-18T20:44:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.822+0000 7fc3349fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc33015b370 0x7fc3300b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.822+0000 7fc3359fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc330071800 0x7fc33006d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55382/0 (socket says 172.21.15.60:55382) 2023-11-18T20:44:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.822+0000 7fc3349fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc33015b370 0x7fc3300b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49852/0 (socket says 172.21.15.60:49852) 2023-11-18T20:44:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.822+0000 7fc3359fe640 1 -- 172.21.15.60:0/616917757 learned_addr learned my addr 172.21.15.60:0/616917757 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.822+0000 7fc3351fd640 1 --2- 172.21.15.60:0/616917757 >> v2:172.21.15.100:3300/0 conn(0x7fc33015af90 0x7fc3300b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.822+0000 7fc3349fc640 1 -- 172.21.15.60:0/616917757 >> v2:172.21.15.100:3300/0 conn(0x7fc33015af90 msgr2=0x7fc3300b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.822+0000 7fc3349fc640 1 --2- 172.21.15.60:0/616917757 >> v2:172.21.15.100:3300/0 conn(0x7fc33015af90 0x7fc3300b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.822+0000 7fc3349fc640 1 -- 172.21.15.60:0/616917757 >> v1:172.21.15.60:6789/0 conn(0x7fc330071800 legacy=0x7fc33006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.822+0000 7fc3349fc640 1 -- 172.21.15.60:0/616917757 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33015b850 con 0x7fc33015b370 2023-11-18T20:44:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.822+0000 7fc32ffff640 1 -- 172.21.15.60:0/616917757 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762398961 0 0) 0x7fc33015f6b0 con 0x7fc330071800 2023-11-18T20:44:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.823+0000 7fc3349fc640 1 --2- 172.21.15.60:0/616917757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 0x7fc3300b4c10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc33006bc50 tx=0x7fc32402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b3ae95104709e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.823+0000 7fc32ffff640 1 -- 172.21.15.60:0/616917757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc324031830 con 0x7fc33015b370 2023-11-18T20:44:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.823+0000 7fc32ffff640 1 -- 172.21.15.60:0/616917757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc324031990 con 0x7fc33015b370 2023-11-18T20:44:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.823+0000 7fc32ffff640 1 -- 172.21.15.60:0/616917757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc324031d20 con 0x7fc33015b370 2023-11-18T20:44:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.823+0000 7fc337461640 1 -- 172.21.15.60:0/616917757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 msgr2=0x7fc3300b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.823+0000 7fc337461640 1 --2- 172.21.15.60:0/616917757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 0x7fc3300b4c10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc33006bc50 tx=0x7fc32402b8d0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.823+0000 7fc337461640 1 -- 172.21.15.60:0/616917757 shutdown_connections 2023-11-18T20:44:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.823+0000 7fc337461640 1 --2- 172.21.15.60:0/616917757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 0x7fc3300b4c10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.823+0000 7fc337461640 1 --2- 172.21.15.60:0/616917757 >> v2:172.21.15.100:3300/0 conn(0x7fc33015af90 0x7fc3300b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.823+0000 7fc337461640 1 -- 172.21.15.60:0/616917757 >> 172.21.15.60:0/616917757 conn(0x7fc330070900 msgr2=0x7fc330070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.824+0000 7fc337461640 1 -- 172.21.15.60:0/616917757 shutdown_connections 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.824+0000 7fc337461640 1 -- 172.21.15.60:0/616917757 wait complete. 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.825+0000 7fc337461640 1 Processor -- start 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.825+0000 7fc337461640 1 -- start start 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.825+0000 7fc337461640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc330071800 0x7fc330143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.825+0000 7fc337461640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc33015af90 0x7fc330156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.825+0000 7fc337461640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 0x7fc330156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.825+0000 7fc337461640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300b4490 con 0x7fc33015b370 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.825+0000 7fc337461640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300b33f0 con 0x7fc33015af90 2023-11-18T20:44:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.825+0000 7fc337461640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300b36d0 con 0x7fc330071800 2023-11-18T20:44:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.825+0000 7fc3359fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 0x7fc330156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc3359fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 0x7fc330156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49864/0 (socket says 172.21.15.60:49864) 2023-11-18T20:44:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc3359fe640 1 -- 172.21.15.60:0/3796312155 learned_addr learned my addr 172.21.15.60:0/3796312155 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc3349fc640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc33015af90 0x7fc330156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc3359fe640 1 -- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc330071800 msgr2=0x7fc330143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc3359fe640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc330071800 0x7fc330143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc3359fe640 1 -- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc33015af90 msgr2=0x7fc330156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc3359fe640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc33015af90 0x7fc330156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc3359fe640 1 -- 172.21.15.60:0/3796312155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc324009400 con 0x7fc33015b370 2023-11-18T20:44:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc3359fe640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 0x7fc330156ff0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc32000c7e0 tx=0x7fc32000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc32dffb640 1 -- 172.21.15.60:0/3796312155 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32000a640 con 0x7fc33015b370 2023-11-18T20:44:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc32dffb640 1 -- 172.21.15.60:0/3796312155 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32000a7c0 con 0x7fc33015b370 2023-11-18T20:44:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc337461640 1 -- 172.21.15.60:0/3796312155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3300b39b0 con 0x7fc33015b370 2023-11-18T20:44:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc32dffb640 1 -- 172.21.15.60:0/3796312155 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32000ab70 con 0x7fc33015b370 2023-11-18T20:44:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.826+0000 7fc337461640 1 -- 172.21.15.60:0/3796312155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3300b3f70 con 0x7fc33015b370 2023-11-18T20:44:34.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.827+0000 7fc32dffb640 1 -- 172.21.15.60:0/3796312155 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc32000acd0 con 0x7fc33015b370 2023-11-18T20:44:34.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.827+0000 7fc337461640 1 -- 172.21.15.60:0/3796312155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3300ce190 con 0x7fc33015b370 2023-11-18T20:44:34.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.828+0000 7fc32dffb640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc31c06ae40 0x7fc31c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:34.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.828+0000 7fc32dffb640 1 -- 172.21.15.60:0/3796312155 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc320088ec0 con 0x7fc33015b370 2023-11-18T20:44:34.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.828+0000 7fc3351fd640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc31c06ae40 0x7fc31c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:34.832 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.829+0000 7fc3351fd640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc31c06ae40 0x7fc31c06d300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc3280098f0 tx=0x7fc328008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:34.833 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:34.832+0000 7fc32dffb640 1 -- 172.21.15.60:0/3796312155 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc320053f60 con 0x7fc33015b370 2023-11-18T20:44:35.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.048+0000 7fc337461640 1 -- 172.21.15.60:0/3796312155 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc330067420 con 0x7fc33015b370 2023-11-18T20:44:35.684 INFO:tasks.ceph.mds.a.smithi060.stderr:2023-11-18T20:44:35.684+0000 7f63a7854640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-18T20:44:35.710 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.709+0000 7fc32dffb640 1 -- 172.21.15.60:0/3796312155 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc320018020 con 0x7fc33015b370 2023-11-18T20:44:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.711+0000 7fc337461640 1 -- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc31c06ae40 msgr2=0x7fc31c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.711+0000 7fc337461640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc31c06ae40 0x7fc31c06d300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc3280098f0 tx=0x7fc328008040 comp rx=0 tx=0).stop 2023-11-18T20:44:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.711+0000 7fc337461640 1 -- 172.21.15.60:0/3796312155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 msgr2=0x7fc330156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.711+0000 7fc337461640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 0x7fc330156ff0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc32000c7e0 tx=0x7fc32000cca0 comp rx=0 tx=0).stop 2023-11-18T20:44:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.711+0000 7fc337461640 1 -- 172.21.15.60:0/3796312155 shutdown_connections 2023-11-18T20:44:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.711+0000 7fc337461640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc31c06ae40 0x7fc31c06d300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.711+0000 7fc337461640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc33015b370 0x7fc330156ff0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.711+0000 7fc337461640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc33015af90 0x7fc330156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.711+0000 7fc337461640 1 --2- 172.21.15.60:0/3796312155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc330071800 0x7fc330143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.711+0000 7fc337461640 1 -- 172.21.15.60:0/3796312155 >> 172.21.15.60:0/3796312155 conn(0x7fc330070900 msgr2=0x7fc33015c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.712+0000 7fc337461640 1 -- 172.21.15.60:0/3796312155 shutdown_connections 2023-11-18T20:44:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.712+0000 7fc337461640 1 -- 172.21.15.60:0/3796312155 wait complete. 2023-11-18T20:44:35.726 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-18T20:44:35.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:44:35.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.838+0000 7f5911161640 1 Processor -- start 2023-11-18T20:44:35.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.838+0000 7f5911161640 1 -- start start 2023-11-18T20:44:35.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.839+0000 7f5911161640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f590c15d1d0 0x7f590c06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:35.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.839+0000 7f5911161640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c161ce0 con 0x7f590c0ad640 2023-11-18T20:44:35.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.839+0000 7f5911161640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c0adae0 con 0x7f590c0aeb80 2023-11-18T20:44:35.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.839+0000 7f5911161640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c0addc0 con 0x7f590c15d1d0 2023-11-18T20:44:35.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.839+0000 7f590ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f590c15d1d0 0x7f590c06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:35.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.839+0000 7f5903fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f590c0ad640 0x7f590c160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55398/0 (socket says 172.21.15.60:55398) 2023-11-18T20:44:35.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.839+0000 7f5903fff640 1 -- 172.21.15.60:0/842001129 learned_addr learned my addr 172.21.15.60:0/842001129 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:35.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.839+0000 7f590ad9d640 1 -- 172.21.15.60:0/842001129 >> v1:172.21.15.100:6789/0 conn(0x7f590c0aeb80 legacy=0x7f590c1615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:35.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.839+0000 7f590ad9d640 1 -- 172.21.15.60:0/842001129 >> v1:172.21.15.60:6789/0 conn(0x7f590c0ad640 legacy=0x7f590c160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:35.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.839+0000 7f590ad9d640 1 -- 172.21.15.60:0/842001129 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c0ae0a0 con 0x7f590c15d1d0 2023-11-18T20:44:35.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.840+0000 7f590ad9d640 1 --2- 172.21.15.60:0/842001129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15d1d0 0x7f590c06b4e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0097a0 tx=0x7f58fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36ea4cafc534f84 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:35.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.840+0000 7f590a59c640 1 -- 172.21.15.60:0/842001129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc031830 con 0x7f590c15d1d0 2023-11-18T20:44:35.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.840+0000 7f590a59c640 1 -- 172.21.15.60:0/842001129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc031990 con 0x7f590c15d1d0 2023-11-18T20:44:35.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.840+0000 7f590a59c640 1 -- 172.21.15.60:0/842001129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc031d20 con 0x7f590c15d1d0 2023-11-18T20:44:35.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.841+0000 7f5911161640 1 -- 172.21.15.60:0/842001129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15d1d0 msgr2=0x7f590c06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:35.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.841+0000 7f5911161640 1 --2- 172.21.15.60:0/842001129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15d1d0 0x7f590c06b4e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0097a0 tx=0x7f58fc02b8d0 comp rx=0 tx=0).stop 2023-11-18T20:44:35.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.841+0000 7f5911161640 1 -- 172.21.15.60:0/842001129 shutdown_connections 2023-11-18T20:44:35.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.841+0000 7f5911161640 1 --2- 172.21.15.60:0/842001129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c15d1d0 0x7f590c06b4e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:35.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.841+0000 7f5911161640 1 -- 172.21.15.60:0/842001129 >> 172.21.15.60:0/842001129 conn(0x7f590c070e70 msgr2=0x7f590c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:35.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.841+0000 7f5911161640 1 -- 172.21.15.60:0/842001129 shutdown_connections 2023-11-18T20:44:35.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.841+0000 7f5911161640 1 -- 172.21.15.60:0/842001129 wait complete. 2023-11-18T20:44:35.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.842+0000 7f5911161640 1 Processor -- start 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.842+0000 7f5911161640 1 -- start start 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5911161640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f590c0ad640 0x7f590c0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5911161640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c0aeb80 0x7f590c15d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5911161640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590c15d1d0 0x7f590c1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5911161640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c15eea0 con 0x7f590c0aeb80 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5911161640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c15f2c0 con 0x7f590c15d1d0 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5911161640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c13eea0 con 0x7f590c0ad640 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5903fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c0aeb80 0x7f590c15d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5903fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c0aeb80 0x7f590c15d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49872/0 (socket says 172.21.15.60:49872) 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5903fff640 1 -- 172.21.15.60:0/69893947 learned_addr learned my addr 172.21.15.60:0/69893947 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:35.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f590b59e640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590c15d1d0 0x7f590c1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f590ad9d640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f590c0ad640 0x7f590c0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5903fff640 1 -- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f590c0ad640 msgr2=0x7f590c0cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5903fff640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f590c0ad640 0x7f590c0cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5903fff640 1 -- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590c15d1d0 msgr2=0x7f590c1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5903fff640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590c15d1d0 0x7f590c1595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5903fff640 1 -- 172.21.15.60:0/69893947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc009400 con 0x7f590c0aeb80 2023-11-18T20:44:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.843+0000 7f5903fff640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c0aeb80 0x7f590c15d910 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f58f400c4d0 tx=0x7f58f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.844+0000 7f59037fe640 1 -- 172.21.15.60:0/69893947 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4005ce0 con 0x7f590c0aeb80 2023-11-18T20:44:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.844+0000 7f59037fe640 1 -- 172.21.15.60:0/69893947 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f4005e40 con 0x7f590c0aeb80 2023-11-18T20:44:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.844+0000 7f59037fe640 1 -- 172.21.15.60:0/69893947 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4007270 con 0x7f590c0aeb80 2023-11-18T20:44:35.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.844+0000 7f5911161640 1 -- 172.21.15.60:0/69893947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c0cf6c0 con 0x7f590c0aeb80 2023-11-18T20:44:35.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.844+0000 7f5911161640 1 -- 172.21.15.60:0/69893947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c15e500 con 0x7f590c0aeb80 2023-11-18T20:44:35.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.845+0000 7f5911161640 1 -- 172.21.15.60:0/69893947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c15efe0 con 0x7f590c0aeb80 2023-11-18T20:44:35.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.845+0000 7f59037fe640 1 -- 172.21.15.60:0/69893947 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58f40073d0 con 0x7f590c0aeb80 2023-11-18T20:44:35.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.845+0000 7f59037fe640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58ec06ae40 0x7f58ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:35.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.845+0000 7f59037fe640 1 -- 172.21.15.60:0/69893947 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58f4088970 con 0x7f590c0aeb80 2023-11-18T20:44:35.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.846+0000 7f590ad9d640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58ec06ae40 0x7f58ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:35.849 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.846+0000 7f590ad9d640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58ec06ae40 0x7f58ec06d300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f58fc036000 tx=0x7f58fc03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:35.850 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:35.850+0000 7f59037fe640 1 -- 172.21.15.60:0/69893947 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f590c15efe0 con 0x7f590c0aeb80 2023-11-18T20:44:36.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.061+0000 7f5911161640 1 -- 172.21.15.60:0/69893947 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f590c15efe0 con 0x7f590c0aeb80 2023-11-18T20:44:36.062 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.061+0000 7f59037fe640 1 -- 172.21.15.60:0/69893947 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f590c15efe0 con 0x7f590c0aeb80 2023-11-18T20:44:36.063 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:36.063 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1578336464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1578336464},{"type":"v1","addr":"172.21.15.100:6836","nonce":1578336464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/88984334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":88984334},{"type":"v1","addr":"172.21.15.100:6837","nonce":88984334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3777286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3777286992},{"type":"v1","addr":"172.21.15.60:6833","nonce":3777286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:44:33.070091+0000","modified":"2023-11-18T20:44:35.050014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3098423505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3098423505},{"type":"v1","addr":"172.21.15.60:6835","nonce":3098423505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-18T20:44:36.063 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 -- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58ec06ae40 msgr2=0x7f58ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58ec06ae40 0x7f58ec06d300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f58fc036000 tx=0x7f58fc03c040 comp rx=0 tx=0).stop 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 -- 172.21.15.60:0/69893947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c0aeb80 msgr2=0x7f590c15d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c0aeb80 0x7f590c15d910 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f58f400c4d0 tx=0x7f58f400c990 comp rx=0 tx=0).stop 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 -- 172.21.15.60:0/69893947 shutdown_connections 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f58ec06ae40 0x7f58ec06d300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590c15d1d0 0x7f590c1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f590c0aeb80 0x7f590c15d910 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 --2- 172.21.15.60:0/69893947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f590c0ad640 0x7f590c0cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 -- 172.21.15.60:0/69893947 >> 172.21.15.60:0/69893947 conn(0x7f590c070e70 msgr2=0x7f590c0b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 -- 172.21.15.60:0/69893947 shutdown_connections 2023-11-18T20:44:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.063+0000 7f5911161640 1 -- 172.21.15.60:0/69893947 wait complete. 2023-11-18T20:44:36.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:36.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.180+0000 7fb1c5f5a640 1 Processor -- start 2023-11-18T20:44:36.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.181+0000 7fb1c5f5a640 1 -- start start 2023-11-18T20:44:36.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.181+0000 7fb1c5f5a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb1c015d1d0 0x7fb1c006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.181+0000 7fb1c5f5a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c0161ce0 con 0x7fb1c00ad640 2023-11-18T20:44:36.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.181+0000 7fb1c5f5a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c00adae0 con 0x7fb1c00aeb80 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.181+0000 7fb1c5f5a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c00addc0 con 0x7fb1c015d1d0 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.181+0000 7fb1beffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb1c00ad640 0x7fb1c0160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55410/0 (socket says 172.21.15.60:55410) 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.181+0000 7fb1beffd640 1 -- 172.21.15.60:0/3584176307 learned_addr learned my addr 172.21.15.60:0/3584176307 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141604315 0 0) 0x7fb1c0161ce0 con 0x7fb1c00ad640 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a8003520 con 0x7fb1c00ad640 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1bf7fe640 1 --2- 172.21.15.60:0/3584176307 >> v2:172.21.15.100:3300/0 conn(0x7fb1c015d1d0 0x7fb1c006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1518515879 0 0) 0x7fb1a8003520 con 0x7fb1c00ad640 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 >> v2:172.21.15.100:3300/0 conn(0x7fb1c015d1d0 msgr2=0x7fb1c006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 --2- 172.21.15.60:0/3584176307 >> v2:172.21.15.100:3300/0 conn(0x7fb1c015d1d0 0x7fb1c006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 >> v1:172.21.15.100:6789/0 conn(0x7fb1c00aeb80 legacy=0x7fb1c01615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00ae0a0 con 0x7fb1c00ad640 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fb1b4001670 con 0x7fb1c00ad640 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 >> v1:172.21.15.60:6789/0 conn(0x7fb1c00ad640 legacy=0x7fb1c0160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1a80047c0 0x7fb1a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a8005180 0x7fb1a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00addc0 con 0x7fb1a8003970 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a800c020 con 0x7fb1a8005180 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a800c1e0 con 0x7fb1a80047c0 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1bf7fe640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1beffd640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1a80047c0 0x7fb1a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1bffff640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a8005180 0x7fb1a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1bf7fe640 1 -- 172.21.15.60:0/3584176307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1a80047c0 msgr2=0x7fb1a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:36.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1bf7fe640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1a80047c0 0x7fb1a8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1bf7fe640 1 -- 172.21.15.60:0/3584176307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a8005180 msgr2=0x7fb1a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1bf7fe640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a8005180 0x7fb1a800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.182+0000 7fb1bf7fe640 1 -- 172.21.15.60:0/3584176307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8003520 con 0x7fb1a8003970 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.183+0000 7fb1beffd640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1a80047c0 0x7fb1a8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.183+0000 7fb1bffff640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a8005180 0x7fb1a800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.183+0000 7fb1bffff640 1 -- 172.21.15.60:0/3584176307 reap_dead start 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.183+0000 7fb1bf7fe640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1a8003d90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009ac0 tx=0x7fb1a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cc347b41e41e40 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.183+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b002c6b0 con 0x7fb1a8003970 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.183+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b002c810 con 0x7fb1a8003970 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.183+0000 7fb1be7fc640 1 -- 172.21.15.60:0/3584176307 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b002cba0 con 0x7fb1a8003970 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.184+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/3584176307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 msgr2=0x7fb1a8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.184+0000 7fb1c5f5a640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1a8003d90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009ac0 tx=0x7fb1a80057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.184+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/3584176307 shutdown_connections 2023-11-18T20:44:36.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.184+0000 7fb1c5f5a640 1 --2- 172.21.15.60:0/3584176307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1a8003d90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.184+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/3584176307 >> 172.21.15.60:0/3584176307 conn(0x7fb1c0070e70 msgr2=0x7fb1c0071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:36.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.184+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/3584176307 shutdown_connections 2023-11-18T20:44:36.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.184+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/3584176307 wait complete. 2023-11-18T20:44:36.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.185+0000 7fb1c5f5a640 1 Processor -- start 2023-11-18T20:44:36.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.185+0000 7fb1c5f5a640 1 -- start start 2023-11-18T20:44:36.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1c5f5a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1c0146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1c5f5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1c015bf30 0x7fb1c015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1c5f5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1c015f370 0x7fb1c015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1c5f5a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0158010 con 0x7fb1a8003970 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1c5f5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c015c8f0 con 0x7fb1c015bf30 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1c5f5a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00cb920 con 0x7fb1c015f370 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1bf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1c0146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1bf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1c0146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49892/0 (socket says 172.21.15.60:49892) 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1bf7fe640 1 -- 172.21.15.60:0/1976671715 learned_addr learned my addr 172.21.15.60:0/1976671715 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1beffd640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1c015bf30 0x7fb1c015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1bffff640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1c015f370 0x7fb1c015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1bf7fe640 1 -- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1c015f370 msgr2=0x7fb1c015f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1bf7fe640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1c015f370 0x7fb1c015f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1bf7fe640 1 -- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1c015bf30 msgr2=0x7fb1c015c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1bf7fe640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1c015bf30 0x7fb1c015c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.186+0000 7fb1bf7fe640 1 -- 172.21.15.60:0/1976671715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c015fd30 con 0x7fb1a8003970 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.187+0000 7fb1bf7fe640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1c0146f70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0037040 tx=0x7fb1b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.187+0000 7fb1a7fff640 1 -- 172.21.15.60:0/1976671715 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0003810 con 0x7fb1a8003970 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.187+0000 7fb1a7fff640 1 -- 172.21.15.60:0/1976671715 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b0003970 con 0x7fb1a8003970 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.187+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/1976671715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c01602a0 con 0x7fb1a8003970 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.187+0000 7fb1a7fff640 1 -- 172.21.15.60:0/1976671715 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0003d20 con 0x7fb1a8003970 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.187+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/1976671715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c01477c0 con 0x7fb1a8003970 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.188+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/1976671715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c0069400 con 0x7fb1a8003970 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.188+0000 7fb1a7fff640 1 -- 172.21.15.60:0/1976671715 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb1b0034440 con 0x7fb1a8003970 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.189+0000 7fb1a7fff640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb1ac06ae40 0x7fb1ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.189+0000 7fb1a7fff640 1 -- 172.21.15.60:0/1976671715 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1b00ab470 con 0x7fb1a8003970 2023-11-18T20:44:36.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.189+0000 7fb1beffd640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb1ac06ae40 0x7fb1ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.189+0000 7fb1beffd640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb1ac06ae40 0x7fb1ac06d300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb1c01f6680 tx=0x7fb1b4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:36.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.193+0000 7fb1a7fff640 1 -- 172.21.15.60:0/1976671715 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1b0076780 con 0x7fb1a8003970 2023-11-18T20:44:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.376+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/1976671715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1c00712e0 con 0x7fb1a8003970 2023-11-18T20:44:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.377+0000 7fb1a7fff640 1 -- 172.21.15.60:0/1976671715 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fb1b0079da0 con 0x7fb1a8003970 2023-11-18T20:44:36.378 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:36.378 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:34.103787+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:36.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.378+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb1ac06ae40 msgr2=0x7fb1ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.378+0000 7fb1c5f5a640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb1ac06ae40 0x7fb1ac06d300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb1c01f6680 tx=0x7fb1b4009040 comp rx=0 tx=0).stop 2023-11-18T20:44:36.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.378+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/1976671715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 msgr2=0x7fb1c0146f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.378+0000 7fb1c5f5a640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1c0146f70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0037040 tx=0x7fb1b0004730 comp rx=0 tx=0).stop 2023-11-18T20:44:36.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.378+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/1976671715 shutdown_connections 2023-11-18T20:44:36.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.379+0000 7fb1c5f5a640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb1ac06ae40 0x7fb1ac06d300 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.379+0000 7fb1c5f5a640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1c015f370 0x7fb1c015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.379+0000 7fb1c5f5a640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1c015bf30 0x7fb1c015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.379+0000 7fb1c5f5a640 1 --2- 172.21.15.60:0/1976671715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1a8003970 0x7fb1c0146f70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.379+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/1976671715 >> 172.21.15.60:0/1976671715 conn(0x7fb1c0070e70 msgr2=0x7fb1c00b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:36.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.379+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/1976671715 shutdown_connections 2023-11-18T20:44:36.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.379+0000 7fb1c5f5a640 1 -- 172.21.15.60:0/1976671715 wait complete. 2023-11-18T20:44:36.391 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-18T20:44:36.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:44:36.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.497+0000 7f7ced226640 1 Processor -- start 2023-11-18T20:44:36.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.497+0000 7f7ced226640 1 -- start start 2023-11-18T20:44:36.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7ced226640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7ce806abe0 0x7f7ce81618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7ced226640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7ce81409d0 0x7f7ce806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7ced226640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce81406f0 con 0x7f7ce8140310 2023-11-18T20:44:36.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7ced226640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce8161e20 con 0x7f7ce806abe0 2023-11-18T20:44:36.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7ced226640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce8143960 con 0x7f7ce81409d0 2023-11-18T20:44:36.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7cdffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7ce806abe0 0x7f7ce81618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7cdffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7ce806abe0 0x7f7ce81618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49904/0 (socket says 172.21.15.60:49904) 2023-11-18T20:44:36.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7cdffff640 1 -- 172.21.15.60:0/1417259695 learned_addr learned my addr 172.21.15.60:0/1417259695 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:36.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7ce759e640 1 --2- 172.21.15.60:0/1417259695 >> v2:172.21.15.100:3300/0 conn(0x7f7ce81409d0 0x7f7ce806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7cdffff640 1 -- 172.21.15.60:0/1417259695 >> v2:172.21.15.100:3300/0 conn(0x7f7ce81409d0 msgr2=0x7f7ce806efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:36.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7cdffff640 1 --2- 172.21.15.60:0/1417259695 >> v2:172.21.15.100:3300/0 conn(0x7f7ce81409d0 0x7f7ce806efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7cdffff640 1 -- 172.21.15.60:0/1417259695 >> v1:172.21.15.100:6789/0 conn(0x7f7ce8140310 legacy=0x7f7ce806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:36.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.498+0000 7f7cdffff640 1 -- 172.21.15.60:0/1417259695 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce8142de0 con 0x7f7ce806abe0 2023-11-18T20:44:36.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.499+0000 7f7cdffff640 1 --2- 172.21.15.60:0/1417259695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce806abe0 0x7f7ce81618e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8154960 tx=0x7f7cd002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ad0de8d227ac05 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.499+0000 7f7ce659c640 1 -- 172.21.15.60:0/1417259695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00317e0 con 0x7f7ce806abe0 2023-11-18T20:44:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.499+0000 7f7ce659c640 1 -- 172.21.15.60:0/1417259695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd0031940 con 0x7f7ce806abe0 2023-11-18T20:44:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.499+0000 7f7ce659c640 1 -- 172.21.15.60:0/1417259695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0031cd0 con 0x7f7ce806abe0 2023-11-18T20:44:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.500+0000 7f7ced226640 1 -- 172.21.15.60:0/1417259695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce806abe0 msgr2=0x7f7ce81618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.500+0000 7f7ced226640 1 --2- 172.21.15.60:0/1417259695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce806abe0 0x7f7ce81618e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8154960 tx=0x7f7cd002b880 comp rx=0 tx=0).stop 2023-11-18T20:44:36.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.500+0000 7f7ced226640 1 -- 172.21.15.60:0/1417259695 shutdown_connections 2023-11-18T20:44:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.500+0000 7f7ced226640 1 --2- 172.21.15.60:0/1417259695 >> v2:172.21.15.100:3300/0 conn(0x7f7ce81409d0 0x7f7ce806efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.500+0000 7f7ced226640 1 --2- 172.21.15.60:0/1417259695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce806abe0 0x7f7ce81618e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.500+0000 7f7ced226640 1 -- 172.21.15.60:0/1417259695 >> 172.21.15.60:0/1417259695 conn(0x7f7ce80b1e50 msgr2=0x7f7ce80b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.500+0000 7f7ced226640 1 -- 172.21.15.60:0/1417259695 shutdown_connections 2023-11-18T20:44:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.500+0000 7f7ced226640 1 -- 172.21.15.60:0/1417259695 wait complete. 2023-11-18T20:44:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.501+0000 7f7ced226640 1 Processor -- start 2023-11-18T20:44:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.501+0000 7f7ced226640 1 -- start start 2023-11-18T20:44:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7ced226640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ce806abe0 0x7f7ce80b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7ced226640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce8140310 0x7f7ce8145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7ced226640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce81409d0 0x7f7ce81462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7ced226640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80aeed0 con 0x7f7ce8140310 2023-11-18T20:44:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7ced226640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce8146810 con 0x7f7ce81409d0 2023-11-18T20:44:36.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7ced226640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce80ae250 con 0x7f7ce806abe0 2023-11-18T20:44:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7cdffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce8140310 0x7f7ce8145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7cdffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce8140310 0x7f7ce8145d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49914/0 (socket says 172.21.15.60:49914) 2023-11-18T20:44:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7cdffff640 1 -- 172.21.15.60:0/1495685673 learned_addr learned my addr 172.21.15.60:0/1495685673 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7ce6d9d640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ce806abe0 0x7f7ce80b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7ce759e640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce81409d0 0x7f7ce81462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7cdffff640 1 -- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ce806abe0 msgr2=0x7f7ce80b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7cdffff640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ce806abe0 0x7f7ce80b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7cdffff640 1 -- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce81409d0 msgr2=0x7f7ce81462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7cdffff640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce81409d0 0x7f7ce81462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7cdffff640 1 -- 172.21.15.60:0/1495685673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd00094d0 con 0x7f7ce8140310 2023-11-18T20:44:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7ce6d9d640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ce806abe0 0x7f7ce80b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.502+0000 7f7ce759e640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce81409d0 0x7f7ce81462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.503+0000 7f7cdffff640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce8140310 0x7f7ce8145d90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7cd002b8c0 tx=0x7f7cd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.503+0000 7f7cdf7fe640 1 -- 172.21.15.60:0/1495685673 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00329c0 con 0x7f7ce8140310 2023-11-18T20:44:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.503+0000 7f7cdf7fe640 1 -- 172.21.15.60:0/1495685673 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd0032b40 con 0x7f7ce8140310 2023-11-18T20:44:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.503+0000 7f7cdf7fe640 1 -- 172.21.15.60:0/1495685673 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00453f0 con 0x7f7ce8140310 2023-11-18T20:44:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.503+0000 7f7ced226640 1 -- 172.21.15.60:0/1495685673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce80ae530 con 0x7f7ce8140310 2023-11-18T20:44:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.503+0000 7f7ced226640 1 -- 172.21.15.60:0/1495685673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce80aeaf0 con 0x7f7ce8140310 2023-11-18T20:44:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.504+0000 7f7ced226640 1 -- 172.21.15.60:0/1495685673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce8069400 con 0x7f7ce8140310 2023-11-18T20:44:36.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.504+0000 7f7cdf7fe640 1 -- 172.21.15.60:0/1495685673 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7cd004d3f0 con 0x7f7ce8140310 2023-11-18T20:44:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.505+0000 7f7cdf7fe640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7cc806ae40 0x7f7cc806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.505+0000 7f7cdf7fe640 1 -- 172.21.15.60:0/1495685673 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cd00ab540 con 0x7f7ce8140310 2023-11-18T20:44:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.505+0000 7f7ce6d9d640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7cc806ae40 0x7f7cc806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.505+0000 7f7ce6d9d640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7cc806ae40 0x7f7cc806d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8007fd0 tx=0x7f7cd8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:36.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.509+0000 7f7cdf7fe640 1 -- 172.21.15.60:0/1495685673 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7cd0076690 con 0x7f7ce8140310 2023-11-18T20:44:36.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.679+0000 7f7ced226640 1 -- 172.21.15.60:0/1495685673 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ce8143680 con 0x7f7cc806ae40 2023-11-18T20:44:36.688 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.688+0000 7f7cdf7fe640 1 -- 172.21.15.60:0/1495685673 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289672 (secure 0 0 0) 0x7f7cd8001370 con 0x7f7cc806ae40 2023-11-18T20:44:36.691 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:36.693 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:44:36.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 -- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7cc806ae40 msgr2=0x7f7cc806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7cc806ae40 0x7f7cc806d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8007fd0 tx=0x7f7cd8009040 comp rx=0 tx=0).stop 2023-11-18T20:44:36.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 -- 172.21.15.60:0/1495685673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce8140310 msgr2=0x7f7ce8145d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce8140310 0x7f7ce8145d90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7cd002b8c0 tx=0x7f7cd00046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.695 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 -- 172.21.15.60:0/1495685673 shutdown_connections 2023-11-18T20:44:36.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7cc806ae40 0x7f7cc806d300 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ce81409d0 0x7f7ce81462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce8140310 0x7f7ce8145d90 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 --2- 172.21.15.60:0/1495685673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ce806abe0 0x7f7ce80b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 -- 172.21.15.60:0/1495685673 >> 172.21.15.60:0/1495685673 conn(0x7f7ce80b1e50 msgr2=0x7f7ce813e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:36.696 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 -- 172.21.15.60:0/1495685673 shutdown_connections 2023-11-18T20:44:36.697 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.691+0000 7f7ced226640 1 -- 172.21.15.60:0/1495685673 wait complete. 2023-11-18T20:44:36.703 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2023-11-18T20:44:35.464651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238492,"kb_used_data":5664,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736228,"statfs":{"total":773094113280,"available":772849897472,"internally_reserved":0,"allocated":5799936,"data_stored":3070761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001080"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115947+0000","last_change":"2023-11-18T20:44:33.116942+0000","last_active":"2023-11-18T20:44:34.115947+0000","last_peered":"2023-11-18T20:44:34.115947+0000","last_clean":"2023-11-18T20:44:34.115947+0000","last_became_active":"2023-11-18T20:44:33.116676+0000","last_became_peered":"2023-11-18T20:44:33.116676+0000","last_unstale":"2023-11-18T20:44:34.115947+0000","last_undegraded":"2023-11-18T20:44:34.115947+0000","last_fullsized":"2023-11-18T20:44:34.115947+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:02:32.890448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.127964+0000","last_change":"2023-11-18T20:44:33.125248+0000","last_active":"2023-11-18T20:44:33.127964+0000","last_peered":"2023-11-18T20:44:33.127964+0000","last_clean":"2023-11-18T20:44:33.127964+0000","last_became_active":"2023-11-18T20:44:33.124831+0000","last_became_peered":"2023-11-18T20:44:33.124831+0000","last_unstale":"2023-11-18T20:44:33.127964+0000","last_undegraded":"2023-11-18T20:44:33.127964+0000","last_fullsized":"2023-11-18T20:44:33.127964+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:32:40.407274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123301+0000","last_change":"2023-11-18T20:44:33.121223+0000","last_active":"2023-11-18T20:44:33.123301+0000","last_peered":"2023-11-18T20:44:33.123301+0000","last_clean":"2023-11-18T20:44:33.123301+0000","last_became_active":"2023-11-18T20:44:33.118789+0000","last_became_peered":"2023-11-18T20:44:33.118789+0000","last_unstale":"2023-11-18T20:44:33.123301+0000","last_undegraded":"2023-11-18T20:44:33.123301+0000","last_fullsized":"2023-11-18T20:44:33.123301+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:06:07.786307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.128020+0000","last_change":"2023-11-18T20:44:33.125771+0000","last_active":"2023-11-18T20:44:33.128020+0000","last_peered":"2023-11-18T20:44:33.128020+0000","last_clean":"2023-11-18T20:44:33.128020+0000","last_became_active":"2023-11-18T20:44:33.125256+0000","last_became_peered":"2023-11-18T20:44:33.125256+0000","last_unstale":"2023-11-18T20:44:33.128020+0000","last_undegraded":"2023-11-18T20:44:33.128020+0000","last_fullsized":"2023-11-18T20:44:33.128020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:41:29.137921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116156+0000","last_change":"2023-11-18T20:44:33.118231+0000","last_active":"2023-11-18T20:44:34.116156+0000","last_peered":"2023-11-18T20:44:34.116156+0000","last_clean":"2023-11-18T20:44:34.116156+0000","last_became_active":"2023-11-18T20:44:33.117934+0000","last_became_peered":"2023-11-18T20:44:33.117934+0000","last_unstale":"2023-11-18T20:44:34.116156+0000","last_undegraded":"2023-11-18T20:44:34.116156+0000","last_fullsized":"2023-11-18T20:44:34.116156+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:51:46.829106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115777+0000","last_change":"2023-11-18T20:44:33.115629+0000","last_active":"2023-11-18T20:44:34.115777+0000","last_peered":"2023-11-18T20:44:34.115777+0000","last_clean":"2023-11-18T20:44:34.115777+0000","last_became_active":"2023-11-18T20:44:33.115359+0000","last_became_peered":"2023-11-18T20:44:33.115359+0000","last_unstale":"2023-11-18T20:44:34.115777+0000","last_undegraded":"2023-11-18T20:44:34.115777+0000","last_fullsized":"2023-11-18T20:44:34.115777+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:33:54.326294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123241+0000","last_change":"2023-11-18T20:44:33.120383+0000","last_active":"2023-11-18T20:44:33.123241+0000","last_peered":"2023-11-18T20:44:33.123241+0000","last_clean":"2023-11-18T20:44:33.123241+0000","last_became_active":"2023-11-18T20:44:33.120107+0000","last_became_peered":"2023-11-18T20:44:33.120107+0000","last_unstale":"2023-11-18T20:44:33.123241+0000","last_undegraded":"2023-11-18T20:44:33.123241+0000","last_fullsized":"2023-11-18T20:44:33.123241+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:06:33.900376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123243+0000","last_change":"2023-11-18T20:44:33.118644+0000","last_active":"2023-11-18T20:44:33.123243+0000","last_peered":"2023-11-18T20:44:33.123243+0000","last_clean":"2023-11-18T20:44:33.123243+0000","last_became_active":"2023-11-18T20:44:33.118363+0000","last_became_peered":"2023-11-18T20:44:33.118363+0000","last_unstale":"2023-11-18T20:44:33.123243+0000","last_undegraded":"2023-11-18T20:44:33.123243+0000","last_fullsized":"2023-11-18T20:44:33.123243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:10:39.243298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126979+0000","last_change":"2023-11-18T20:44:33.124673+0000","last_active":"2023-11-18T20:44:33.126979+0000","last_peered":"2023-11-18T20:44:33.126979+0000","last_clean":"2023-11-18T20:44:33.126979+0000","last_became_active":"2023-11-18T20:44:33.124432+0000","last_became_peered":"2023-11-18T20:44:33.124432+0000","last_unstale":"2023-11-18T20:44:33.126979+0000","last_undegraded":"2023-11-18T20:44:33.126979+0000","last_fullsized":"2023-11-18T20:44:33.126979+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:54:29.346693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116004+0000","last_change":"2023-11-18T20:44:33.116176+0000","last_active":"2023-11-18T20:44:34.116004+0000","last_peered":"2023-11-18T20:44:34.116004+0000","last_clean":"2023-11-18T20:44:34.116004+0000","last_became_active":"2023-11-18T20:44:33.115899+0000","last_became_peered":"2023-11-18T20:44:33.115899+0000","last_unstale":"2023-11-18T20:44:34.116004+0000","last_undegraded":"2023-11-18T20:44:34.116004+0000","last_fullsized":"2023-11-18T20:44:34.116004+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:06:48.969899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.112770+0000","last_change":"2023-11-18T20:44:33.123459+0000","last_active":"2023-11-18T20:44:34.112770+0000","last_peered":"2023-11-18T20:44:34.112770+0000","last_clean":"2023-11-18T20:44:34.112770+0000","last_became_active":"2023-11-18T20:44:33.122722+0000","last_became_peered":"2023-11-18T20:44:33.122722+0000","last_unstale":"2023-11-18T20:44:34.112770+0000","last_undegraded":"2023-11-18T20:44:34.112770+0000","last_fullsized":"2023-11-18T20:44:34.112770+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:24:45.422200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.124150+0000","last_change":"2023-11-18T20:44:33.119993+0000","last_active":"2023-11-18T20:44:33.124150+0000","last_peered":"2023-11-18T20:44:33.124150+0000","last_clean":"2023-11-18T20:44:33.124150+0000","last_became_active":"2023-11-18T20:44:33.119729+0000","last_became_peered":"2023-11-18T20:44:33.119729+0000","last_unstale":"2023-11-18T20:44:33.124150+0000","last_undegraded":"2023-11-18T20:44:33.124150+0000","last_fullsized":"2023-11-18T20:44:33.124150+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:06:48.969899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115725+0000","last_change":"2023-11-18T20:44:33.109703+0000","last_active":"2023-11-18T20:44:34.115725+0000","last_peered":"2023-11-18T20:44:34.115725+0000","last_clean":"2023-11-18T20:44:34.115725+0000","last_became_active":"2023-11-18T20:44:33.109428+0000","last_became_peered":"2023-11-18T20:44:33.109428+0000","last_unstale":"2023-11-18T20:44:34.115725+0000","last_undegraded":"2023-11-18T20:44:34.115725+0000","last_fullsized":"2023-11-18T20:44:34.115725+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:20:37.793311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123324+0000","last_change":"2023-11-18T20:44:33.120658+0000","last_active":"2023-11-18T20:44:33.123324+0000","last_peered":"2023-11-18T20:44:33.123324+0000","last_clean":"2023-11-18T20:44:33.123324+0000","last_became_active":"2023-11-18T20:44:33.118985+0000","last_became_peered":"2023-11-18T20:44:33.118985+0000","last_unstale":"2023-11-18T20:44:33.123324+0000","last_undegraded":"2023-11-18T20:44:33.123324+0000","last_fullsized":"2023-11-18T20:44:33.123324+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:01:59.298618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.121579+0000","last_change":"2023-11-18T20:44:33.120176+0000","last_active":"2023-11-18T20:44:33.121579+0000","last_peered":"2023-11-18T20:44:33.121579+0000","last_clean":"2023-11-18T20:44:33.121579+0000","last_became_active":"2023-11-18T20:44:33.119922+0000","last_became_peered":"2023-11-18T20:44:33.119922+0000","last_unstale":"2023-11-18T20:44:33.121579+0000","last_undegraded":"2023-11-18T20:44:33.121579+0000","last_fullsized":"2023-11-18T20:44:33.121579+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:54:20.419244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115646+0000","last_change":"2023-11-18T20:44:33.110026+0000","last_active":"2023-11-18T20:44:34.115646+0000","last_peered":"2023-11-18T20:44:34.115646+0000","last_clean":"2023-11-18T20:44:34.115646+0000","last_became_active":"2023-11-18T20:44:33.109743+0000","last_became_peered":"2023-11-18T20:44:33.109743+0000","last_unstale":"2023-11-18T20:44:34.115646+0000","last_undegraded":"2023-11-18T20:44:34.115646+0000","last_fullsized":"2023-11-18T20:44:34.115646+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:26:14.401043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.117717+0000","last_change":"2023-11-18T20:44:33.115926+0000","last_active":"2023-11-18T20:44:33.117717+0000","last_peered":"2023-11-18T20:44:33.117717+0000","last_clean":"2023-11-18T20:44:33.117717+0000","last_became_active":"2023-11-18T20:44:33.115544+0000","last_became_peered":"2023-11-18T20:44:33.115544+0000","last_unstale":"2023-11-18T20:44:33.117717+0000","last_undegraded":"2023-11-18T20:44:33.117717+0000","last_fullsized":"2023-11-18T20:44:33.117717+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:37:18.028311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.127543+0000","last_change":"2023-11-18T20:44:33.124555+0000","last_active":"2023-11-18T20:44:33.127543+0000","last_peered":"2023-11-18T20:44:33.127543+0000","last_clean":"2023-11-18T20:44:33.127543+0000","last_became_active":"2023-11-18T20:44:33.124294+0000","last_became_peered":"2023-11-18T20:44:33.124294+0000","last_unstale":"2023-11-18T20:44:33.127543+0000","last_undegraded":"2023-11-18T20:44:33.127543+0000","last_fullsized":"2023-11-18T20:44:33.127543+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:26:14.401043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.124003+0000","last_change":"2023-11-18T20:44:33.122988+0000","last_active":"2023-11-18T20:44:33.124003+0000","last_peered":"2023-11-18T20:44:33.124003+0000","last_clean":"2023-11-18T20:44:33.124003+0000","last_became_active":"2023-11-18T20:44:33.122737+0000","last_became_peered":"2023-11-18T20:44:33.122737+0000","last_unstale":"2023-11-18T20:44:33.124003+0000","last_undegraded":"2023-11-18T20:44:33.124003+0000","last_fullsized":"2023-11-18T20:44:33.124003+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:41:29.137921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123545+0000","last_change":"2023-11-18T20:44:33.121172+0000","last_active":"2023-11-18T20:44:33.123545+0000","last_peered":"2023-11-18T20:44:33.123545+0000","last_clean":"2023-11-18T20:44:33.123545+0000","last_became_active":"2023-11-18T20:44:33.120870+0000","last_became_peered":"2023-11-18T20:44:33.120870+0000","last_unstale":"2023-11-18T20:44:33.123545+0000","last_undegraded":"2023-11-18T20:44:33.123545+0000","last_fullsized":"2023-11-18T20:44:33.123545+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:41.474148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.120609+0000","last_change":"2023-11-18T20:44:33.118345+0000","last_active":"2023-11-18T20:44:33.120609+0000","last_peered":"2023-11-18T20:44:33.120609+0000","last_clean":"2023-11-18T20:44:33.120609+0000","last_became_active":"2023-11-18T20:44:33.117979+0000","last_became_peered":"2023-11-18T20:44:33.117979+0000","last_unstale":"2023-11-18T20:44:33.120609+0000","last_undegraded":"2023-11-18T20:44:33.120609+0000","last_fullsized":"2023-11-18T20:44:33.120609+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:14:25.168203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123191+0000","last_change":"2023-11-18T20:44:33.118614+0000","last_active":"2023-11-18T20:44:33.123191+0000","last_peered":"2023-11-18T20:44:33.123191+0000","last_clean":"2023-11-18T20:44:33.123191+0000","last_became_active":"2023-11-18T20:44:33.118331+0000","last_became_peered":"2023-11-18T20:44:33.118331+0000","last_unstale":"2023-11-18T20:44:33.123191+0000","last_undegraded":"2023-11-18T20:44:33.123191+0000","last_fullsized":"2023-11-18T20:44:33.123191+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:37:18.028311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.112525+0000","last_change":"2023-11-18T20:44:33.118064+0000","last_active":"2023-11-18T20:44:34.112525+0000","last_peered":"2023-11-18T20:44:34.112525+0000","last_clean":"2023-11-18T20:44:34.112525+0000","last_became_active":"2023-11-18T20:44:33.117769+0000","last_became_peered":"2023-11-18T20:44:33.117769+0000","last_unstale":"2023-11-18T20:44:34.112525+0000","last_undegraded":"2023-11-18T20:44:34.112525+0000","last_fullsized":"2023-11-18T20:44:34.112525+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:27:47.528034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123093+0000","last_change":"2023-11-18T20:44:33.117627+0000","last_active":"2023-11-18T20:44:33.123093+0000","last_peered":"2023-11-18T20:44:33.123093+0000","last_clean":"2023-11-18T20:44:33.123093+0000","last_became_active":"2023-11-18T20:44:33.117366+0000","last_became_peered":"2023-11-18T20:44:33.117366+0000","last_unstale":"2023-11-18T20:44:33.123093+0000","last_undegraded":"2023-11-18T20:44:33.123093+0000","last_fullsized":"2023-11-18T20:44:33.123093+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:44:27.625798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126479+0000","last_change":"2023-11-18T20:44:33.125109+0000","last_active":"2023-11-18T20:44:33.126479+0000","last_peered":"2023-11-18T20:44:33.126479+0000","last_clean":"2023-11-18T20:44:33.126479+0000","last_became_active":"2023-11-18T20:44:33.124820+0000","last_became_peered":"2023-11-18T20:44:33.124820+0000","last_unstale":"2023-11-18T20:44:33.126479+0000","last_undegraded":"2023-11-18T20:44:33.126479+0000","last_fullsized":"2023-11-18T20:44:33.126479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:33:54.326294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126689+0000","last_change":"2023-11-18T20:44:33.123975+0000","last_active":"2023-11-18T20:44:33.126689+0000","last_peered":"2023-11-18T20:44:33.126689+0000","last_clean":"2023-11-18T20:44:33.126689+0000","last_became_active":"2023-11-18T20:44:33.123323+0000","last_became_peered":"2023-11-18T20:44:33.123323+0000","last_unstale":"2023-11-18T20:44:33.126689+0000","last_undegraded":"2023-11-18T20:44:33.126689+0000","last_fullsized":"2023-11-18T20:44:33.126689+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:57:38.502768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116020+0000","last_change":"2023-11-18T20:44:33.118477+0000","last_active":"2023-11-18T20:44:34.116020+0000","last_peered":"2023-11-18T20:44:34.116020+0000","last_clean":"2023-11-18T20:44:34.116020+0000","last_became_active":"2023-11-18T20:44:33.118221+0000","last_became_peered":"2023-11-18T20:44:33.118221+0000","last_unstale":"2023-11-18T20:44:34.116020+0000","last_undegraded":"2023-11-18T20:44:34.116020+0000","last_fullsized":"2023-11-18T20:44:34.116020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:20:57.880034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.121044+0000","last_change":"2023-11-18T20:44:33.118385+0000","last_active":"2023-11-18T20:44:33.121044+0000","last_peered":"2023-11-18T20:44:33.121044+0000","last_clean":"2023-11-18T20:44:33.121044+0000","last_became_active":"2023-11-18T20:44:33.118123+0000","last_became_peered":"2023-11-18T20:44:33.118123+0000","last_unstale":"2023-11-18T20:44:33.121044+0000","last_undegraded":"2023-11-18T20:44:33.121044+0000","last_fullsized":"2023-11-18T20:44:33.121044+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:21:17.602054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123026+0000","last_change":"2023-11-18T20:44:33.117894+0000","last_active":"2023-11-18T20:44:33.123026+0000","last_peered":"2023-11-18T20:44:33.123026+0000","last_clean":"2023-11-18T20:44:33.123026+0000","last_became_active":"2023-11-18T20:44:33.117402+0000","last_became_peered":"2023-11-18T20:44:33.117402+0000","last_unstale":"2023-11-18T20:44:33.123026+0000","last_undegraded":"2023-11-18T20:44:33.123026+0000","last_fullsized":"2023-11-18T20:44:33.123026+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:51:46.829106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123457+0000","last_change":"2023-11-18T20:44:33.122306+0000","last_active":"2023-11-18T20:44:33.123457+0000","last_peered":"2023-11-18T20:44:33.123457+0000","last_clean":"2023-11-18T20:44:33.123457+0000","last_became_active":"2023-11-18T20:44:33.121421+0000","last_became_peered":"2023-11-18T20:44:33.121421+0000","last_unstale":"2023-11-18T20:44:33.123457+0000","last_undegraded":"2023-11-18T20:44:33.123457+0000","last_fullsized":"2023-11-18T20:44:33.123457+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:06:07.786307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123223+0000","last_change":"2023-11-18T20:44:33.120333+0000","last_active":"2023-11-18T20:44:33.123223+0000","last_peered":"2023-11-18T20:44:33.123223+0000","last_clean":"2023-11-18T20:44:33.123223+0000","last_became_active":"2023-11-18T20:44:33.119276+0000","last_became_peered":"2023-11-18T20:44:33.119276+0000","last_unstale":"2023-11-18T20:44:33.123223+0000","last_undegraded":"2023-11-18T20:44:33.123223+0000","last_fullsized":"2023-11-18T20:44:33.123223+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:11:39.561038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.127515+0000","last_change":"2023-11-18T20:44:33.123413+0000","last_active":"2023-11-18T20:44:33.127515+0000","last_peered":"2023-11-18T20:44:33.127515+0000","last_clean":"2023-11-18T20:44:33.127515+0000","last_became_active":"2023-11-18T20:44:33.123097+0000","last_became_peered":"2023-11-18T20:44:33.123097+0000","last_unstale":"2023-11-18T20:44:33.127515+0000","last_undegraded":"2023-11-18T20:44:33.127515+0000","last_fullsized":"2023-11-18T20:44:33.127515+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:54:54.404042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.120562+0000","last_change":"2023-11-18T20:44:33.119113+0000","last_active":"2023-11-18T20:44:33.120562+0000","last_peered":"2023-11-18T20:44:33.120562+0000","last_clean":"2023-11-18T20:44:33.120562+0000","last_became_active":"2023-11-18T20:44:33.117333+0000","last_became_peered":"2023-11-18T20:44:33.117333+0000","last_unstale":"2023-11-18T20:44:33.120562+0000","last_undegraded":"2023-11-18T20:44:33.120562+0000","last_fullsized":"2023-11-18T20:44:33.120562+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:13:57.153760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.708281+0000","last_change":"2023-11-18T20:44:32.709740+0000","last_active":"2023-11-18T20:44:33.708281+0000","last_peered":"2023-11-18T20:44:33.708281+0000","last_clean":"2023-11-18T20:44:33.708281+0000","last_became_active":"2023-11-18T20:44:32.709460+0000","last_became_peered":"2023-11-18T20:44:32.709460+0000","last_unstale":"2023-11-18T20:44:33.708281+0000","last_undegraded":"2023-11-18T20:44:33.708281+0000","last_fullsized":"2023-11-18T20:44:33.708281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:01:11.915156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.118628+0000","last_change":"2023-11-18T20:44:33.117519+0000","last_active":"2023-11-18T20:44:33.118628+0000","last_peered":"2023-11-18T20:44:33.118628+0000","last_clean":"2023-11-18T20:44:33.118628+0000","last_became_active":"2023-11-18T20:44:33.117261+0000","last_became_peered":"2023-11-18T20:44:33.117261+0000","last_unstale":"2023-11-18T20:44:33.118628+0000","last_undegraded":"2023-11-18T20:44:33.118628+0000","last_fullsized":"2023-11-18T20:44:33.118628+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:11:39.561038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115941+0000","last_change":"2023-11-18T20:44:32.714052+0000","last_active":"2023-11-18T20:44:34.115941+0000","last_peered":"2023-11-18T20:44:34.115941+0000","last_clean":"2023-11-18T20:44:34.115941+0000","last_became_active":"2023-11-18T20:44:32.713569+0000","last_became_peered":"2023-11-18T20:44:32.713569+0000","last_unstale":"2023-11-18T20:44:34.115941+0000","last_undegraded":"2023-11-18T20:44:34.115941+0000","last_fullsized":"2023-11-18T20:44:34.115941+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:54:19.414371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.124935+0000","last_change":"2023-11-18T20:44:33.123311+0000","last_active":"2023-11-18T20:44:33.124935+0000","last_peered":"2023-11-18T20:44:33.124935+0000","last_clean":"2023-11-18T20:44:33.124935+0000","last_became_active":"2023-11-18T20:44:33.123039+0000","last_became_peered":"2023-11-18T20:44:33.123039+0000","last_unstale":"2023-11-18T20:44:33.124935+0000","last_undegraded":"2023-11-18T20:44:33.124935+0000","last_fullsized":"2023-11-18T20:44:33.124935+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:50:01.856323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115987+0000","last_change":"2023-11-18T20:44:32.708656+0000","last_active":"2023-11-18T20:44:34.115987+0000","last_peered":"2023-11-18T20:44:34.115987+0000","last_clean":"2023-11-18T20:44:34.115987+0000","last_became_active":"2023-11-18T20:44:32.708373+0000","last_became_peered":"2023-11-18T20:44:32.708373+0000","last_unstale":"2023-11-18T20:44:34.115987+0000","last_undegraded":"2023-11-18T20:44:34.115987+0000","last_fullsized":"2023-11-18T20:44:34.115987+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:59:19.079526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123573+0000","last_change":"2023-11-18T20:44:33.119348+0000","last_active":"2023-11-18T20:44:33.123573+0000","last_peered":"2023-11-18T20:44:33.123573+0000","last_clean":"2023-11-18T20:44:33.123573+0000","last_became_active":"2023-11-18T20:44:33.119103+0000","last_became_peered":"2023-11-18T20:44:33.119103+0000","last_unstale":"2023-11-18T20:44:33.123573+0000","last_undegraded":"2023-11-18T20:44:33.123573+0000","last_fullsized":"2023-11-18T20:44:33.123573+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:20:57.880034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.704535+0000","last_change":"2023-11-18T20:44:32.711272+0000","last_active":"2023-11-18T20:44:33.704535+0000","last_peered":"2023-11-18T20:44:33.704535+0000","last_clean":"2023-11-18T20:44:33.704535+0000","last_became_active":"2023-11-18T20:44:32.710967+0000","last_became_peered":"2023-11-18T20:44:32.710967+0000","last_unstale":"2023-11-18T20:44:33.704535+0000","last_undegraded":"2023-11-18T20:44:33.704535+0000","last_fullsized":"2023-11-18T20:44:33.704535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:54:19.414371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116360+0000","last_change":"2023-11-18T20:44:33.112210+0000","last_active":"2023-11-18T20:44:34.116360+0000","last_peered":"2023-11-18T20:44:34.116360+0000","last_clean":"2023-11-18T20:44:34.116360+0000","last_became_active":"2023-11-18T20:44:33.111901+0000","last_became_peered":"2023-11-18T20:44:33.111901+0000","last_unstale":"2023-11-18T20:44:34.116360+0000","last_undegraded":"2023-11-18T20:44:34.116360+0000","last_fullsized":"2023-11-18T20:44:34.116360+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:59:20.084399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.706398+0000","last_change":"2023-11-18T20:44:32.707864+0000","last_active":"2023-11-18T20:44:33.706398+0000","last_peered":"2023-11-18T20:44:33.706398+0000","last_clean":"2023-11-18T20:44:33.706398+0000","last_became_active":"2023-11-18T20:44:32.706893+0000","last_became_peered":"2023-11-18T20:44:32.706893+0000","last_unstale":"2023-11-18T20:44:33.706398+0000","last_undegraded":"2023-11-18T20:44:33.706398+0000","last_fullsized":"2023-11-18T20:44:33.706398+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:54:19.414371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123998+0000","last_change":"2023-11-18T20:44:33.122677+0000","last_active":"2023-11-18T20:44:33.123998+0000","last_peered":"2023-11-18T20:44:33.123998+0000","last_clean":"2023-11-18T20:44:33.123998+0000","last_became_active":"2023-11-18T20:44:33.121136+0000","last_became_peered":"2023-11-18T20:44:33.121136+0000","last_unstale":"2023-11-18T20:44:33.123998+0000","last_undegraded":"2023-11-18T20:44:33.123998+0000","last_fullsized":"2023-11-18T20:44:33.123998+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:20:37.793311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116015+0000","last_change":"2023-11-18T20:44:32.706595+0000","last_active":"2023-11-18T20:44:34.116015+0000","last_peered":"2023-11-18T20:44:34.116015+0000","last_clean":"2023-11-18T20:44:34.116015+0000","last_became_active":"2023-11-18T20:44:32.706211+0000","last_became_peered":"2023-11-18T20:44:32.706211+0000","last_unstale":"2023-11-18T20:44:34.116015+0000","last_undegraded":"2023-11-18T20:44:34.116015+0000","last_fullsized":"2023-11-18T20:44:34.116015+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:01:58.293745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126709+0000","last_change":"2023-11-18T20:44:33.124122+0000","last_active":"2023-11-18T20:44:33.126709+0000","last_peered":"2023-11-18T20:44:33.126709+0000","last_clean":"2023-11-18T20:44:33.126709+0000","last_became_active":"2023-11-18T20:44:33.123675+0000","last_became_peered":"2023-11-18T20:44:33.123675+0000","last_unstale":"2023-11-18T20:44:33.126709+0000","last_undegraded":"2023-11-18T20:44:33.126709+0000","last_fullsized":"2023-11-18T20:44:33.126709+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:26:32.984520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.112124+0000","last_change":"2023-11-18T20:44:32.705013+0000","last_active":"2023-11-18T20:44:33.112124+0000","last_peered":"2023-11-18T20:44:33.112124+0000","last_clean":"2023-11-18T20:44:33.112124+0000","last_became_active":"2023-11-18T20:44:32.704734+0000","last_became_peered":"2023-11-18T20:44:32.704734+0000","last_unstale":"2023-11-18T20:44:33.112124+0000","last_undegraded":"2023-11-18T20:44:33.112124+0000","last_fullsized":"2023-11-18T20:44:33.112124+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:02:56.669743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.128013+0000","last_change":"2023-11-18T20:44:33.125639+0000","last_active":"2023-11-18T20:44:33.128013+0000","last_peered":"2023-11-18T20:44:33.128013+0000","last_clean":"2023-11-18T20:44:33.128013+0000","last_became_active":"2023-11-18T20:44:33.125309+0000","last_became_peered":"2023-11-18T20:44:33.125309+0000","last_unstale":"2023-11-18T20:44:33.128013+0000","last_undegraded":"2023-11-18T20:44:33.128013+0000","last_fullsized":"2023-11-18T20:44:33.128013+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:37:18.028311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.107823+0000","last_change":"2023-11-18T20:44:32.707569+0000","last_active":"2023-11-18T20:44:33.107823+0000","last_peered":"2023-11-18T20:44:33.107823+0000","last_clean":"2023-11-18T20:44:33.107823+0000","last_became_active":"2023-11-18T20:44:32.707269+0000","last_became_peered":"2023-11-18T20:44:32.707269+0000","last_unstale":"2023-11-18T20:44:33.107823+0000","last_undegraded":"2023-11-18T20:44:33.107823+0000","last_fullsized":"2023-11-18T20:44:33.107823+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:26:20.158332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116082+0000","last_change":"2023-11-18T20:44:33.119997+0000","last_active":"2023-11-18T20:44:34.116082+0000","last_peered":"2023-11-18T20:44:34.116082+0000","last_clean":"2023-11-18T20:44:34.116082+0000","last_became_active":"2023-11-18T20:44:33.119162+0000","last_became_peered":"2023-11-18T20:44:33.119162+0000","last_unstale":"2023-11-18T20:44:34.116082+0000","last_undegraded":"2023-11-18T20:44:34.116082+0000","last_fullsized":"2023-11-18T20:44:34.116082+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:06:33.900376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.117047+0000","last_change":"2023-11-18T20:44:32.707847+0000","last_active":"2023-11-18T20:44:33.117047+0000","last_peered":"2023-11-18T20:44:33.117047+0000","last_clean":"2023-11-18T20:44:33.117047+0000","last_became_active":"2023-11-18T20:44:32.707528+0000","last_became_peered":"2023-11-18T20:44:32.707528+0000","last_unstale":"2023-11-18T20:44:33.117047+0000","last_undegraded":"2023-11-18T20:44:33.117047+0000","last_fullsized":"2023-11-18T20:44:33.117047+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:10:38.238426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.122490+0000","last_change":"2023-11-18T20:44:33.120107+0000","last_active":"2023-11-18T20:44:33.122490+0000","last_peered":"2023-11-18T20:44:33.122490+0000","last_clean":"2023-11-18T20:44:33.122490+0000","last_became_active":"2023-11-18T20:44:33.119458+0000","last_became_peered":"2023-11-18T20:44:33.119458+0000","last_unstale":"2023-11-18T20:44:33.122490+0000","last_undegraded":"2023-11-18T20:44:33.122490+0000","last_fullsized":"2023-11-18T20:44:33.122490+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:51:46.829106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116296+0000","last_change":"2023-11-18T20:44:32.706237+0000","last_active":"2023-11-18T20:44:34.116296+0000","last_peered":"2023-11-18T20:44:34.116296+0000","last_clean":"2023-11-18T20:44:34.116296+0000","last_became_active":"2023-11-18T20:44:32.705933+0000","last_became_peered":"2023-11-18T20:44:32.705933+0000","last_unstale":"2023-11-18T20:44:34.116296+0000","last_undegraded":"2023-11-18T20:44:34.116296+0000","last_fullsized":"2023-11-18T20:44:34.116296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:11:38.556165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123704+0000","last_change":"2023-11-18T20:44:33.122460+0000","last_active":"2023-11-18T20:44:33.123704+0000","last_peered":"2023-11-18T20:44:33.123704+0000","last_clean":"2023-11-18T20:44:33.123704+0000","last_became_active":"2023-11-18T20:44:33.121060+0000","last_became_peered":"2023-11-18T20:44:33.121060+0000","last_unstale":"2023-11-18T20:44:33.123704+0000","last_undegraded":"2023-11-18T20:44:33.123704+0000","last_fullsized":"2023-11-18T20:44:33.123704+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:29:16.432705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.706741+0000","last_change":"2023-11-18T20:44:32.704847+0000","last_active":"2023-11-18T20:44:33.706741+0000","last_peered":"2023-11-18T20:44:33.706741+0000","last_clean":"2023-11-18T20:44:33.706741+0000","last_became_active":"2023-11-18T20:44:32.704568+0000","last_became_peered":"2023-11-18T20:44:32.704568+0000","last_unstale":"2023-11-18T20:44:33.706741+0000","last_undegraded":"2023-11-18T20:44:33.706741+0000","last_fullsized":"2023-11-18T20:44:33.706741+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:33:53.321421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116247+0000","last_change":"2023-11-18T20:44:33.115826+0000","last_active":"2023-11-18T20:44:34.116247+0000","last_peered":"2023-11-18T20:44:34.116247+0000","last_clean":"2023-11-18T20:44:34.116247+0000","last_became_active":"2023-11-18T20:44:33.115552+0000","last_became_peered":"2023-11-18T20:44:33.115552+0000","last_unstale":"2023-11-18T20:44:34.116247+0000","last_undegraded":"2023-11-18T20:44:34.116247+0000","last_fullsized":"2023-11-18T20:44:34.116247+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:06:07.786307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.709539+0000","last_change":"2023-11-18T20:44:32.715407+0000","last_active":"2023-11-18T20:44:33.709539+0000","last_peered":"2023-11-18T20:44:33.709539+0000","last_clean":"2023-11-18T20:44:33.709539+0000","last_became_active":"2023-11-18T20:44:32.715093+0000","last_became_peered":"2023-11-18T20:44:32.715093+0000","last_unstale":"2023-11-18T20:44:33.709539+0000","last_undegraded":"2023-11-18T20:44:33.709539+0000","last_fullsized":"2023-11-18T20:44:33.709539+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:11.455213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.127553+0000","last_change":"2023-11-18T20:44:33.124043+0000","last_active":"2023-11-18T20:44:33.127553+0000","last_peered":"2023-11-18T20:44:33.127553+0000","last_clean":"2023-11-18T20:44:33.127553+0000","last_became_active":"2023-11-18T20:44:33.123681+0000","last_became_peered":"2023-11-18T20:44:33.123681+0000","last_unstale":"2023-11-18T20:44:33.127553+0000","last_undegraded":"2023-11-18T20:44:33.127553+0000","last_fullsized":"2023-11-18T20:44:33.127553+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:06:07.786307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.708398+0000","last_change":"2023-11-18T20:44:32.710524+0000","last_active":"2023-11-18T20:44:33.708398+0000","last_peered":"2023-11-18T20:44:33.708398+0000","last_clean":"2023-11-18T20:44:33.708398+0000","last_became_active":"2023-11-18T20:44:32.709992+0000","last_became_peered":"2023-11-18T20:44:32.709992+0000","last_unstale":"2023-11-18T20:44:33.708398+0000","last_undegraded":"2023-11-18T20:44:33.708398+0000","last_fullsized":"2023-11-18T20:44:33.708398+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:26:13.396170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.118237+0000","last_change":"2023-11-18T20:44:33.116699+0000","last_active":"2023-11-18T20:44:33.118237+0000","last_peered":"2023-11-18T20:44:33.118237+0000","last_clean":"2023-11-18T20:44:33.118237+0000","last_became_active":"2023-11-18T20:44:33.116431+0000","last_became_peered":"2023-11-18T20:44:33.116431+0000","last_unstale":"2023-11-18T20:44:33.118237+0000","last_undegraded":"2023-11-18T20:44:33.118237+0000","last_fullsized":"2023-11-18T20:44:33.118237+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:54:54.404042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.117611+0000","last_change":"2023-11-18T20:44:32.708820+0000","last_active":"2023-11-18T20:44:33.117611+0000","last_peered":"2023-11-18T20:44:33.117611+0000","last_clean":"2023-11-18T20:44:33.117611+0000","last_became_active":"2023-11-18T20:44:32.708474+0000","last_became_peered":"2023-11-18T20:44:32.708474+0000","last_unstale":"2023-11-18T20:44:33.117611+0000","last_undegraded":"2023-11-18T20:44:33.117611+0000","last_fullsized":"2023-11-18T20:44:33.117611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:12:28.042350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123829+0000","last_change":"2023-11-18T20:44:33.118832+0000","last_active":"2023-11-18T20:44:33.123829+0000","last_peered":"2023-11-18T20:44:33.123829+0000","last_clean":"2023-11-18T20:44:33.123829+0000","last_became_active":"2023-11-18T20:44:33.118555+0000","last_became_peered":"2023-11-18T20:44:33.118555+0000","last_unstale":"2023-11-18T20:44:33.123829+0000","last_undegraded":"2023-11-18T20:44:33.123829+0000","last_fullsized":"2023-11-18T20:44:33.123829+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:38:48.032417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.111699+0000","last_change":"2023-11-18T20:44:32.705943+0000","last_active":"2023-11-18T20:44:33.111699+0000","last_peered":"2023-11-18T20:44:33.111699+0000","last_clean":"2023-11-18T20:44:33.111699+0000","last_became_active":"2023-11-18T20:44:32.705631+0000","last_became_peered":"2023-11-18T20:44:32.705631+0000","last_unstale":"2023-11-18T20:44:33.111699+0000","last_undegraded":"2023-11-18T20:44:33.111699+0000","last_fullsized":"2023-11-18T20:44:33.111699+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:59:19.079526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.117694+0000","last_change":"2023-11-18T20:44:33.115774+0000","last_active":"2023-11-18T20:44:33.117694+0000","last_peered":"2023-11-18T20:44:33.117694+0000","last_clean":"2023-11-18T20:44:33.117694+0000","last_became_active":"2023-11-18T20:44:33.115236+0000","last_became_peered":"2023-11-18T20:44:33.115236+0000","last_unstale":"2023-11-18T20:44:33.117694+0000","last_undegraded":"2023-11-18T20:44:33.117694+0000","last_fullsized":"2023-11-18T20:44:33.117694+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:20:37.793311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.708066+0000","last_change":"2023-11-18T20:44:32.706703+0000","last_active":"2023-11-18T20:44:33.708066+0000","last_peered":"2023-11-18T20:44:33.708066+0000","last_clean":"2023-11-18T20:44:33.708066+0000","last_became_active":"2023-11-18T20:44:32.706238+0000","last_became_peered":"2023-11-18T20:44:32.706238+0000","last_unstale":"2023-11-18T20:44:33.708066+0000","last_undegraded":"2023-11-18T20:44:33.708066+0000","last_fullsized":"2023-11-18T20:44:33.708066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:27:26.687286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126721+0000","last_change":"2023-11-18T20:44:33.122436+0000","last_active":"2023-11-18T20:44:33.126721+0000","last_peered":"2023-11-18T20:44:33.126721+0000","last_clean":"2023-11-18T20:44:33.126721+0000","last_became_active":"2023-11-18T20:44:33.122175+0000","last_became_peered":"2023-11-18T20:44:33.122175+0000","last_unstale":"2023-11-18T20:44:33.126721+0000","last_undegraded":"2023-11-18T20:44:33.126721+0000","last_fullsized":"2023-11-18T20:44:33.126721+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:02:32.890448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.706456+0000","last_change":"2023-11-18T20:44:32.711248+0000","last_active":"2023-11-18T20:44:33.706456+0000","last_peered":"2023-11-18T20:44:33.706456+0000","last_clean":"2023-11-18T20:44:33.706456+0000","last_became_active":"2023-11-18T20:44:32.710964+0000","last_became_peered":"2023-11-18T20:44:32.710964+0000","last_unstale":"2023-11-18T20:44:33.706456+0000","last_undegraded":"2023-11-18T20:44:33.706456+0000","last_fullsized":"2023-11-18T20:44:33.706456+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:10:38.238426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115719+0000","last_change":"2023-11-18T20:44:33.112324+0000","last_active":"2023-11-18T20:44:34.115719+0000","last_peered":"2023-11-18T20:44:34.115719+0000","last_clean":"2023-11-18T20:44:34.115719+0000","last_became_active":"2023-11-18T20:44:33.112052+0000","last_became_peered":"2023-11-18T20:44:33.112052+0000","last_unstale":"2023-11-18T20:44:34.115719+0000","last_undegraded":"2023-11-18T20:44:34.115719+0000","last_fullsized":"2023-11-18T20:44:34.115719+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:41:29.137921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.112806+0000","last_change":"2023-11-18T20:44:32.709435+0000","last_active":"2023-11-18T20:44:34.112806+0000","last_peered":"2023-11-18T20:44:34.112806+0000","last_clean":"2023-11-18T20:44:34.112806+0000","last_became_active":"2023-11-18T20:44:32.709151+0000","last_became_peered":"2023-11-18T20:44:32.709151+0000","last_unstale":"2023-11-18T20:44:34.112806+0000","last_undegraded":"2023-11-18T20:44:34.112806+0000","last_fullsized":"2023-11-18T20:44:34.112806+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:11:43.615416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116011+0000","last_change":"2023-11-18T20:44:33.112491+0000","last_active":"2023-11-18T20:44:34.116011+0000","last_peered":"2023-11-18T20:44:34.116011+0000","last_clean":"2023-11-18T20:44:34.116011+0000","last_became_active":"2023-11-18T20:44:33.112231+0000","last_became_peered":"2023-11-18T20:44:33.112231+0000","last_unstale":"2023-11-18T20:44:34.116011+0000","last_undegraded":"2023-11-18T20:44:34.116011+0000","last_fullsized":"2023-11-18T20:44:34.116011+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:54:54.404042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.706049+0000","last_change":"2023-11-18T20:44:32.708428+0000","last_active":"2023-11-18T20:44:33.706049+0000","last_peered":"2023-11-18T20:44:33.706049+0000","last_clean":"2023-11-18T20:44:33.706049+0000","last_became_active":"2023-11-18T20:44:32.708123+0000","last_became_peered":"2023-11-18T20:44:32.708123+0000","last_unstale":"2023-11-18T20:44:33.706049+0000","last_undegraded":"2023-11-18T20:44:33.706049+0000","last_fullsized":"2023-11-18T20:44:33.706049+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:09:31.551437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115767+0000","last_change":"2023-11-18T20:44:33.115276+0000","last_active":"2023-11-18T20:44:34.115767+0000","last_peered":"2023-11-18T20:44:34.115767+0000","last_clean":"2023-11-18T20:44:34.115767+0000","last_became_active":"2023-11-18T20:44:33.114998+0000","last_became_peered":"2023-11-18T20:44:33.114998+0000","last_unstale":"2023-11-18T20:44:34.115767+0000","last_undegraded":"2023-11-18T20:44:34.115767+0000","last_fullsized":"2023-11-18T20:44:34.115767+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:32:40.407274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.113438+0000","last_change":"2023-11-18T20:44:32.714570+0000","last_active":"2023-11-18T20:44:33.113438+0000","last_peered":"2023-11-18T20:44:33.113438+0000","last_clean":"2023-11-18T20:44:33.113438+0000","last_became_active":"2023-11-18T20:44:32.714302+0000","last_became_peered":"2023-11-18T20:44:32.714302+0000","last_unstale":"2023-11-18T20:44:33.113438+0000","last_undegraded":"2023-11-18T20:44:33.113438+0000","last_fullsized":"2023-11-18T20:44:33.113438+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:35:21.213045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.122521+0000","last_change":"2023-11-18T20:44:33.119839+0000","last_active":"2023-11-18T20:44:33.122521+0000","last_peered":"2023-11-18T20:44:33.122521+0000","last_clean":"2023-11-18T20:44:33.122521+0000","last_became_active":"2023-11-18T20:44:33.119527+0000","last_became_peered":"2023-11-18T20:44:33.119527+0000","last_unstale":"2023-11-18T20:44:33.122521+0000","last_undegraded":"2023-11-18T20:44:33.122521+0000","last_fullsized":"2023-11-18T20:44:33.122521+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:06:33.900376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115816+0000","last_change":"2023-11-18T20:44:32.708787+0000","last_active":"2023-11-18T20:44:34.115816+0000","last_peered":"2023-11-18T20:44:34.115816+0000","last_clean":"2023-11-18T20:44:34.115816+0000","last_became_active":"2023-11-18T20:44:32.708503+0000","last_became_peered":"2023-11-18T20:44:32.708503+0000","last_unstale":"2023-11-18T20:44:34.115816+0000","last_undegraded":"2023-11-18T20:44:34.115816+0000","last_fullsized":"2023-11-18T20:44:34.115816+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:51:45.824233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123970+0000","last_change":"2023-11-18T20:44:33.122118+0000","last_active":"2023-11-18T20:44:33.123970+0000","last_peered":"2023-11-18T20:44:33.123970+0000","last_clean":"2023-11-18T20:44:33.123970+0000","last_became_active":"2023-11-18T20:44:33.121849+0000","last_became_peered":"2023-11-18T20:44:33.121849+0000","last_unstale":"2023-11-18T20:44:33.123970+0000","last_undegraded":"2023-11-18T20:44:33.123970+0000","last_fullsized":"2023-11-18T20:44:33.123970+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:50:07.789515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.114926+0000","last_change":"2023-11-18T20:44:32.713564+0000","last_active":"2023-11-18T20:44:33.114926+0000","last_peered":"2023-11-18T20:44:33.114926+0000","last_clean":"2023-11-18T20:44:33.114926+0000","last_became_active":"2023-11-18T20:44:32.712918+0000","last_became_peered":"2023-11-18T20:44:32.712918+0000","last_unstale":"2023-11-18T20:44:33.114926+0000","last_undegraded":"2023-11-18T20:44:33.114926+0000","last_fullsized":"2023-11-18T20:44:33.114926+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:29:28.392199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123354+0000","last_change":"2023-11-18T20:44:33.120827+0000","last_active":"2023-11-18T20:44:33.123354+0000","last_peered":"2023-11-18T20:44:33.123354+0000","last_clean":"2023-11-18T20:44:33.123354+0000","last_became_active":"2023-11-18T20:44:33.120286+0000","last_became_peered":"2023-11-18T20:44:33.120286+0000","last_unstale":"2023-11-18T20:44:33.123354+0000","last_undegraded":"2023-11-18T20:44:33.123354+0000","last_fullsized":"2023-11-18T20:44:33.123354+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:59:20.084399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.108516+0000","last_change":"2023-11-18T20:44:32.705264+0000","last_active":"2023-11-18T20:44:33.108516+0000","last_peered":"2023-11-18T20:44:33.108516+0000","last_clean":"2023-11-18T20:44:33.108516+0000","last_became_active":"2023-11-18T20:44:32.703632+0000","last_became_peered":"2023-11-18T20:44:32.703632+0000","last_unstale":"2023-11-18T20:44:33.108516+0000","last_undegraded":"2023-11-18T20:44:33.108516+0000","last_fullsized":"2023-11-18T20:44:33.108516+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:13:56.148888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115957+0000","last_change":"2023-11-18T20:44:33.124916+0000","last_active":"2023-11-18T20:44:34.115957+0000","last_peered":"2023-11-18T20:44:34.115957+0000","last_clean":"2023-11-18T20:44:34.115957+0000","last_became_active":"2023-11-18T20:44:33.124606+0000","last_became_peered":"2023-11-18T20:44:33.124606+0000","last_unstale":"2023-11-18T20:44:34.115957+0000","last_undegraded":"2023-11-18T20:44:34.115957+0000","last_fullsized":"2023-11-18T20:44:34.115957+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:10:39.243298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.112418+0000","last_change":"2023-11-18T20:44:32.709411+0000","last_active":"2023-11-18T20:44:34.112418+0000","last_peered":"2023-11-18T20:44:34.112418+0000","last_clean":"2023-11-18T20:44:34.112418+0000","last_became_active":"2023-11-18T20:44:32.709102+0000","last_became_peered":"2023-11-18T20:44:32.709102+0000","last_unstale":"2023-11-18T20:44:34.112418+0000","last_undegraded":"2023-11-18T20:44:34.112418+0000","last_fullsized":"2023-11-18T20:44:34.112418+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:37:53.020182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.127493+0000","last_change":"2023-11-18T20:44:33.123375+0000","last_active":"2023-11-18T20:44:33.127493+0000","last_peered":"2023-11-18T20:44:33.127493+0000","last_clean":"2023-11-18T20:44:33.127493+0000","last_became_active":"2023-11-18T20:44:33.122908+0000","last_became_peered":"2023-11-18T20:44:33.122908+0000","last_unstale":"2023-11-18T20:44:33.127493+0000","last_undegraded":"2023-11-18T20:44:33.127493+0000","last_fullsized":"2023-11-18T20:44:33.127493+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:26:21.163205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.705304+0000","last_change":"2023-11-18T20:44:32.711633+0000","last_active":"2023-11-18T20:44:33.705304+0000","last_peered":"2023-11-18T20:44:33.705304+0000","last_clean":"2023-11-18T20:44:33.705304+0000","last_became_active":"2023-11-18T20:44:32.711323+0000","last_became_peered":"2023-11-18T20:44:32.711323+0000","last_unstale":"2023-11-18T20:44:33.705304+0000","last_undegraded":"2023-11-18T20:44:33.705304+0000","last_fullsized":"2023-11-18T20:44:33.705304+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:14:02.621745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115815+0000","last_change":"2023-11-18T20:44:33.110025+0000","last_active":"2023-11-18T20:44:34.115815+0000","last_peered":"2023-11-18T20:44:34.115815+0000","last_clean":"2023-11-18T20:44:34.115815+0000","last_became_active":"2023-11-18T20:44:33.109462+0000","last_became_peered":"2023-11-18T20:44:33.109462+0000","last_unstale":"2023-11-18T20:44:34.115815+0000","last_undegraded":"2023-11-18T20:44:34.115815+0000","last_fullsized":"2023-11-18T20:44:34.115815+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:37:18.028311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.117028+0000","last_change":"2023-11-18T20:44:32.704380+0000","last_active":"2023-11-18T20:44:33.117028+0000","last_peered":"2023-11-18T20:44:33.117028+0000","last_clean":"2023-11-18T20:44:33.117028+0000","last_became_active":"2023-11-18T20:44:32.703862+0000","last_became_peered":"2023-11-18T20:44:32.703862+0000","last_unstale":"2023-11-18T20:44:33.117028+0000","last_undegraded":"2023-11-18T20:44:33.117028+0000","last_fullsized":"2023-11-18T20:44:33.117028+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:02:31.885575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.125158+0000","last_change":"2023-11-18T20:44:33.122223+0000","last_active":"2023-11-18T20:44:33.125158+0000","last_peered":"2023-11-18T20:44:33.125158+0000","last_clean":"2023-11-18T20:44:33.125158+0000","last_became_active":"2023-11-18T20:44:33.121918+0000","last_became_peered":"2023-11-18T20:44:33.121918+0000","last_unstale":"2023-11-18T20:44:33.125158+0000","last_undegraded":"2023-11-18T20:44:33.125158+0000","last_fullsized":"2023-11-18T20:44:33.125158+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:03:47.903678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.107753+0000","last_change":"2023-11-18T20:44:32.711007+0000","last_active":"2023-11-18T20:44:33.107753+0000","last_peered":"2023-11-18T20:44:33.107753+0000","last_clean":"2023-11-18T20:44:33.107753+0000","last_became_active":"2023-11-18T20:44:32.710532+0000","last_became_peered":"2023-11-18T20:44:32.710532+0000","last_unstale":"2023-11-18T20:44:33.107753+0000","last_undegraded":"2023-11-18T20:44:33.107753+0000","last_fullsized":"2023-11-18T20:44:33.107753+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:04:14.743171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123479+0000","last_change":"2023-11-18T20:44:33.121604+0000","last_active":"2023-11-18T20:44:33.123479+0000","last_peered":"2023-11-18T20:44:33.123479+0000","last_clean":"2023-11-18T20:44:33.123479+0000","last_became_active":"2023-11-18T20:44:33.120807+0000","last_became_peered":"2023-11-18T20:44:33.120807+0000","last_unstale":"2023-11-18T20:44:33.123479+0000","last_undegraded":"2023-11-18T20:44:33.123479+0000","last_fullsized":"2023-11-18T20:44:33.123479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:10:54.408039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.115340+0000","last_change":"2023-11-18T20:44:32.715757+0000","last_active":"2023-11-18T20:44:33.115340+0000","last_peered":"2023-11-18T20:44:33.115340+0000","last_clean":"2023-11-18T20:44:33.115340+0000","last_became_active":"2023-11-18T20:44:32.714748+0000","last_became_peered":"2023-11-18T20:44:32.714748+0000","last_unstale":"2023-11-18T20:44:33.115340+0000","last_undegraded":"2023-11-18T20:44:33.115340+0000","last_fullsized":"2023-11-18T20:44:33.115340+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:12:10.434962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.125153+0000","last_change":"2023-11-18T20:44:33.121799+0000","last_active":"2023-11-18T20:44:33.125153+0000","last_peered":"2023-11-18T20:44:33.125153+0000","last_clean":"2023-11-18T20:44:33.125153+0000","last_became_active":"2023-11-18T20:44:33.120734+0000","last_became_peered":"2023-11-18T20:44:33.120734+0000","last_unstale":"2023-11-18T20:44:33.125153+0000","last_undegraded":"2023-11-18T20:44:33.125153+0000","last_fullsized":"2023-11-18T20:44:33.125153+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:01:59.298618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.115051+0000","last_change":"2023-11-18T20:44:32.713756+0000","last_active":"2023-11-18T20:44:33.115051+0000","last_peered":"2023-11-18T20:44:33.115051+0000","last_clean":"2023-11-18T20:44:33.115051+0000","last_became_active":"2023-11-18T20:44:32.713302+0000","last_became_peered":"2023-11-18T20:44:32.713302+0000","last_unstale":"2023-11-18T20:44:33.115051+0000","last_undegraded":"2023-11-18T20:44:33.115051+0000","last_fullsized":"2023-11-18T20:44:33.115051+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:54:23.282569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.124004+0000","last_change":"2023-11-18T20:44:33.120978+0000","last_active":"2023-11-18T20:44:33.124004+0000","last_peered":"2023-11-18T20:44:33.124004+0000","last_clean":"2023-11-18T20:44:33.124004+0000","last_became_active":"2023-11-18T20:44:33.120596+0000","last_became_peered":"2023-11-18T20:44:33.120596+0000","last_unstale":"2023-11-18T20:44:33.124004+0000","last_undegraded":"2023-11-18T20:44:33.124004+0000","last_fullsized":"2023-11-18T20:44:33.124004+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:32:40.407274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115962+0000","last_change":"2023-11-18T20:44:32.709561+0000","last_active":"2023-11-18T20:44:34.115962+0000","last_peered":"2023-11-18T20:44:34.115962+0000","last_clean":"2023-11-18T20:44:34.115962+0000","last_became_active":"2023-11-18T20:44:32.709278+0000","last_became_peered":"2023-11-18T20:44:32.709278+0000","last_unstale":"2023-11-18T20:44:34.115962+0000","last_undegraded":"2023-11-18T20:44:34.115962+0000","last_fullsized":"2023-11-18T20:44:34.115962+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:04:14.743171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126963+0000","last_change":"2023-11-18T20:44:33.124163+0000","last_active":"2023-11-18T20:44:33.126963+0000","last_peered":"2023-11-18T20:44:33.126963+0000","last_clean":"2023-11-18T20:44:33.126963+0000","last_became_active":"2023-11-18T20:44:33.123694+0000","last_became_peered":"2023-11-18T20:44:33.123694+0000","last_unstale":"2023-11-18T20:44:33.126963+0000","last_undegraded":"2023-11-18T20:44:33.126963+0000","last_fullsized":"2023-11-18T20:44:33.126963+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:03:07.674062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.112812+0000","last_change":"2023-11-18T20:44:32.710940+0000","last_active":"2023-11-18T20:44:33.112812+0000","last_peered":"2023-11-18T20:44:33.112812+0000","last_clean":"2023-11-18T20:44:33.112812+0000","last_became_active":"2023-11-18T20:44:32.710657+0000","last_became_peered":"2023-11-18T20:44:32.710657+0000","last_unstale":"2023-11-18T20:44:33.112812+0000","last_undegraded":"2023-11-18T20:44:33.112812+0000","last_fullsized":"2023-11-18T20:44:33.112812+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:59:19.079526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.121500+0000","last_change":"2023-11-18T20:44:33.118740+0000","last_active":"2023-11-18T20:44:33.121500+0000","last_peered":"2023-11-18T20:44:33.121500+0000","last_clean":"2023-11-18T20:44:33.121500+0000","last_became_active":"2023-11-18T20:44:33.118493+0000","last_became_peered":"2023-11-18T20:44:33.118493+0000","last_unstale":"2023-11-18T20:44:33.121500+0000","last_undegraded":"2023-11-18T20:44:33.121500+0000","last_fullsized":"2023-11-18T20:44:33.121500+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:52:12.610261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.109262+0000","last_change":"2023-11-18T20:44:32.706159+0000","last_active":"2023-11-18T20:44:33.109262+0000","last_peered":"2023-11-18T20:44:33.109262+0000","last_clean":"2023-11-18T20:44:33.109262+0000","last_became_active":"2023-11-18T20:44:32.705892+0000","last_became_peered":"2023-11-18T20:44:32.705892+0000","last_unstale":"2023-11-18T20:44:33.109262+0000","last_undegraded":"2023-11-18T20:44:33.109262+0000","last_fullsized":"2023-11-18T20:44:33.109262+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:02:31.885575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-18T20:44:32.698732+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:44:32.698732+0000","last_peered":"2023-11-18T20:44:32.698732+0000","last_clean":"2023-11-18T20:44:32.698732+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:44:32.698732+0000","last_undegraded":"2023-11-18T20:44:32.698732+0000","last_fullsized":"2023-11-18T20:44:32.698732+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1163264,"data_stored":825331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.49399999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71699999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.50900000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.40000000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.45600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55200000000000005}]},{"osd":4,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72699999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73499999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.79800000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.44900000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79800000000000004,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77600000000000002,"5min":0.876,"15min":0.876},"last":0.57999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":577536,"data_stored":235615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.36599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.68300000000000005}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.82499999999999996}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.71099999999999997}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.80000000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75700000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.624},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53500000000000003}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76600000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":234963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":1.012},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.38600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.94699999999999995}]},{"osd":1,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.002},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.93500000000000005}]},{"osd":2,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.45900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.996}]},{"osd":3,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91900000000000004,"5min":1.046,"15min":1.046},"last":0.81399999999999995}]},{"osd":4,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.98299999999999998}]},{"osd":6,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77000000000000002,"5min":0.875,"15min":0.875},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85899999999999999}]},{"osd":7,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95999999999999996,"5min":1.099,"15min":1.099},"last":0.95999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":235545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54700000000000004}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.39800000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71499999999999997,"5min":0.747,"15min":0.747},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51100000000000001}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.56899999999999995}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73299999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.752,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.58699999999999997}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63600000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.34699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.623}]},{"osd":7,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81699999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.69799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":236597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.441,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33400000000000002}]},{"osd":1,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.317,"15min":0.315},"min":{"1min":0.26100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35699999999999998}]},{"osd":2,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.32900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53100000000000003,"5min":0.995,"15min":0.995},"last":0.38100000000000001}]},{"osd":4,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77700000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.626},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61799999999999999}]},{"osd":6,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77000000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56499999999999995}]},{"osd":7,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81299999999999994,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":581632,"data_stored":236575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33800000000000002,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.754},{"interface":"front","average":{"1min":0.375,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":2.3149999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":2.875},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41699999999999998,"15min":0.42299999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":2.1869999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54500000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.54600000000000004,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":1.9470000000000001}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":1.093},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71999999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":2.3500000000000001}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":2.7029999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.1339999999999999}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.68999999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83899999999999997,"5min":1.859,"15min":1.859},"last":2.3849999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":2.7450000000000001}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.76100000000000001,"5min":1.464,"15min":1.464},"last":2.2810000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.68600000000000005,"15min":0.70099999999999996},"min":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.749,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":2.7829999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":589824,"data_stored":241423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.38400000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.371},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48999999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47799999999999998}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33300000000000002},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51300000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.57099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55400000000000005}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":1.105,"15min":1.105},"last":0.439},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.52500000000000002}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.50900000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72499999999999998,"5min":0.995,"15min":0.995},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76900000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.496}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1159168,"data_stored":824712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33500000000000002,"15min":0.33600000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.374,"15min":0.375},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":0.62,"15min":0.62},"last":0.36099999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.51300000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.37,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.38100000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.438},{"interface":"front","average":{"1min":0.438,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.48299999999999998}]},{"osd":4,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63100000000000001}]},{"osd":5,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81100000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72499999999999998}]},{"osd":7,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.625},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95899999999999996,"5min":1.006,"15min":1.006},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:44:36.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:44:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.848+0000 7f7e6cb3a640 1 Processor -- start 2023-11-18T20:44:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.848+0000 7f7e6cb3a640 1 -- start start 2023-11-18T20:44:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.849+0000 7f7e6cb3a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7e681415f0 0x7f7e68141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.849+0000 7f7e6cb3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e68144a00 0x7f7e6806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.849+0000 7f7e6cb3a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e680b2eb0 con 0x7f7e68144440 2023-11-18T20:44:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.849+0000 7f7e6cb3a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e680b3190 con 0x7f7e681415f0 2023-11-18T20:44:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.849+0000 7f7e6cb3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680b3470 con 0x7f7e68144a00 2023-11-18T20:44:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.849+0000 7f7e66d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7e68144440 0x7f7e680b3f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55414/0 (socket says 172.21.15.60:55414) 2023-11-18T20:44:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.849+0000 7f7e66d9d640 1 -- 172.21.15.60:0/1675097187 learned_addr learned my addr 172.21.15.60:0/1675097187 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:36.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.849+0000 7f7e6659c640 1 --2- 172.21.15.60:0/1675097187 >> v2:172.21.15.100:3300/0 conn(0x7f7e681415f0 0x7f7e68141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.849+0000 7f7e65d9b640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e68144a00 0x7f7e6806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.849+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439507995 0 0) 0x7f7e680b2eb0 con 0x7f7e68144440 2023-11-18T20:44:36.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e54003520 con 0x7f7e68144440 2023-11-18T20:44:36.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 785172270 0 0) 0x7f7e54003520 con 0x7f7e68144440 2023-11-18T20:44:36.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e68144a00 msgr2=0x7f7e6806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:36.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e68144a00 0x7f7e6806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 >> v2:172.21.15.100:3300/0 conn(0x7f7e681415f0 msgr2=0x7f7e68141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:36.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 --2- 172.21.15.60:0/1675097187 >> v2:172.21.15.100:3300/0 conn(0x7f7e681415f0 0x7f7e68141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680b3750 con 0x7f7e68144440 2023-11-18T20:44:36.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f7e580019f0 con 0x7f7e68144440 2023-11-18T20:44:36.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 >> v1:172.21.15.60:6789/0 conn(0x7f7e68144440 legacy=0x7f7e680b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e68144a00 0x7f7e54003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 0x7f7e540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54004d00 0x7f7e54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680b3190 con 0x7f7e54003ef0 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680b3470 con 0x7f7e54004d00 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5400b980 con 0x7f7e68144a00 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e65d9b640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 0x7f7e540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e6659c640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e68144a00 0x7f7e54003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e66d9d640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54004d00 0x7f7e54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e65d9b640 1 -- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e68144a00 msgr2=0x7f7e54003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:36.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e65d9b640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e68144a00 0x7f7e54003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e65d9b640 1 -- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54004d00 msgr2=0x7f7e54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:36.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e65d9b640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e54004d00 0x7f7e54005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.850+0000 7f7e65d9b640 1 -- 172.21.15.60:0/1675097187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54003520 con 0x7f7e54003ef0 2023-11-18T20:44:36.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.851+0000 7f7e65d9b640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 0x7f7e540047c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009ac0 tx=0x7f7e540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ab470c2ae1681c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:36.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.851+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5002c6b0 con 0x7f7e54003ef0 2023-11-18T20:44:36.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.851+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5002c810 con 0x7f7e54003ef0 2023-11-18T20:44:36.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.851+0000 7f7e6559a640 1 -- 172.21.15.60:0/1675097187 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5002cba0 con 0x7f7e54003ef0 2023-11-18T20:44:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.852+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/1675097187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 msgr2=0x7f7e540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.852+0000 7f7e6cb3a640 1 --2- 172.21.15.60:0/1675097187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 0x7f7e540047c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009ac0 tx=0x7f7e540057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.852+0000 7f7e66d9d640 1 -- 172.21.15.60:0/1675097187 reap_dead start 2023-11-18T20:44:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.852+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/1675097187 shutdown_connections 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.852+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/1675097187 >> 172.21.15.60:0/1675097187 conn(0x7f7e68070e70 msgr2=0x7f7e68071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.852+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/1675097187 shutdown_connections 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.852+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/1675097187 wait complete. 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.853+0000 7f7e6cb3a640 1 Processor -- start 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.853+0000 7f7e6cb3a640 1 -- start start 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6cb3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 0x7f7e6813ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6cb3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e681576f0 0x7f7e6815c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6cb3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e6815c9b0 0x7f7e6806d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6cb3a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681572d0 con 0x7f7e54003ef0 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6cb3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6815ce30 con 0x7f7e681576f0 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6cb3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68140440 con 0x7f7e6815c9b0 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 0x7f7e6813ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 0x7f7e6813ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49930/0 (socket says 172.21.15.60:49930) 2023-11-18T20:44:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6659c640 1 -- 172.21.15.60:0/2389406958 learned_addr learned my addr 172.21.15.60:0/2389406958 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e65d9b640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e681576f0 0x7f7e6815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e66d9d640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e6815c9b0 0x7f7e6806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6659c640 1 -- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e6815c9b0 msgr2=0x7f7e6806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6659c640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e6815c9b0 0x7f7e6806d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6659c640 1 -- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e681576f0 msgr2=0x7f7e6815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6659c640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e681576f0 0x7f7e6815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6659c640 1 -- 172.21.15.60:0/2389406958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6806df50 con 0x7f7e54003ef0 2023-11-18T20:44:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e65d9b640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e681576f0 0x7f7e6815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:36.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e66d9d640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e6815c9b0 0x7f7e6806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6659c640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 0x7f7e6813ff00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7e60011a70 tx=0x7f7e60011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e5effd640 1 -- 172.21.15.60:0/2389406958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60009680 con 0x7f7e54003ef0 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e5effd640 1 -- 172.21.15.60:0/2389406958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6000fb50 con 0x7f7e54003ef0 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e5effd640 1 -- 172.21.15.60:0/2389406958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6000fee0 con 0x7f7e54003ef0 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.854+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/2389406958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e68156c10 con 0x7f7e54003ef0 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.855+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/2389406958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e68157b70 con 0x7f7e54003ef0 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.855+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/2389406958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e68157540 con 0x7f7e54003ef0 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.856+0000 7f7e5effd640 1 -- 172.21.15.60:0/2389406958 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e60014440 con 0x7f7e54003ef0 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.856+0000 7f7e5effd640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7e4806ae70 0x7f7e4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.856+0000 7f7e5effd640 1 -- 172.21.15.60:0/2389406958 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e60089fb0 con 0x7f7e54003ef0 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.856+0000 7f7e65d9b640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7e4806ae70 0x7f7e4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:36.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.857+0000 7f7e65d9b640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7e4806ae70 0x7f7e4806d330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7e50036000 tx=0x7f7e50030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:36.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:36.860+0000 7f7e5effd640 1 -- 172.21.15.60:0/2389406958 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e60054fc0 con 0x7f7e54003ef0 2023-11-18T20:44:37.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.033+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/2389406958 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e68142340 con 0x7f7e4806ae70 2023-11-18T20:44:37.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.041+0000 7f7e5effd640 1 -- 172.21.15.60:0/2389406958 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289672 (secure 0 0 0) 0x7f7e50004140 con 0x7f7e4806ae70 2023-11-18T20:44:37.044 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:37.046 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:44:37.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.043+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7e4806ae70 msgr2=0x7f7e4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.043+0000 7f7e6cb3a640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7e4806ae70 0x7f7e4806d330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7e50036000 tx=0x7f7e50030000 comp rx=0 tx=0).stop 2023-11-18T20:44:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.043+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/2389406958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 msgr2=0x7f7e6813ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.043+0000 7f7e6cb3a640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 0x7f7e6813ff00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7e60011a70 tx=0x7f7e60011f30 comp rx=0 tx=0).stop 2023-11-18T20:44:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.043+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/2389406958 shutdown_connections 2023-11-18T20:44:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.043+0000 7f7e6cb3a640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7e4806ae70 0x7f7e4806d330 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.043+0000 7f7e6cb3a640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e6815c9b0 0x7f7e6806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.043+0000 7f7e6cb3a640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e681576f0 0x7f7e6815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.043+0000 7f7e6cb3a640 1 --2- 172.21.15.60:0/2389406958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e54003ef0 0x7f7e6813ff00 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.043+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/2389406958 >> 172.21.15.60:0/2389406958 conn(0x7f7e68070e70 msgr2=0x7f7e680b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.044+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/2389406958 shutdown_connections 2023-11-18T20:44:37.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.044+0000 7f7e6cb3a640 1 -- 172.21.15.60:0/2389406958 wait complete. 2023-11-18T20:44:37.055 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2023-11-18T20:44:35.464651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238492,"kb_used_data":5664,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736228,"statfs":{"total":773094113280,"available":772849897472,"internally_reserved":0,"allocated":5799936,"data_stored":3070761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001080"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115947+0000","last_change":"2023-11-18T20:44:33.116942+0000","last_active":"2023-11-18T20:44:34.115947+0000","last_peered":"2023-11-18T20:44:34.115947+0000","last_clean":"2023-11-18T20:44:34.115947+0000","last_became_active":"2023-11-18T20:44:33.116676+0000","last_became_peered":"2023-11-18T20:44:33.116676+0000","last_unstale":"2023-11-18T20:44:34.115947+0000","last_undegraded":"2023-11-18T20:44:34.115947+0000","last_fullsized":"2023-11-18T20:44:34.115947+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:02:32.890448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.127964+0000","last_change":"2023-11-18T20:44:33.125248+0000","last_active":"2023-11-18T20:44:33.127964+0000","last_peered":"2023-11-18T20:44:33.127964+0000","last_clean":"2023-11-18T20:44:33.127964+0000","last_became_active":"2023-11-18T20:44:33.124831+0000","last_became_peered":"2023-11-18T20:44:33.124831+0000","last_unstale":"2023-11-18T20:44:33.127964+0000","last_undegraded":"2023-11-18T20:44:33.127964+0000","last_fullsized":"2023-11-18T20:44:33.127964+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:32:40.407274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123301+0000","last_change":"2023-11-18T20:44:33.121223+0000","last_active":"2023-11-18T20:44:33.123301+0000","last_peered":"2023-11-18T20:44:33.123301+0000","last_clean":"2023-11-18T20:44:33.123301+0000","last_became_active":"2023-11-18T20:44:33.118789+0000","last_became_peered":"2023-11-18T20:44:33.118789+0000","last_unstale":"2023-11-18T20:44:33.123301+0000","last_undegraded":"2023-11-18T20:44:33.123301+0000","last_fullsized":"2023-11-18T20:44:33.123301+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:06:07.786307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.128020+0000","last_change":"2023-11-18T20:44:33.125771+0000","last_active":"2023-11-18T20:44:33.128020+0000","last_peered":"2023-11-18T20:44:33.128020+0000","last_clean":"2023-11-18T20:44:33.128020+0000","last_became_active":"2023-11-18T20:44:33.125256+0000","last_became_peered":"2023-11-18T20:44:33.125256+0000","last_unstale":"2023-11-18T20:44:33.128020+0000","last_undegraded":"2023-11-18T20:44:33.128020+0000","last_fullsized":"2023-11-18T20:44:33.128020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:41:29.137921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116156+0000","last_change":"2023-11-18T20:44:33.118231+0000","last_active":"2023-11-18T20:44:34.116156+0000","last_peered":"2023-11-18T20:44:34.116156+0000","last_clean":"2023-11-18T20:44:34.116156+0000","last_became_active":"2023-11-18T20:44:33.117934+0000","last_became_peered":"2023-11-18T20:44:33.117934+0000","last_unstale":"2023-11-18T20:44:34.116156+0000","last_undegraded":"2023-11-18T20:44:34.116156+0000","last_fullsized":"2023-11-18T20:44:34.116156+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:51:46.829106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115777+0000","last_change":"2023-11-18T20:44:33.115629+0000","last_active":"2023-11-18T20:44:34.115777+0000","last_peered":"2023-11-18T20:44:34.115777+0000","last_clean":"2023-11-18T20:44:34.115777+0000","last_became_active":"2023-11-18T20:44:33.115359+0000","last_became_peered":"2023-11-18T20:44:33.115359+0000","last_unstale":"2023-11-18T20:44:34.115777+0000","last_undegraded":"2023-11-18T20:44:34.115777+0000","last_fullsized":"2023-11-18T20:44:34.115777+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:33:54.326294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123241+0000","last_change":"2023-11-18T20:44:33.120383+0000","last_active":"2023-11-18T20:44:33.123241+0000","last_peered":"2023-11-18T20:44:33.123241+0000","last_clean":"2023-11-18T20:44:33.123241+0000","last_became_active":"2023-11-18T20:44:33.120107+0000","last_became_peered":"2023-11-18T20:44:33.120107+0000","last_unstale":"2023-11-18T20:44:33.123241+0000","last_undegraded":"2023-11-18T20:44:33.123241+0000","last_fullsized":"2023-11-18T20:44:33.123241+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:06:33.900376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123243+0000","last_change":"2023-11-18T20:44:33.118644+0000","last_active":"2023-11-18T20:44:33.123243+0000","last_peered":"2023-11-18T20:44:33.123243+0000","last_clean":"2023-11-18T20:44:33.123243+0000","last_became_active":"2023-11-18T20:44:33.118363+0000","last_became_peered":"2023-11-18T20:44:33.118363+0000","last_unstale":"2023-11-18T20:44:33.123243+0000","last_undegraded":"2023-11-18T20:44:33.123243+0000","last_fullsized":"2023-11-18T20:44:33.123243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:10:39.243298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126979+0000","last_change":"2023-11-18T20:44:33.124673+0000","last_active":"2023-11-18T20:44:33.126979+0000","last_peered":"2023-11-18T20:44:33.126979+0000","last_clean":"2023-11-18T20:44:33.126979+0000","last_became_active":"2023-11-18T20:44:33.124432+0000","last_became_peered":"2023-11-18T20:44:33.124432+0000","last_unstale":"2023-11-18T20:44:33.126979+0000","last_undegraded":"2023-11-18T20:44:33.126979+0000","last_fullsized":"2023-11-18T20:44:33.126979+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:54:29.346693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116004+0000","last_change":"2023-11-18T20:44:33.116176+0000","last_active":"2023-11-18T20:44:34.116004+0000","last_peered":"2023-11-18T20:44:34.116004+0000","last_clean":"2023-11-18T20:44:34.116004+0000","last_became_active":"2023-11-18T20:44:33.115899+0000","last_became_peered":"2023-11-18T20:44:33.115899+0000","last_unstale":"2023-11-18T20:44:34.116004+0000","last_undegraded":"2023-11-18T20:44:34.116004+0000","last_fullsized":"2023-11-18T20:44:34.116004+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:06:48.969899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.112770+0000","last_change":"2023-11-18T20:44:33.123459+0000","last_active":"2023-11-18T20:44:34.112770+0000","last_peered":"2023-11-18T20:44:34.112770+0000","last_clean":"2023-11-18T20:44:34.112770+0000","last_became_active":"2023-11-18T20:44:33.122722+0000","last_became_peered":"2023-11-18T20:44:33.122722+0000","last_unstale":"2023-11-18T20:44:34.112770+0000","last_undegraded":"2023-11-18T20:44:34.112770+0000","last_fullsized":"2023-11-18T20:44:34.112770+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:24:45.422200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.124150+0000","last_change":"2023-11-18T20:44:33.119993+0000","last_active":"2023-11-18T20:44:33.124150+0000","last_peered":"2023-11-18T20:44:33.124150+0000","last_clean":"2023-11-18T20:44:33.124150+0000","last_became_active":"2023-11-18T20:44:33.119729+0000","last_became_peered":"2023-11-18T20:44:33.119729+0000","last_unstale":"2023-11-18T20:44:33.124150+0000","last_undegraded":"2023-11-18T20:44:33.124150+0000","last_fullsized":"2023-11-18T20:44:33.124150+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:06:48.969899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115725+0000","last_change":"2023-11-18T20:44:33.109703+0000","last_active":"2023-11-18T20:44:34.115725+0000","last_peered":"2023-11-18T20:44:34.115725+0000","last_clean":"2023-11-18T20:44:34.115725+0000","last_became_active":"2023-11-18T20:44:33.109428+0000","last_became_peered":"2023-11-18T20:44:33.109428+0000","last_unstale":"2023-11-18T20:44:34.115725+0000","last_undegraded":"2023-11-18T20:44:34.115725+0000","last_fullsized":"2023-11-18T20:44:34.115725+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:20:37.793311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123324+0000","last_change":"2023-11-18T20:44:33.120658+0000","last_active":"2023-11-18T20:44:33.123324+0000","last_peered":"2023-11-18T20:44:33.123324+0000","last_clean":"2023-11-18T20:44:33.123324+0000","last_became_active":"2023-11-18T20:44:33.118985+0000","last_became_peered":"2023-11-18T20:44:33.118985+0000","last_unstale":"2023-11-18T20:44:33.123324+0000","last_undegraded":"2023-11-18T20:44:33.123324+0000","last_fullsized":"2023-11-18T20:44:33.123324+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:01:59.298618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.121579+0000","last_change":"2023-11-18T20:44:33.120176+0000","last_active":"2023-11-18T20:44:33.121579+0000","last_peered":"2023-11-18T20:44:33.121579+0000","last_clean":"2023-11-18T20:44:33.121579+0000","last_became_active":"2023-11-18T20:44:33.119922+0000","last_became_peered":"2023-11-18T20:44:33.119922+0000","last_unstale":"2023-11-18T20:44:33.121579+0000","last_undegraded":"2023-11-18T20:44:33.121579+0000","last_fullsized":"2023-11-18T20:44:33.121579+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:54:20.419244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115646+0000","last_change":"2023-11-18T20:44:33.110026+0000","last_active":"2023-11-18T20:44:34.115646+0000","last_peered":"2023-11-18T20:44:34.115646+0000","last_clean":"2023-11-18T20:44:34.115646+0000","last_became_active":"2023-11-18T20:44:33.109743+0000","last_became_peered":"2023-11-18T20:44:33.109743+0000","last_unstale":"2023-11-18T20:44:34.115646+0000","last_undegraded":"2023-11-18T20:44:34.115646+0000","last_fullsized":"2023-11-18T20:44:34.115646+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:26:14.401043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.117717+0000","last_change":"2023-11-18T20:44:33.115926+0000","last_active":"2023-11-18T20:44:33.117717+0000","last_peered":"2023-11-18T20:44:33.117717+0000","last_clean":"2023-11-18T20:44:33.117717+0000","last_became_active":"2023-11-18T20:44:33.115544+0000","last_became_peered":"2023-11-18T20:44:33.115544+0000","last_unstale":"2023-11-18T20:44:33.117717+0000","last_undegraded":"2023-11-18T20:44:33.117717+0000","last_fullsized":"2023-11-18T20:44:33.117717+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:37:18.028311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.127543+0000","last_change":"2023-11-18T20:44:33.124555+0000","last_active":"2023-11-18T20:44:33.127543+0000","last_peered":"2023-11-18T20:44:33.127543+0000","last_clean":"2023-11-18T20:44:33.127543+0000","last_became_active":"2023-11-18T20:44:33.124294+0000","last_became_peered":"2023-11-18T20:44:33.124294+0000","last_unstale":"2023-11-18T20:44:33.127543+0000","last_undegraded":"2023-11-18T20:44:33.127543+0000","last_fullsized":"2023-11-18T20:44:33.127543+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:26:14.401043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.124003+0000","last_change":"2023-11-18T20:44:33.122988+0000","last_active":"2023-11-18T20:44:33.124003+0000","last_peered":"2023-11-18T20:44:33.124003+0000","last_clean":"2023-11-18T20:44:33.124003+0000","last_became_active":"2023-11-18T20:44:33.122737+0000","last_became_peered":"2023-11-18T20:44:33.122737+0000","last_unstale":"2023-11-18T20:44:33.124003+0000","last_undegraded":"2023-11-18T20:44:33.124003+0000","last_fullsized":"2023-11-18T20:44:33.124003+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:41:29.137921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123545+0000","last_change":"2023-11-18T20:44:33.121172+0000","last_active":"2023-11-18T20:44:33.123545+0000","last_peered":"2023-11-18T20:44:33.123545+0000","last_clean":"2023-11-18T20:44:33.123545+0000","last_became_active":"2023-11-18T20:44:33.120870+0000","last_became_peered":"2023-11-18T20:44:33.120870+0000","last_unstale":"2023-11-18T20:44:33.123545+0000","last_undegraded":"2023-11-18T20:44:33.123545+0000","last_fullsized":"2023-11-18T20:44:33.123545+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:12:41.474148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.120609+0000","last_change":"2023-11-18T20:44:33.118345+0000","last_active":"2023-11-18T20:44:33.120609+0000","last_peered":"2023-11-18T20:44:33.120609+0000","last_clean":"2023-11-18T20:44:33.120609+0000","last_became_active":"2023-11-18T20:44:33.117979+0000","last_became_peered":"2023-11-18T20:44:33.117979+0000","last_unstale":"2023-11-18T20:44:33.120609+0000","last_undegraded":"2023-11-18T20:44:33.120609+0000","last_fullsized":"2023-11-18T20:44:33.120609+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:14:25.168203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123191+0000","last_change":"2023-11-18T20:44:33.118614+0000","last_active":"2023-11-18T20:44:33.123191+0000","last_peered":"2023-11-18T20:44:33.123191+0000","last_clean":"2023-11-18T20:44:33.123191+0000","last_became_active":"2023-11-18T20:44:33.118331+0000","last_became_peered":"2023-11-18T20:44:33.118331+0000","last_unstale":"2023-11-18T20:44:33.123191+0000","last_undegraded":"2023-11-18T20:44:33.123191+0000","last_fullsized":"2023-11-18T20:44:33.123191+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:37:18.028311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.112525+0000","last_change":"2023-11-18T20:44:33.118064+0000","last_active":"2023-11-18T20:44:34.112525+0000","last_peered":"2023-11-18T20:44:34.112525+0000","last_clean":"2023-11-18T20:44:34.112525+0000","last_became_active":"2023-11-18T20:44:33.117769+0000","last_became_peered":"2023-11-18T20:44:33.117769+0000","last_unstale":"2023-11-18T20:44:34.112525+0000","last_undegraded":"2023-11-18T20:44:34.112525+0000","last_fullsized":"2023-11-18T20:44:34.112525+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:27:47.528034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123093+0000","last_change":"2023-11-18T20:44:33.117627+0000","last_active":"2023-11-18T20:44:33.123093+0000","last_peered":"2023-11-18T20:44:33.123093+0000","last_clean":"2023-11-18T20:44:33.123093+0000","last_became_active":"2023-11-18T20:44:33.117366+0000","last_became_peered":"2023-11-18T20:44:33.117366+0000","last_unstale":"2023-11-18T20:44:33.123093+0000","last_undegraded":"2023-11-18T20:44:33.123093+0000","last_fullsized":"2023-11-18T20:44:33.123093+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:44:27.625798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126479+0000","last_change":"2023-11-18T20:44:33.125109+0000","last_active":"2023-11-18T20:44:33.126479+0000","last_peered":"2023-11-18T20:44:33.126479+0000","last_clean":"2023-11-18T20:44:33.126479+0000","last_became_active":"2023-11-18T20:44:33.124820+0000","last_became_peered":"2023-11-18T20:44:33.124820+0000","last_unstale":"2023-11-18T20:44:33.126479+0000","last_undegraded":"2023-11-18T20:44:33.126479+0000","last_fullsized":"2023-11-18T20:44:33.126479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:33:54.326294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126689+0000","last_change":"2023-11-18T20:44:33.123975+0000","last_active":"2023-11-18T20:44:33.126689+0000","last_peered":"2023-11-18T20:44:33.126689+0000","last_clean":"2023-11-18T20:44:33.126689+0000","last_became_active":"2023-11-18T20:44:33.123323+0000","last_became_peered":"2023-11-18T20:44:33.123323+0000","last_unstale":"2023-11-18T20:44:33.126689+0000","last_undegraded":"2023-11-18T20:44:33.126689+0000","last_fullsized":"2023-11-18T20:44:33.126689+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:57:38.502768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116020+0000","last_change":"2023-11-18T20:44:33.118477+0000","last_active":"2023-11-18T20:44:34.116020+0000","last_peered":"2023-11-18T20:44:34.116020+0000","last_clean":"2023-11-18T20:44:34.116020+0000","last_became_active":"2023-11-18T20:44:33.118221+0000","last_became_peered":"2023-11-18T20:44:33.118221+0000","last_unstale":"2023-11-18T20:44:34.116020+0000","last_undegraded":"2023-11-18T20:44:34.116020+0000","last_fullsized":"2023-11-18T20:44:34.116020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:20:57.880034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.121044+0000","last_change":"2023-11-18T20:44:33.118385+0000","last_active":"2023-11-18T20:44:33.121044+0000","last_peered":"2023-11-18T20:44:33.121044+0000","last_clean":"2023-11-18T20:44:33.121044+0000","last_became_active":"2023-11-18T20:44:33.118123+0000","last_became_peered":"2023-11-18T20:44:33.118123+0000","last_unstale":"2023-11-18T20:44:33.121044+0000","last_undegraded":"2023-11-18T20:44:33.121044+0000","last_fullsized":"2023-11-18T20:44:33.121044+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:21:17.602054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123026+0000","last_change":"2023-11-18T20:44:33.117894+0000","last_active":"2023-11-18T20:44:33.123026+0000","last_peered":"2023-11-18T20:44:33.123026+0000","last_clean":"2023-11-18T20:44:33.123026+0000","last_became_active":"2023-11-18T20:44:33.117402+0000","last_became_peered":"2023-11-18T20:44:33.117402+0000","last_unstale":"2023-11-18T20:44:33.123026+0000","last_undegraded":"2023-11-18T20:44:33.123026+0000","last_fullsized":"2023-11-18T20:44:33.123026+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:51:46.829106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123457+0000","last_change":"2023-11-18T20:44:33.122306+0000","last_active":"2023-11-18T20:44:33.123457+0000","last_peered":"2023-11-18T20:44:33.123457+0000","last_clean":"2023-11-18T20:44:33.123457+0000","last_became_active":"2023-11-18T20:44:33.121421+0000","last_became_peered":"2023-11-18T20:44:33.121421+0000","last_unstale":"2023-11-18T20:44:33.123457+0000","last_undegraded":"2023-11-18T20:44:33.123457+0000","last_fullsized":"2023-11-18T20:44:33.123457+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:06:07.786307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123223+0000","last_change":"2023-11-18T20:44:33.120333+0000","last_active":"2023-11-18T20:44:33.123223+0000","last_peered":"2023-11-18T20:44:33.123223+0000","last_clean":"2023-11-18T20:44:33.123223+0000","last_became_active":"2023-11-18T20:44:33.119276+0000","last_became_peered":"2023-11-18T20:44:33.119276+0000","last_unstale":"2023-11-18T20:44:33.123223+0000","last_undegraded":"2023-11-18T20:44:33.123223+0000","last_fullsized":"2023-11-18T20:44:33.123223+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:11:39.561038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.127515+0000","last_change":"2023-11-18T20:44:33.123413+0000","last_active":"2023-11-18T20:44:33.127515+0000","last_peered":"2023-11-18T20:44:33.127515+0000","last_clean":"2023-11-18T20:44:33.127515+0000","last_became_active":"2023-11-18T20:44:33.123097+0000","last_became_peered":"2023-11-18T20:44:33.123097+0000","last_unstale":"2023-11-18T20:44:33.127515+0000","last_undegraded":"2023-11-18T20:44:33.127515+0000","last_fullsized":"2023-11-18T20:44:33.127515+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:54:54.404042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.120562+0000","last_change":"2023-11-18T20:44:33.119113+0000","last_active":"2023-11-18T20:44:33.120562+0000","last_peered":"2023-11-18T20:44:33.120562+0000","last_clean":"2023-11-18T20:44:33.120562+0000","last_became_active":"2023-11-18T20:44:33.117333+0000","last_became_peered":"2023-11-18T20:44:33.117333+0000","last_unstale":"2023-11-18T20:44:33.120562+0000","last_undegraded":"2023-11-18T20:44:33.120562+0000","last_fullsized":"2023-11-18T20:44:33.120562+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:13:57.153760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.708281+0000","last_change":"2023-11-18T20:44:32.709740+0000","last_active":"2023-11-18T20:44:33.708281+0000","last_peered":"2023-11-18T20:44:33.708281+0000","last_clean":"2023-11-18T20:44:33.708281+0000","last_became_active":"2023-11-18T20:44:32.709460+0000","last_became_peered":"2023-11-18T20:44:32.709460+0000","last_unstale":"2023-11-18T20:44:33.708281+0000","last_undegraded":"2023-11-18T20:44:33.708281+0000","last_fullsized":"2023-11-18T20:44:33.708281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:01:11.915156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.118628+0000","last_change":"2023-11-18T20:44:33.117519+0000","last_active":"2023-11-18T20:44:33.118628+0000","last_peered":"2023-11-18T20:44:33.118628+0000","last_clean":"2023-11-18T20:44:33.118628+0000","last_became_active":"2023-11-18T20:44:33.117261+0000","last_became_peered":"2023-11-18T20:44:33.117261+0000","last_unstale":"2023-11-18T20:44:33.118628+0000","last_undegraded":"2023-11-18T20:44:33.118628+0000","last_fullsized":"2023-11-18T20:44:33.118628+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:11:39.561038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115941+0000","last_change":"2023-11-18T20:44:32.714052+0000","last_active":"2023-11-18T20:44:34.115941+0000","last_peered":"2023-11-18T20:44:34.115941+0000","last_clean":"2023-11-18T20:44:34.115941+0000","last_became_active":"2023-11-18T20:44:32.713569+0000","last_became_peered":"2023-11-18T20:44:32.713569+0000","last_unstale":"2023-11-18T20:44:34.115941+0000","last_undegraded":"2023-11-18T20:44:34.115941+0000","last_fullsized":"2023-11-18T20:44:34.115941+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:54:19.414371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.124935+0000","last_change":"2023-11-18T20:44:33.123311+0000","last_active":"2023-11-18T20:44:33.124935+0000","last_peered":"2023-11-18T20:44:33.124935+0000","last_clean":"2023-11-18T20:44:33.124935+0000","last_became_active":"2023-11-18T20:44:33.123039+0000","last_became_peered":"2023-11-18T20:44:33.123039+0000","last_unstale":"2023-11-18T20:44:33.124935+0000","last_undegraded":"2023-11-18T20:44:33.124935+0000","last_fullsized":"2023-11-18T20:44:33.124935+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:50:01.856323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115987+0000","last_change":"2023-11-18T20:44:32.708656+0000","last_active":"2023-11-18T20:44:34.115987+0000","last_peered":"2023-11-18T20:44:34.115987+0000","last_clean":"2023-11-18T20:44:34.115987+0000","last_became_active":"2023-11-18T20:44:32.708373+0000","last_became_peered":"2023-11-18T20:44:32.708373+0000","last_unstale":"2023-11-18T20:44:34.115987+0000","last_undegraded":"2023-11-18T20:44:34.115987+0000","last_fullsized":"2023-11-18T20:44:34.115987+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:59:19.079526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123573+0000","last_change":"2023-11-18T20:44:33.119348+0000","last_active":"2023-11-18T20:44:33.123573+0000","last_peered":"2023-11-18T20:44:33.123573+0000","last_clean":"2023-11-18T20:44:33.123573+0000","last_became_active":"2023-11-18T20:44:33.119103+0000","last_became_peered":"2023-11-18T20:44:33.119103+0000","last_unstale":"2023-11-18T20:44:33.123573+0000","last_undegraded":"2023-11-18T20:44:33.123573+0000","last_fullsized":"2023-11-18T20:44:33.123573+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:20:57.880034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.704535+0000","last_change":"2023-11-18T20:44:32.711272+0000","last_active":"2023-11-18T20:44:33.704535+0000","last_peered":"2023-11-18T20:44:33.704535+0000","last_clean":"2023-11-18T20:44:33.704535+0000","last_became_active":"2023-11-18T20:44:32.710967+0000","last_became_peered":"2023-11-18T20:44:32.710967+0000","last_unstale":"2023-11-18T20:44:33.704535+0000","last_undegraded":"2023-11-18T20:44:33.704535+0000","last_fullsized":"2023-11-18T20:44:33.704535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:54:19.414371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116360+0000","last_change":"2023-11-18T20:44:33.112210+0000","last_active":"2023-11-18T20:44:34.116360+0000","last_peered":"2023-11-18T20:44:34.116360+0000","last_clean":"2023-11-18T20:44:34.116360+0000","last_became_active":"2023-11-18T20:44:33.111901+0000","last_became_peered":"2023-11-18T20:44:33.111901+0000","last_unstale":"2023-11-18T20:44:34.116360+0000","last_undegraded":"2023-11-18T20:44:34.116360+0000","last_fullsized":"2023-11-18T20:44:34.116360+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:59:20.084399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.706398+0000","last_change":"2023-11-18T20:44:32.707864+0000","last_active":"2023-11-18T20:44:33.706398+0000","last_peered":"2023-11-18T20:44:33.706398+0000","last_clean":"2023-11-18T20:44:33.706398+0000","last_became_active":"2023-11-18T20:44:32.706893+0000","last_became_peered":"2023-11-18T20:44:32.706893+0000","last_unstale":"2023-11-18T20:44:33.706398+0000","last_undegraded":"2023-11-18T20:44:33.706398+0000","last_fullsized":"2023-11-18T20:44:33.706398+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:54:19.414371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123998+0000","last_change":"2023-11-18T20:44:33.122677+0000","last_active":"2023-11-18T20:44:33.123998+0000","last_peered":"2023-11-18T20:44:33.123998+0000","last_clean":"2023-11-18T20:44:33.123998+0000","last_became_active":"2023-11-18T20:44:33.121136+0000","last_became_peered":"2023-11-18T20:44:33.121136+0000","last_unstale":"2023-11-18T20:44:33.123998+0000","last_undegraded":"2023-11-18T20:44:33.123998+0000","last_fullsized":"2023-11-18T20:44:33.123998+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:20:37.793311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116015+0000","last_change":"2023-11-18T20:44:32.706595+0000","last_active":"2023-11-18T20:44:34.116015+0000","last_peered":"2023-11-18T20:44:34.116015+0000","last_clean":"2023-11-18T20:44:34.116015+0000","last_became_active":"2023-11-18T20:44:32.706211+0000","last_became_peered":"2023-11-18T20:44:32.706211+0000","last_unstale":"2023-11-18T20:44:34.116015+0000","last_undegraded":"2023-11-18T20:44:34.116015+0000","last_fullsized":"2023-11-18T20:44:34.116015+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:01:58.293745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126709+0000","last_change":"2023-11-18T20:44:33.124122+0000","last_active":"2023-11-18T20:44:33.126709+0000","last_peered":"2023-11-18T20:44:33.126709+0000","last_clean":"2023-11-18T20:44:33.126709+0000","last_became_active":"2023-11-18T20:44:33.123675+0000","last_became_peered":"2023-11-18T20:44:33.123675+0000","last_unstale":"2023-11-18T20:44:33.126709+0000","last_undegraded":"2023-11-18T20:44:33.126709+0000","last_fullsized":"2023-11-18T20:44:33.126709+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:26:32.984520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.112124+0000","last_change":"2023-11-18T20:44:32.705013+0000","last_active":"2023-11-18T20:44:33.112124+0000","last_peered":"2023-11-18T20:44:33.112124+0000","last_clean":"2023-11-18T20:44:33.112124+0000","last_became_active":"2023-11-18T20:44:32.704734+0000","last_became_peered":"2023-11-18T20:44:32.704734+0000","last_unstale":"2023-11-18T20:44:33.112124+0000","last_undegraded":"2023-11-18T20:44:33.112124+0000","last_fullsized":"2023-11-18T20:44:33.112124+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:02:56.669743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.128013+0000","last_change":"2023-11-18T20:44:33.125639+0000","last_active":"2023-11-18T20:44:33.128013+0000","last_peered":"2023-11-18T20:44:33.128013+0000","last_clean":"2023-11-18T20:44:33.128013+0000","last_became_active":"2023-11-18T20:44:33.125309+0000","last_became_peered":"2023-11-18T20:44:33.125309+0000","last_unstale":"2023-11-18T20:44:33.128013+0000","last_undegraded":"2023-11-18T20:44:33.128013+0000","last_fullsized":"2023-11-18T20:44:33.128013+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:37:18.028311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.107823+0000","last_change":"2023-11-18T20:44:32.707569+0000","last_active":"2023-11-18T20:44:33.107823+0000","last_peered":"2023-11-18T20:44:33.107823+0000","last_clean":"2023-11-18T20:44:33.107823+0000","last_became_active":"2023-11-18T20:44:32.707269+0000","last_became_peered":"2023-11-18T20:44:32.707269+0000","last_unstale":"2023-11-18T20:44:33.107823+0000","last_undegraded":"2023-11-18T20:44:33.107823+0000","last_fullsized":"2023-11-18T20:44:33.107823+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:26:20.158332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116082+0000","last_change":"2023-11-18T20:44:33.119997+0000","last_active":"2023-11-18T20:44:34.116082+0000","last_peered":"2023-11-18T20:44:34.116082+0000","last_clean":"2023-11-18T20:44:34.116082+0000","last_became_active":"2023-11-18T20:44:33.119162+0000","last_became_peered":"2023-11-18T20:44:33.119162+0000","last_unstale":"2023-11-18T20:44:34.116082+0000","last_undegraded":"2023-11-18T20:44:34.116082+0000","last_fullsized":"2023-11-18T20:44:34.116082+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:06:33.900376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.117047+0000","last_change":"2023-11-18T20:44:32.707847+0000","last_active":"2023-11-18T20:44:33.117047+0000","last_peered":"2023-11-18T20:44:33.117047+0000","last_clean":"2023-11-18T20:44:33.117047+0000","last_became_active":"2023-11-18T20:44:32.707528+0000","last_became_peered":"2023-11-18T20:44:32.707528+0000","last_unstale":"2023-11-18T20:44:33.117047+0000","last_undegraded":"2023-11-18T20:44:33.117047+0000","last_fullsized":"2023-11-18T20:44:33.117047+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:10:38.238426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.122490+0000","last_change":"2023-11-18T20:44:33.120107+0000","last_active":"2023-11-18T20:44:33.122490+0000","last_peered":"2023-11-18T20:44:33.122490+0000","last_clean":"2023-11-18T20:44:33.122490+0000","last_became_active":"2023-11-18T20:44:33.119458+0000","last_became_peered":"2023-11-18T20:44:33.119458+0000","last_unstale":"2023-11-18T20:44:33.122490+0000","last_undegraded":"2023-11-18T20:44:33.122490+0000","last_fullsized":"2023-11-18T20:44:33.122490+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:51:46.829106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116296+0000","last_change":"2023-11-18T20:44:32.706237+0000","last_active":"2023-11-18T20:44:34.116296+0000","last_peered":"2023-11-18T20:44:34.116296+0000","last_clean":"2023-11-18T20:44:34.116296+0000","last_became_active":"2023-11-18T20:44:32.705933+0000","last_became_peered":"2023-11-18T20:44:32.705933+0000","last_unstale":"2023-11-18T20:44:34.116296+0000","last_undegraded":"2023-11-18T20:44:34.116296+0000","last_fullsized":"2023-11-18T20:44:34.116296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:11:38.556165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123704+0000","last_change":"2023-11-18T20:44:33.122460+0000","last_active":"2023-11-18T20:44:33.123704+0000","last_peered":"2023-11-18T20:44:33.123704+0000","last_clean":"2023-11-18T20:44:33.123704+0000","last_became_active":"2023-11-18T20:44:33.121060+0000","last_became_peered":"2023-11-18T20:44:33.121060+0000","last_unstale":"2023-11-18T20:44:33.123704+0000","last_undegraded":"2023-11-18T20:44:33.123704+0000","last_fullsized":"2023-11-18T20:44:33.123704+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:29:16.432705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.706741+0000","last_change":"2023-11-18T20:44:32.704847+0000","last_active":"2023-11-18T20:44:33.706741+0000","last_peered":"2023-11-18T20:44:33.706741+0000","last_clean":"2023-11-18T20:44:33.706741+0000","last_became_active":"2023-11-18T20:44:32.704568+0000","last_became_peered":"2023-11-18T20:44:32.704568+0000","last_unstale":"2023-11-18T20:44:33.706741+0000","last_undegraded":"2023-11-18T20:44:33.706741+0000","last_fullsized":"2023-11-18T20:44:33.706741+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:33:53.321421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116247+0000","last_change":"2023-11-18T20:44:33.115826+0000","last_active":"2023-11-18T20:44:34.116247+0000","last_peered":"2023-11-18T20:44:34.116247+0000","last_clean":"2023-11-18T20:44:34.116247+0000","last_became_active":"2023-11-18T20:44:33.115552+0000","last_became_peered":"2023-11-18T20:44:33.115552+0000","last_unstale":"2023-11-18T20:44:34.116247+0000","last_undegraded":"2023-11-18T20:44:34.116247+0000","last_fullsized":"2023-11-18T20:44:34.116247+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:06:07.786307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.709539+0000","last_change":"2023-11-18T20:44:32.715407+0000","last_active":"2023-11-18T20:44:33.709539+0000","last_peered":"2023-11-18T20:44:33.709539+0000","last_clean":"2023-11-18T20:44:33.709539+0000","last_became_active":"2023-11-18T20:44:32.715093+0000","last_became_peered":"2023-11-18T20:44:32.715093+0000","last_unstale":"2023-11-18T20:44:33.709539+0000","last_undegraded":"2023-11-18T20:44:33.709539+0000","last_fullsized":"2023-11-18T20:44:33.709539+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:49:11.455213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.127553+0000","last_change":"2023-11-18T20:44:33.124043+0000","last_active":"2023-11-18T20:44:33.127553+0000","last_peered":"2023-11-18T20:44:33.127553+0000","last_clean":"2023-11-18T20:44:33.127553+0000","last_became_active":"2023-11-18T20:44:33.123681+0000","last_became_peered":"2023-11-18T20:44:33.123681+0000","last_unstale":"2023-11-18T20:44:33.127553+0000","last_undegraded":"2023-11-18T20:44:33.127553+0000","last_fullsized":"2023-11-18T20:44:33.127553+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:06:07.786307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.708398+0000","last_change":"2023-11-18T20:44:32.710524+0000","last_active":"2023-11-18T20:44:33.708398+0000","last_peered":"2023-11-18T20:44:33.708398+0000","last_clean":"2023-11-18T20:44:33.708398+0000","last_became_active":"2023-11-18T20:44:32.709992+0000","last_became_peered":"2023-11-18T20:44:32.709992+0000","last_unstale":"2023-11-18T20:44:33.708398+0000","last_undegraded":"2023-11-18T20:44:33.708398+0000","last_fullsized":"2023-11-18T20:44:33.708398+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:26:13.396170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.118237+0000","last_change":"2023-11-18T20:44:33.116699+0000","last_active":"2023-11-18T20:44:33.118237+0000","last_peered":"2023-11-18T20:44:33.118237+0000","last_clean":"2023-11-18T20:44:33.118237+0000","last_became_active":"2023-11-18T20:44:33.116431+0000","last_became_peered":"2023-11-18T20:44:33.116431+0000","last_unstale":"2023-11-18T20:44:33.118237+0000","last_undegraded":"2023-11-18T20:44:33.118237+0000","last_fullsized":"2023-11-18T20:44:33.118237+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:54:54.404042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.117611+0000","last_change":"2023-11-18T20:44:32.708820+0000","last_active":"2023-11-18T20:44:33.117611+0000","last_peered":"2023-11-18T20:44:33.117611+0000","last_clean":"2023-11-18T20:44:33.117611+0000","last_became_active":"2023-11-18T20:44:32.708474+0000","last_became_peered":"2023-11-18T20:44:32.708474+0000","last_unstale":"2023-11-18T20:44:33.117611+0000","last_undegraded":"2023-11-18T20:44:33.117611+0000","last_fullsized":"2023-11-18T20:44:33.117611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:12:28.042350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123829+0000","last_change":"2023-11-18T20:44:33.118832+0000","last_active":"2023-11-18T20:44:33.123829+0000","last_peered":"2023-11-18T20:44:33.123829+0000","last_clean":"2023-11-18T20:44:33.123829+0000","last_became_active":"2023-11-18T20:44:33.118555+0000","last_became_peered":"2023-11-18T20:44:33.118555+0000","last_unstale":"2023-11-18T20:44:33.123829+0000","last_undegraded":"2023-11-18T20:44:33.123829+0000","last_fullsized":"2023-11-18T20:44:33.123829+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:38:48.032417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.111699+0000","last_change":"2023-11-18T20:44:32.705943+0000","last_active":"2023-11-18T20:44:33.111699+0000","last_peered":"2023-11-18T20:44:33.111699+0000","last_clean":"2023-11-18T20:44:33.111699+0000","last_became_active":"2023-11-18T20:44:32.705631+0000","last_became_peered":"2023-11-18T20:44:32.705631+0000","last_unstale":"2023-11-18T20:44:33.111699+0000","last_undegraded":"2023-11-18T20:44:33.111699+0000","last_fullsized":"2023-11-18T20:44:33.111699+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:59:19.079526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.117694+0000","last_change":"2023-11-18T20:44:33.115774+0000","last_active":"2023-11-18T20:44:33.117694+0000","last_peered":"2023-11-18T20:44:33.117694+0000","last_clean":"2023-11-18T20:44:33.117694+0000","last_became_active":"2023-11-18T20:44:33.115236+0000","last_became_peered":"2023-11-18T20:44:33.115236+0000","last_unstale":"2023-11-18T20:44:33.117694+0000","last_undegraded":"2023-11-18T20:44:33.117694+0000","last_fullsized":"2023-11-18T20:44:33.117694+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:20:37.793311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.708066+0000","last_change":"2023-11-18T20:44:32.706703+0000","last_active":"2023-11-18T20:44:33.708066+0000","last_peered":"2023-11-18T20:44:33.708066+0000","last_clean":"2023-11-18T20:44:33.708066+0000","last_became_active":"2023-11-18T20:44:32.706238+0000","last_became_peered":"2023-11-18T20:44:32.706238+0000","last_unstale":"2023-11-18T20:44:33.708066+0000","last_undegraded":"2023-11-18T20:44:33.708066+0000","last_fullsized":"2023-11-18T20:44:33.708066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:27:26.687286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126721+0000","last_change":"2023-11-18T20:44:33.122436+0000","last_active":"2023-11-18T20:44:33.126721+0000","last_peered":"2023-11-18T20:44:33.126721+0000","last_clean":"2023-11-18T20:44:33.126721+0000","last_became_active":"2023-11-18T20:44:33.122175+0000","last_became_peered":"2023-11-18T20:44:33.122175+0000","last_unstale":"2023-11-18T20:44:33.126721+0000","last_undegraded":"2023-11-18T20:44:33.126721+0000","last_fullsized":"2023-11-18T20:44:33.126721+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:02:32.890448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.706456+0000","last_change":"2023-11-18T20:44:32.711248+0000","last_active":"2023-11-18T20:44:33.706456+0000","last_peered":"2023-11-18T20:44:33.706456+0000","last_clean":"2023-11-18T20:44:33.706456+0000","last_became_active":"2023-11-18T20:44:32.710964+0000","last_became_peered":"2023-11-18T20:44:32.710964+0000","last_unstale":"2023-11-18T20:44:33.706456+0000","last_undegraded":"2023-11-18T20:44:33.706456+0000","last_fullsized":"2023-11-18T20:44:33.706456+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:10:38.238426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115719+0000","last_change":"2023-11-18T20:44:33.112324+0000","last_active":"2023-11-18T20:44:34.115719+0000","last_peered":"2023-11-18T20:44:34.115719+0000","last_clean":"2023-11-18T20:44:34.115719+0000","last_became_active":"2023-11-18T20:44:33.112052+0000","last_became_peered":"2023-11-18T20:44:33.112052+0000","last_unstale":"2023-11-18T20:44:34.115719+0000","last_undegraded":"2023-11-18T20:44:34.115719+0000","last_fullsized":"2023-11-18T20:44:34.115719+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:41:29.137921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.112806+0000","last_change":"2023-11-18T20:44:32.709435+0000","last_active":"2023-11-18T20:44:34.112806+0000","last_peered":"2023-11-18T20:44:34.112806+0000","last_clean":"2023-11-18T20:44:34.112806+0000","last_became_active":"2023-11-18T20:44:32.709151+0000","last_became_peered":"2023-11-18T20:44:32.709151+0000","last_unstale":"2023-11-18T20:44:34.112806+0000","last_undegraded":"2023-11-18T20:44:34.112806+0000","last_fullsized":"2023-11-18T20:44:34.112806+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:11:43.615416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.116011+0000","last_change":"2023-11-18T20:44:33.112491+0000","last_active":"2023-11-18T20:44:34.116011+0000","last_peered":"2023-11-18T20:44:34.116011+0000","last_clean":"2023-11-18T20:44:34.116011+0000","last_became_active":"2023-11-18T20:44:33.112231+0000","last_became_peered":"2023-11-18T20:44:33.112231+0000","last_unstale":"2023-11-18T20:44:34.116011+0000","last_undegraded":"2023-11-18T20:44:34.116011+0000","last_fullsized":"2023-11-18T20:44:34.116011+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:54:54.404042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.706049+0000","last_change":"2023-11-18T20:44:32.708428+0000","last_active":"2023-11-18T20:44:33.706049+0000","last_peered":"2023-11-18T20:44:33.706049+0000","last_clean":"2023-11-18T20:44:33.706049+0000","last_became_active":"2023-11-18T20:44:32.708123+0000","last_became_peered":"2023-11-18T20:44:32.708123+0000","last_unstale":"2023-11-18T20:44:33.706049+0000","last_undegraded":"2023-11-18T20:44:33.706049+0000","last_fullsized":"2023-11-18T20:44:33.706049+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:09:31.551437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115767+0000","last_change":"2023-11-18T20:44:33.115276+0000","last_active":"2023-11-18T20:44:34.115767+0000","last_peered":"2023-11-18T20:44:34.115767+0000","last_clean":"2023-11-18T20:44:34.115767+0000","last_became_active":"2023-11-18T20:44:33.114998+0000","last_became_peered":"2023-11-18T20:44:33.114998+0000","last_unstale":"2023-11-18T20:44:34.115767+0000","last_undegraded":"2023-11-18T20:44:34.115767+0000","last_fullsized":"2023-11-18T20:44:34.115767+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:32:40.407274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.113438+0000","last_change":"2023-11-18T20:44:32.714570+0000","last_active":"2023-11-18T20:44:33.113438+0000","last_peered":"2023-11-18T20:44:33.113438+0000","last_clean":"2023-11-18T20:44:33.113438+0000","last_became_active":"2023-11-18T20:44:32.714302+0000","last_became_peered":"2023-11-18T20:44:32.714302+0000","last_unstale":"2023-11-18T20:44:33.113438+0000","last_undegraded":"2023-11-18T20:44:33.113438+0000","last_fullsized":"2023-11-18T20:44:33.113438+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:35:21.213045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.122521+0000","last_change":"2023-11-18T20:44:33.119839+0000","last_active":"2023-11-18T20:44:33.122521+0000","last_peered":"2023-11-18T20:44:33.122521+0000","last_clean":"2023-11-18T20:44:33.122521+0000","last_became_active":"2023-11-18T20:44:33.119527+0000","last_became_peered":"2023-11-18T20:44:33.119527+0000","last_unstale":"2023-11-18T20:44:33.122521+0000","last_undegraded":"2023-11-18T20:44:33.122521+0000","last_fullsized":"2023-11-18T20:44:33.122521+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T02:06:33.900376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115816+0000","last_change":"2023-11-18T20:44:32.708787+0000","last_active":"2023-11-18T20:44:34.115816+0000","last_peered":"2023-11-18T20:44:34.115816+0000","last_clean":"2023-11-18T20:44:34.115816+0000","last_became_active":"2023-11-18T20:44:32.708503+0000","last_became_peered":"2023-11-18T20:44:32.708503+0000","last_unstale":"2023-11-18T20:44:34.115816+0000","last_undegraded":"2023-11-18T20:44:34.115816+0000","last_fullsized":"2023-11-18T20:44:34.115816+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:51:45.824233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123970+0000","last_change":"2023-11-18T20:44:33.122118+0000","last_active":"2023-11-18T20:44:33.123970+0000","last_peered":"2023-11-18T20:44:33.123970+0000","last_clean":"2023-11-18T20:44:33.123970+0000","last_became_active":"2023-11-18T20:44:33.121849+0000","last_became_peered":"2023-11-18T20:44:33.121849+0000","last_unstale":"2023-11-18T20:44:33.123970+0000","last_undegraded":"2023-11-18T20:44:33.123970+0000","last_fullsized":"2023-11-18T20:44:33.123970+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:50:07.789515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.114926+0000","last_change":"2023-11-18T20:44:32.713564+0000","last_active":"2023-11-18T20:44:33.114926+0000","last_peered":"2023-11-18T20:44:33.114926+0000","last_clean":"2023-11-18T20:44:33.114926+0000","last_became_active":"2023-11-18T20:44:32.712918+0000","last_became_peered":"2023-11-18T20:44:32.712918+0000","last_unstale":"2023-11-18T20:44:33.114926+0000","last_undegraded":"2023-11-18T20:44:33.114926+0000","last_fullsized":"2023-11-18T20:44:33.114926+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:29:28.392199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123354+0000","last_change":"2023-11-18T20:44:33.120827+0000","last_active":"2023-11-18T20:44:33.123354+0000","last_peered":"2023-11-18T20:44:33.123354+0000","last_clean":"2023-11-18T20:44:33.123354+0000","last_became_active":"2023-11-18T20:44:33.120286+0000","last_became_peered":"2023-11-18T20:44:33.120286+0000","last_unstale":"2023-11-18T20:44:33.123354+0000","last_undegraded":"2023-11-18T20:44:33.123354+0000","last_fullsized":"2023-11-18T20:44:33.123354+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:59:20.084399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.108516+0000","last_change":"2023-11-18T20:44:32.705264+0000","last_active":"2023-11-18T20:44:33.108516+0000","last_peered":"2023-11-18T20:44:33.108516+0000","last_clean":"2023-11-18T20:44:33.108516+0000","last_became_active":"2023-11-18T20:44:32.703632+0000","last_became_peered":"2023-11-18T20:44:32.703632+0000","last_unstale":"2023-11-18T20:44:33.108516+0000","last_undegraded":"2023-11-18T20:44:33.108516+0000","last_fullsized":"2023-11-18T20:44:33.108516+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:13:56.148888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115957+0000","last_change":"2023-11-18T20:44:33.124916+0000","last_active":"2023-11-18T20:44:34.115957+0000","last_peered":"2023-11-18T20:44:34.115957+0000","last_clean":"2023-11-18T20:44:34.115957+0000","last_became_active":"2023-11-18T20:44:33.124606+0000","last_became_peered":"2023-11-18T20:44:33.124606+0000","last_unstale":"2023-11-18T20:44:34.115957+0000","last_undegraded":"2023-11-18T20:44:34.115957+0000","last_fullsized":"2023-11-18T20:44:34.115957+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:10:39.243298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.112418+0000","last_change":"2023-11-18T20:44:32.709411+0000","last_active":"2023-11-18T20:44:34.112418+0000","last_peered":"2023-11-18T20:44:34.112418+0000","last_clean":"2023-11-18T20:44:34.112418+0000","last_became_active":"2023-11-18T20:44:32.709102+0000","last_became_peered":"2023-11-18T20:44:32.709102+0000","last_unstale":"2023-11-18T20:44:34.112418+0000","last_undegraded":"2023-11-18T20:44:34.112418+0000","last_fullsized":"2023-11-18T20:44:34.112418+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T06:37:53.020182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.127493+0000","last_change":"2023-11-18T20:44:33.123375+0000","last_active":"2023-11-18T20:44:33.127493+0000","last_peered":"2023-11-18T20:44:33.127493+0000","last_clean":"2023-11-18T20:44:33.127493+0000","last_became_active":"2023-11-18T20:44:33.122908+0000","last_became_peered":"2023-11-18T20:44:33.122908+0000","last_unstale":"2023-11-18T20:44:33.127493+0000","last_undegraded":"2023-11-18T20:44:33.127493+0000","last_fullsized":"2023-11-18T20:44:33.127493+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:26:21.163205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.705304+0000","last_change":"2023-11-18T20:44:32.711633+0000","last_active":"2023-11-18T20:44:33.705304+0000","last_peered":"2023-11-18T20:44:33.705304+0000","last_clean":"2023-11-18T20:44:33.705304+0000","last_became_active":"2023-11-18T20:44:32.711323+0000","last_became_peered":"2023-11-18T20:44:32.711323+0000","last_unstale":"2023-11-18T20:44:33.705304+0000","last_undegraded":"2023-11-18T20:44:33.705304+0000","last_fullsized":"2023-11-18T20:44:33.705304+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:14:02.621745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115815+0000","last_change":"2023-11-18T20:44:33.110025+0000","last_active":"2023-11-18T20:44:34.115815+0000","last_peered":"2023-11-18T20:44:34.115815+0000","last_clean":"2023-11-18T20:44:34.115815+0000","last_became_active":"2023-11-18T20:44:33.109462+0000","last_became_peered":"2023-11-18T20:44:33.109462+0000","last_unstale":"2023-11-18T20:44:34.115815+0000","last_undegraded":"2023-11-18T20:44:34.115815+0000","last_fullsized":"2023-11-18T20:44:34.115815+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T04:37:18.028311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.117028+0000","last_change":"2023-11-18T20:44:32.704380+0000","last_active":"2023-11-18T20:44:33.117028+0000","last_peered":"2023-11-18T20:44:33.117028+0000","last_clean":"2023-11-18T20:44:33.117028+0000","last_became_active":"2023-11-18T20:44:32.703862+0000","last_became_peered":"2023-11-18T20:44:32.703862+0000","last_unstale":"2023-11-18T20:44:33.117028+0000","last_undegraded":"2023-11-18T20:44:33.117028+0000","last_fullsized":"2023-11-18T20:44:33.117028+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:02:31.885575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.125158+0000","last_change":"2023-11-18T20:44:33.122223+0000","last_active":"2023-11-18T20:44:33.125158+0000","last_peered":"2023-11-18T20:44:33.125158+0000","last_clean":"2023-11-18T20:44:33.125158+0000","last_became_active":"2023-11-18T20:44:33.121918+0000","last_became_peered":"2023-11-18T20:44:33.121918+0000","last_unstale":"2023-11-18T20:44:33.125158+0000","last_undegraded":"2023-11-18T20:44:33.125158+0000","last_fullsized":"2023-11-18T20:44:33.125158+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:03:47.903678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.107753+0000","last_change":"2023-11-18T20:44:32.711007+0000","last_active":"2023-11-18T20:44:33.107753+0000","last_peered":"2023-11-18T20:44:33.107753+0000","last_clean":"2023-11-18T20:44:33.107753+0000","last_became_active":"2023-11-18T20:44:32.710532+0000","last_became_peered":"2023-11-18T20:44:32.710532+0000","last_unstale":"2023-11-18T20:44:33.107753+0000","last_undegraded":"2023-11-18T20:44:33.107753+0000","last_fullsized":"2023-11-18T20:44:33.107753+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:04:14.743171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.123479+0000","last_change":"2023-11-18T20:44:33.121604+0000","last_active":"2023-11-18T20:44:33.123479+0000","last_peered":"2023-11-18T20:44:33.123479+0000","last_clean":"2023-11-18T20:44:33.123479+0000","last_became_active":"2023-11-18T20:44:33.120807+0000","last_became_peered":"2023-11-18T20:44:33.120807+0000","last_unstale":"2023-11-18T20:44:33.123479+0000","last_undegraded":"2023-11-18T20:44:33.123479+0000","last_fullsized":"2023-11-18T20:44:33.123479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T00:10:54.408039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.115340+0000","last_change":"2023-11-18T20:44:32.715757+0000","last_active":"2023-11-18T20:44:33.115340+0000","last_peered":"2023-11-18T20:44:33.115340+0000","last_clean":"2023-11-18T20:44:33.115340+0000","last_became_active":"2023-11-18T20:44:32.714748+0000","last_became_peered":"2023-11-18T20:44:32.714748+0000","last_unstale":"2023-11-18T20:44:33.115340+0000","last_undegraded":"2023-11-18T20:44:33.115340+0000","last_fullsized":"2023-11-18T20:44:33.115340+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:12:10.434962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.125153+0000","last_change":"2023-11-18T20:44:33.121799+0000","last_active":"2023-11-18T20:44:33.125153+0000","last_peered":"2023-11-18T20:44:33.125153+0000","last_clean":"2023-11-18T20:44:33.125153+0000","last_became_active":"2023-11-18T20:44:33.120734+0000","last_became_peered":"2023-11-18T20:44:33.120734+0000","last_unstale":"2023-11-18T20:44:33.125153+0000","last_undegraded":"2023-11-18T20:44:33.125153+0000","last_fullsized":"2023-11-18T20:44:33.125153+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T01:01:59.298618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.115051+0000","last_change":"2023-11-18T20:44:32.713756+0000","last_active":"2023-11-18T20:44:33.115051+0000","last_peered":"2023-11-18T20:44:33.115051+0000","last_clean":"2023-11-18T20:44:33.115051+0000","last_became_active":"2023-11-18T20:44:32.713302+0000","last_became_peered":"2023-11-18T20:44:32.713302+0000","last_unstale":"2023-11-18T20:44:33.115051+0000","last_undegraded":"2023-11-18T20:44:33.115051+0000","last_fullsized":"2023-11-18T20:44:33.115051+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T21:54:23.282569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.124004+0000","last_change":"2023-11-18T20:44:33.120978+0000","last_active":"2023-11-18T20:44:33.124004+0000","last_peered":"2023-11-18T20:44:33.124004+0000","last_clean":"2023-11-18T20:44:33.124004+0000","last_became_active":"2023-11-18T20:44:33.120596+0000","last_became_peered":"2023-11-18T20:44:33.120596+0000","last_unstale":"2023-11-18T20:44:33.124004+0000","last_undegraded":"2023-11-18T20:44:33.124004+0000","last_fullsized":"2023-11-18T20:44:33.124004+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T05:32:40.407274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.115962+0000","last_change":"2023-11-18T20:44:32.709561+0000","last_active":"2023-11-18T20:44:34.115962+0000","last_peered":"2023-11-18T20:44:34.115962+0000","last_clean":"2023-11-18T20:44:34.115962+0000","last_became_active":"2023-11-18T20:44:32.709278+0000","last_became_peered":"2023-11-18T20:44:32.709278+0000","last_unstale":"2023-11-18T20:44:34.115962+0000","last_undegraded":"2023-11-18T20:44:34.115962+0000","last_fullsized":"2023-11-18T20:44:34.115962+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:04:14.743171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.126963+0000","last_change":"2023-11-18T20:44:33.124163+0000","last_active":"2023-11-18T20:44:33.126963+0000","last_peered":"2023-11-18T20:44:33.126963+0000","last_clean":"2023-11-18T20:44:33.126963+0000","last_became_active":"2023-11-18T20:44:33.123694+0000","last_became_peered":"2023-11-18T20:44:33.123694+0000","last_unstale":"2023-11-18T20:44:33.126963+0000","last_undegraded":"2023-11-18T20:44:33.126963+0000","last_fullsized":"2023-11-18T20:44:33.126963+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T22:03:07.674062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.112812+0000","last_change":"2023-11-18T20:44:32.710940+0000","last_active":"2023-11-18T20:44:33.112812+0000","last_peered":"2023-11-18T20:44:33.112812+0000","last_clean":"2023-11-18T20:44:33.112812+0000","last_became_active":"2023-11-18T20:44:32.710657+0000","last_became_peered":"2023-11-18T20:44:32.710657+0000","last_unstale":"2023-11-18T20:44:33.112812+0000","last_undegraded":"2023-11-18T20:44:33.112812+0000","last_fullsized":"2023-11-18T20:44:33.112812+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T03:59:19.079526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.121500+0000","last_change":"2023-11-18T20:44:33.118740+0000","last_active":"2023-11-18T20:44:33.121500+0000","last_peered":"2023-11-18T20:44:33.121500+0000","last_clean":"2023-11-18T20:44:33.121500+0000","last_became_active":"2023-11-18T20:44:33.118493+0000","last_became_peered":"2023-11-18T20:44:33.118493+0000","last_unstale":"2023-11-18T20:44:33.121500+0000","last_undegraded":"2023-11-18T20:44:33.121500+0000","last_fullsized":"2023-11-18T20:44:33.121500+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:32.685029+0000","last_clean_scrub_stamp":"2023-11-18T20:44:32.685029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T23:52:12.610261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-18T20:44:33.109262+0000","last_change":"2023-11-18T20:44:32.706159+0000","last_active":"2023-11-18T20:44:33.109262+0000","last_peered":"2023-11-18T20:44:33.109262+0000","last_clean":"2023-11-18T20:44:33.109262+0000","last_became_active":"2023-11-18T20:44:32.705892+0000","last_became_peered":"2023-11-18T20:44:32.705892+0000","last_unstale":"2023-11-18T20:44:33.109262+0000","last_undegraded":"2023-11-18T20:44:33.109262+0000","last_fullsized":"2023-11-18T20:44:33.109262+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:44:31.680157+0000","last_clean_scrub_stamp":"2023-11-18T20:44:31.680157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T08:02:31.885575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-18T20:44:32.698732+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:44:32.698732+0000","last_peered":"2023-11-18T20:44:32.698732+0000","last_clean":"2023-11-18T20:44:32.698732+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:44:32.698732+0000","last_undegraded":"2023-11-18T20:44:32.698732+0000","last_fullsized":"2023-11-18T20:44:32.698732+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1163264,"data_stored":825331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.49399999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71699999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.50900000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.40000000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.45600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55200000000000005}]},{"osd":4,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72699999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73499999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.79800000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.44900000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79800000000000004,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77600000000000002,"5min":0.876,"15min":0.876},"last":0.57999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":577536,"data_stored":235615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.36599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.68300000000000005}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.82499999999999996}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.71099999999999997}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.80000000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75700000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.624},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53500000000000003}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76600000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":234963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":1.012},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.38600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.94699999999999995}]},{"osd":1,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.002},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.93500000000000005}]},{"osd":2,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.45900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.996}]},{"osd":3,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91900000000000004,"5min":1.046,"15min":1.046},"last":0.81399999999999995}]},{"osd":4,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.98299999999999998}]},{"osd":6,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77000000000000002,"5min":0.875,"15min":0.875},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85899999999999999}]},{"osd":7,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95999999999999996,"5min":1.099,"15min":1.099},"last":0.95999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":235545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54700000000000004}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.39800000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71499999999999997,"5min":0.747,"15min":0.747},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51100000000000001}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.56899999999999995}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73299999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.752,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.58699999999999997}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63600000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.34699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.623}]},{"osd":7,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81699999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.69799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":236597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.441,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33400000000000002}]},{"osd":1,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.317,"15min":0.315},"min":{"1min":0.26100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35699999999999998}]},{"osd":2,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.32900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53100000000000003,"5min":0.995,"15min":0.995},"last":0.38100000000000001}]},{"osd":4,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77700000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.626},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61799999999999999}]},{"osd":6,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77000000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56499999999999995}]},{"osd":7,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81299999999999994,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":581632,"data_stored":236575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33800000000000002,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.754},{"interface":"front","average":{"1min":0.375,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":2.3149999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":2.875},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41699999999999998,"15min":0.42299999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":2.1869999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54500000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.54600000000000004,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":1.9470000000000001}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":1.093},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71999999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":2.3500000000000001}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":2.7029999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.1339999999999999}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.68999999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83899999999999997,"5min":1.859,"15min":1.859},"last":2.3849999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":2.7450000000000001}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.76100000000000001,"5min":1.464,"15min":1.464},"last":2.2810000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.68600000000000005,"15min":0.70099999999999996},"min":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.749,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":2.7829999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":589824,"data_stored":241423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.38400000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.371},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48999999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47799999999999998}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33300000000000002},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51300000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.57099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55400000000000005}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":1.105,"15min":1.105},"last":0.439},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.52500000000000002}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.50900000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72499999999999998,"5min":0.995,"15min":0.995},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76900000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.496}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1159168,"data_stored":824712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33500000000000002,"15min":0.33600000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.374,"15min":0.375},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":0.62,"15min":0.62},"last":0.36099999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.51300000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.37,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.38100000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.438},{"interface":"front","average":{"1min":0.438,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.48299999999999998}]},{"osd":4,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63100000000000001}]},{"osd":5,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81100000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72499999999999998}]},{"osd":7,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.625},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95899999999999996,"5min":1.006,"15min":1.006},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:44:37.060 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-18T20:44:37.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-18T20:44:37.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.207+0000 7f073967c640 1 Processor -- start 2023-11-18T20:44:37.209 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.208+0000 7f073967c640 1 -- start start 2023-11-18T20:44:37.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.208+0000 7f073967c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0734146010 0x7f07340cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.208+0000 7f073967c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07340b3400 0x7f073406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.210 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.208+0000 7f073967c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0734161170 con 0x7f0734146d70 2023-11-18T20:44:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.208+0000 7f073967c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0734161450 con 0x7f07340b3400 2023-11-18T20:44:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.208+0000 7f073967c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0734161730 con 0x7f0734146010 2023-11-18T20:44:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.208+0000 7f07337fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07340b3400 0x7f073406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.209+0000 7f07337fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07340b3400 0x7f073406c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49944/0 (socket says 172.21.15.60:49944) 2023-11-18T20:44:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.209+0000 7f07337fe640 1 -- 172.21.15.60:0/2860800534 learned_addr learned my addr 172.21.15.60:0/2860800534 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.209+0000 7f0732ffd640 1 --2- 172.21.15.60:0/2860800534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0734146010 0x7f07340cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.209+0000 7f07337fe640 1 -- 172.21.15.60:0/2860800534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0734146010 msgr2=0x7f07340cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.209+0000 7f07337fe640 1 --2- 172.21.15.60:0/2860800534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0734146010 0x7f07340cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.209+0000 7f07337fe640 1 -- 172.21.15.60:0/2860800534 >> v1:172.21.15.100:6789/0 conn(0x7f0734146d70 legacy=0x7f07340b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.209+0000 7f07337fe640 1 -- 172.21.15.60:0/2860800534 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734161a10 con 0x7f07340b3400 2023-11-18T20:44:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f07337fe640 1 --2- 172.21.15.60:0/2860800534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 0x7f073406c6b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0724009e50 tx=0x7f072402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed53ea468b3a083 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f0731ffb640 1 -- 172.21.15.60:0/2860800534 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072402cb00 con 0x7f07340b3400 2023-11-18T20:44:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f0731ffb640 1 -- 172.21.15.60:0/2860800534 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072402cc60 con 0x7f07340b3400 2023-11-18T20:44:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f0731ffb640 1 -- 172.21.15.60:0/2860800534 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240343f0 con 0x7f07340b3400 2023-11-18T20:44:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f073967c640 1 -- 172.21.15.60:0/2860800534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 msgr2=0x7f073406c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f073967c640 1 --2- 172.21.15.60:0/2860800534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 0x7f073406c6b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0724009e50 tx=0x7f072402bf60 comp rx=0 tx=0).stop 2023-11-18T20:44:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f073967c640 1 -- 172.21.15.60:0/2860800534 shutdown_connections 2023-11-18T20:44:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f073967c640 1 --2- 172.21.15.60:0/2860800534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 0x7f073406c6b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f073967c640 1 --2- 172.21.15.60:0/2860800534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0734146010 0x7f07340cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f073967c640 1 -- 172.21.15.60:0/2860800534 >> 172.21.15.60:0/2860800534 conn(0x7f073406a4d0 msgr2=0x7f073406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:37.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f073967c640 1 -- 172.21.15.60:0/2860800534 shutdown_connections 2023-11-18T20:44:37.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.210+0000 7f073967c640 1 -- 172.21.15.60:0/2860800534 wait complete. 2023-11-18T20:44:37.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.212+0000 7f073967c640 1 Processor -- start 2023-11-18T20:44:37.214 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.212+0000 7f073967c640 1 -- start start 2023-11-18T20:44:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.212+0000 7f073967c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 0x7f07341f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.212+0000 7f073967c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734146010 0x7f073415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.212+0000 7f073967c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0734146d70 0x7f07340ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.212+0000 7f073967c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07341f4070 con 0x7f07340b3400 2023-11-18T20:44:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.212+0000 7f073967c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07341f74c0 con 0x7f0734146010 2023-11-18T20:44:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.212+0000 7f073967c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07341f3460 con 0x7f0734146d70 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f0732ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 0x7f07341f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f0732ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 0x7f07341f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49954/0 (socket says 172.21.15.60:49954) 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f0732ffd640 1 -- 172.21.15.60:0/4165164481 learned_addr learned my addr 172.21.15.60:0/4165164481 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f07327fc640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734146010 0x7f073415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f0732ffd640 1 -- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0734146d70 msgr2=0x7f07340ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f07337fe640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0734146d70 0x7f07340ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f0732ffd640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0734146d70 0x7f07340ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f0732ffd640 1 -- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734146010 msgr2=0x7f073415cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f0732ffd640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734146010 0x7f073415cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f0732ffd640 1 -- 172.21.15.60:0/4165164481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0724009b30 con 0x7f07340b3400 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f07327fc640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734146010 0x7f073415cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f0732ffd640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 0x7f07341f6f80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f072c00d800 tx=0x7f072c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f072b7fe640 1 -- 172.21.15.60:0/4165164481 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00b6c0 con 0x7f07340b3400 2023-11-18T20:44:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f072b7fe640 1 -- 172.21.15.60:0/4165164481 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c00b820 con 0x7f07340b3400 2023-11-18T20:44:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f073967c640 1 -- 172.21.15.60:0/4165164481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0734146570 con 0x7f07340b3400 2023-11-18T20:44:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f072b7fe640 1 -- 172.21.15.60:0/4165164481 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00bbb0 con 0x7f07340b3400 2023-11-18T20:44:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.213+0000 7f073967c640 1 -- 172.21.15.60:0/4165164481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07341f3a20 con 0x7f07340b3400 2023-11-18T20:44:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.214+0000 7f073967c640 1 -- 172.21.15.60:0/4165164481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07340b0a40 con 0x7f07340b3400 2023-11-18T20:44:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.215+0000 7f072b7fe640 1 -- 172.21.15.60:0/4165164481 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f072c00bd10 con 0x7f07340b3400 2023-11-18T20:44:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.215+0000 7f072b7fe640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f071406ae40 0x7f071406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.215+0000 7f072b7fe640 1 -- 172.21.15.60:0/4165164481 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f072c089b70 con 0x7f07340b3400 2023-11-18T20:44:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.215+0000 7f07327fc640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f071406ae40 0x7f071406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.216+0000 7f07327fc640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f071406ae40 0x7f071406d300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f073415f880 tx=0x7f071c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.219+0000 7f072b7fe640 1 -- 172.21.15.60:0/4165164481 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f072c054cb0 con 0x7f07340b3400 2023-11-18T20:44:37.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.464+0000 7f073967c640 1 -- 172.21.15.60:0/4165164481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f07341f41b0 con 0x7f07340b3400 2023-11-18T20:44:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.467+0000 7f072b7fe640 1 -- 172.21.15.60:0/4165164481 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f072c0582d0 con 0x7f07340b3400 2023-11-18T20:44:37.468 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-11-18T20:44:37.468 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA== 2023-11-18T20:44:37.468 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-11-18T20:44:37.468 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-11-18T20:44:37.468 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-18T20:44:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-11-18T20:44:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 -- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f071406ae40 msgr2=0x7f071406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f071406ae40 0x7f071406d300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f073415f880 tx=0x7f071c009040 comp rx=0 tx=0).stop 2023-11-18T20:44:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 -- 172.21.15.60:0/4165164481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 msgr2=0x7f07341f6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 0x7f07341f6f80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f072c00d800 tx=0x7f072c00dcc0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 -- 172.21.15.60:0/4165164481 shutdown_connections 2023-11-18T20:44:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f071406ae40 0x7f071406d300 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0734146d70 0x7f07340ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0734146010 0x7f073415cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 --2- 172.21.15.60:0/4165164481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07340b3400 0x7f07341f6f80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 -- 172.21.15.60:0/4165164481 >> 172.21.15.60:0/4165164481 conn(0x7f073406a4d0 msgr2=0x7f0734143600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 -- 172.21.15.60:0/4165164481 shutdown_connections 2023-11-18T20:44:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.469+0000 7f073967c640 1 -- 172.21.15.60:0/4165164481 wait complete. 2023-11-18T20:44:37.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:44:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.586+0000 7f74f0221640 1 Processor -- start 2023-11-18T20:44:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.586+0000 7f74f0221640 1 -- start start 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.587+0000 7f74f0221640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f74e8143490 0x7f74e80b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.587+0000 7f74f0221640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f74e815abf0 0x7f74e80b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.587+0000 7f74f0221640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e815e2a0 con 0x7f74e815e730 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.587+0000 7f74f0221640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e815b520 con 0x7f74e8143490 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.587+0000 7f74f0221640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e8143870 con 0x7f74e815abf0 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.587+0000 7f74edfbd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f74e8143490 0x7f74e80b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.587+0000 7f74edfbd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f74e8143490 0x7f74e80b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49964/0 (socket says 172.21.15.60:49964) 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.587+0000 7f74edfbd640 1 -- 172.21.15.60:0/3709649039 learned_addr learned my addr 172.21.15.60:0/3709649039 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.587+0000 7f74ed7bc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f74e815e730 0x7f74e815a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55428/0 (socket says 172.21.15.60:55428) 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.588+0000 7f74edfbd640 1 -- 172.21.15.60:0/3709649039 >> v2:172.21.15.100:3300/0 conn(0x7f74e815abf0 msgr2=0x7f74e80b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.588+0000 7f74edfbd640 1 --2- 172.21.15.60:0/3709649039 >> v2:172.21.15.100:3300/0 conn(0x7f74e815abf0 0x7f74e80b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.588+0000 7f74edfbd640 1 -- 172.21.15.60:0/3709649039 >> v1:172.21.15.60:6789/0 conn(0x7f74e815e730 legacy=0x7f74e815a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.588+0000 7f74edfbd640 1 -- 172.21.15.60:0/3709649039 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e815b240 con 0x7f74e8143490 2023-11-18T20:44:37.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.588+0000 7f74edfbd640 1 --2- 172.21.15.60:0/3709649039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e8143490 0x7f74e80b0cf0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f74e80b4ca0 tx=0x7f74e002dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dc6fea3effcef6 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:37.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.588+0000 7f74ecfbb640 1 -- 172.21.15.60:0/3709649039 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e002e960 con 0x7f74e8143490 2023-11-18T20:44:37.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.589+0000 7f74ecfbb640 1 -- 172.21.15.60:0/3709649039 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e002eac0 con 0x7f74e8143490 2023-11-18T20:44:37.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.589+0000 7f74ecfbb640 1 -- 172.21.15.60:0/3709649039 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e002ee50 con 0x7f74e8143490 2023-11-18T20:44:37.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.589+0000 7f74f0221640 1 -- 172.21.15.60:0/3709649039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e8143490 msgr2=0x7f74e80b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.589+0000 7f74f0221640 1 --2- 172.21.15.60:0/3709649039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e8143490 0x7f74e80b0cf0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f74e80b4ca0 tx=0x7f74e002dbb0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.589+0000 7f74f0221640 1 -- 172.21.15.60:0/3709649039 shutdown_connections 2023-11-18T20:44:37.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.589+0000 7f74f0221640 1 --2- 172.21.15.60:0/3709649039 >> v2:172.21.15.100:3300/0 conn(0x7f74e815abf0 0x7f74e80b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.589+0000 7f74f0221640 1 --2- 172.21.15.60:0/3709649039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e8143490 0x7f74e80b0cf0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.589+0000 7f74f0221640 1 -- 172.21.15.60:0/3709649039 >> 172.21.15.60:0/3709649039 conn(0x7f74e806a4d0 msgr2=0x7f74e806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:37.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.589+0000 7f74f0221640 1 -- 172.21.15.60:0/3709649039 shutdown_connections 2023-11-18T20:44:37.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.589+0000 7f74f0221640 1 -- 172.21.15.60:0/3709649039 wait complete. 2023-11-18T20:44:37.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.590+0000 7f74f0221640 1 Processor -- start 2023-11-18T20:44:37.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.590+0000 7f74f0221640 1 -- start start 2023-11-18T20:44:37.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74f0221640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74e8143490 0x7f74e80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74f0221640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74e815abf0 0x7f74e80ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74f0221640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e815e730 0x7f74e80cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74f0221640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e812f7e0 con 0x7f74e815e730 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74f0221640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e812fb30 con 0x7f74e815abf0 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74f0221640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80a9b80 con 0x7f74e8143490 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74ee7be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e815e730 0x7f74e80cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74ee7be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e815e730 0x7f74e80cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49976/0 (socket says 172.21.15.60:49976) 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74ee7be640 1 -- 172.21.15.60:0/165945548 learned_addr learned my addr 172.21.15.60:0/165945548 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74edfbd640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74e8143490 0x7f74e80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74ed7bc640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74e815abf0 0x7f74e80ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74ee7be640 1 -- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74e8143490 msgr2=0x7f74e80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74ee7be640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74e8143490 0x7f74e80ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74ee7be640 1 -- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74e815abf0 msgr2=0x7f74e80ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74ee7be640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74e815abf0 0x7f74e80ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74ee7be640 1 -- 172.21.15.60:0/165945548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00094d0 con 0x7f74e815e730 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.591+0000 7f74ed7bc640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74e815abf0 0x7f74e80ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.592+0000 7f74ee7be640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e815e730 0x7f74e80cab30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f74d800b970 tx=0x7f74d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.592+0000 7f74e67fc640 1 -- 172.21.15.60:0/165945548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d800cb40 con 0x7f74e815e730 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.592+0000 7f74e67fc640 1 -- 172.21.15.60:0/165945548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d800ccc0 con 0x7f74e815e730 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.592+0000 7f74f0221640 1 -- 172.21.15.60:0/165945548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e812fdd0 con 0x7f74e815e730 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.592+0000 7f74e67fc640 1 -- 172.21.15.60:0/165945548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8014680 con 0x7f74e815e730 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.592+0000 7f74f0221640 1 -- 172.21.15.60:0/165945548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e80a9e60 con 0x7f74e815e730 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.593+0000 7f74f0221640 1 -- 172.21.15.60:0/165945548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e8069400 con 0x7f74e815e730 2023-11-18T20:44:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.593+0000 7f74e67fc640 1 -- 172.21.15.60:0/165945548 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74d801d3b0 con 0x7f74e815e730 2023-11-18T20:44:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.594+0000 7f74e67fc640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f74d406ae40 0x7f74d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.594+0000 7f74e67fc640 1 -- 172.21.15.60:0/165945548 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74d8089590 con 0x7f74e815e730 2023-11-18T20:44:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.594+0000 7f74edfbd640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f74d406ae40 0x7f74d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.594+0000 7f74edfbd640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f74d406ae40 0x7f74d406d300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f74e0038000 tx=0x7f74e0032000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:37.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.598+0000 7f74e67fc640 1 -- 172.21.15.60:0/165945548 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74d8054640 con 0x7f74e815e730 2023-11-18T20:44:37.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.810+0000 7f74f0221640 1 -- 172.21.15.60:0/165945548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74e81302f0 con 0x7f74e815e730 2023-11-18T20:44:37.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.810+0000 7f74e67fc640 1 -- 172.21.15.60:0/165945548 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f74d8057c60 con 0x7f74e815e730 2023-11-18T20:44:37.812 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:37.812 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1578336464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1578336464},{"type":"v1","addr":"172.21.15.100:6836","nonce":1578336464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/88984334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":88984334},{"type":"v1","addr":"172.21.15.100:6837","nonce":88984334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3777286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3777286992},{"type":"v1","addr":"172.21.15.60:6833","nonce":3777286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:44:33.070091+0000","modified":"2023-11-18T20:44:35.050014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3098423505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3098423505},{"type":"v1","addr":"172.21.15.60:6835","nonce":3098423505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-18T20:44:37.812 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2023-11-18T20:44:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.812+0000 7f74f0221640 1 -- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f74d406ae40 msgr2=0x7f74d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.812+0000 7f74f0221640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f74d406ae40 0x7f74d406d300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f74e0038000 tx=0x7f74e0032000 comp rx=0 tx=0).stop 2023-11-18T20:44:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.812+0000 7f74f0221640 1 -- 172.21.15.60:0/165945548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e815e730 msgr2=0x7f74e80cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.812+0000 7f74f0221640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e815e730 0x7f74e80cab30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f74d800b970 tx=0x7f74d800be30 comp rx=0 tx=0).stop 2023-11-18T20:44:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.812+0000 7f74f0221640 1 -- 172.21.15.60:0/165945548 shutdown_connections 2023-11-18T20:44:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.812+0000 7f74f0221640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f74d406ae40 0x7f74d406d300 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.812+0000 7f74f0221640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e815e730 0x7f74e80cab30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.812+0000 7f74f0221640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74e815abf0 0x7f74e80ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.812+0000 7f74f0221640 1 --2- 172.21.15.60:0/165945548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74e8143490 0x7f74e80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.812+0000 7f74f0221640 1 -- 172.21.15.60:0/165945548 >> 172.21.15.60:0/165945548 conn(0x7f74e806a4d0 msgr2=0x7f74e8071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.813+0000 7f74f0221640 1 -- 172.21.15.60:0/165945548 shutdown_connections 2023-11-18T20:44:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:37.813+0000 7f74f0221640 1 -- 172.21.15.60:0/165945548 wait complete. 2023-11-18T20:44:38.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-18T20:44:33.070091+0000', 'modified': '2023-11-18T20:44:35.050014+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5886}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5886': {'gid': 5886, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/3098423505', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 3098423505}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 3098423505}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-18T20:44:38.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-18T20:44:38.227 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-18T20:44:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.336+0000 7f70662bb640 1 Processor -- start 2023-11-18T20:44:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.336+0000 7f70662bb640 1 -- start start 2023-11-18T20:44:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.336+0000 7f70662bb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f706006ea50 0x7f706006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.336+0000 7f70662bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7060160fb0 0x7f7060161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.336+0000 7f70662bb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7060161d20 con 0x7f70600b3ff0 2023-11-18T20:44:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.336+0000 7f70662bb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70600d1ee0 con 0x7f706006ea50 2023-11-18T20:44:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.336+0000 7f70662bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706006f370 con 0x7f7060160fb0 2023-11-18T20:44:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.337+0000 7f705ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f706006ea50 0x7f706006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.337+0000 7f705ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f706006ea50 0x7f706006ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49992/0 (socket says 172.21.15.60:49992) 2023-11-18T20:44:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.337+0000 7f705ffff640 1 -- 172.21.15.60:0/2041570625 learned_addr learned my addr 172.21.15.60:0/2041570625 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.337+0000 7f705ffff640 1 -- 172.21.15.60:0/2041570625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7060160fb0 msgr2=0x7f7060161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.337+0000 7f705ffff640 1 --2- 172.21.15.60:0/2041570625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7060160fb0 0x7f7060161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.337+0000 7f705ffff640 1 -- 172.21.15.60:0/2041570625 >> v1:172.21.15.100:6789/0 conn(0x7f70600b3ff0 legacy=0x7f70600b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.337+0000 7f705ffff640 1 -- 172.21.15.60:0/2041570625 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706015e760 con 0x7f706006ea50 2023-11-18T20:44:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f705ffff640 1 --2- 172.21.15.60:0/2041570625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706006ea50 0x7f706006ee30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7050009870 tx=0x7f705002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcaed8a5437dfee server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f705effd640 1 -- 172.21.15.60:0/2041570625 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050031640 con 0x7f706006ea50 2023-11-18T20:44:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f705effd640 1 -- 172.21.15.60:0/2041570625 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70500317a0 con 0x7f706006ea50 2023-11-18T20:44:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f705effd640 1 -- 172.21.15.60:0/2041570625 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050031b30 con 0x7f706006ea50 2023-11-18T20:44:38.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f70662bb640 1 -- 172.21.15.60:0/2041570625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706006ea50 msgr2=0x7f706006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.342 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f70662bb640 1 --2- 172.21.15.60:0/2041570625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706006ea50 0x7f706006ee30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7050009870 tx=0x7f705002b8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f70662bb640 1 -- 172.21.15.60:0/2041570625 shutdown_connections 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f70662bb640 1 --2- 172.21.15.60:0/2041570625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7060160fb0 0x7f7060161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f70662bb640 1 --2- 172.21.15.60:0/2041570625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706006ea50 0x7f706006ee30 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f70662bb640 1 -- 172.21.15.60:0/2041570625 >> 172.21.15.60:0/2041570625 conn(0x7f706006a6b0 msgr2=0x7f706006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f70662bb640 1 -- 172.21.15.60:0/2041570625 shutdown_connections 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.338+0000 7f70662bb640 1 -- 172.21.15.60:0/2041570625 wait complete. 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.339+0000 7f70662bb640 1 Processor -- start 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.340+0000 7f70662bb640 1 -- start start 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.340+0000 7f70662bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706006ea50 0x7f706013ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.340+0000 7f70662bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70600b3ff0 0x7f706015a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.340+0000 7f70662bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7060160fb0 0x7f7060158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.340+0000 7f70662bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7060151a10 con 0x7f7060160fb0 2023-11-18T20:44:38.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.340+0000 7f70662bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706015ae30 con 0x7f706006ea50 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.340+0000 7f70662bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706015ab50 con 0x7f70600b3ff0 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.340+0000 7f7064858640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7060160fb0 0x7f7060158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f7064858640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7060160fb0 0x7f7060158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50008/0 (socket says 172.21.15.60:50008) 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f7064858640 1 -- 172.21.15.60:0/287150855 learned_addr learned my addr 172.21.15.60:0/287150855 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f705f7fe640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70600b3ff0 0x7f706015a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f705ffff640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706006ea50 0x7f706013ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f7064858640 1 -- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70600b3ff0 msgr2=0x7f706015a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f7064858640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70600b3ff0 0x7f706015a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f7064858640 1 -- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706006ea50 msgr2=0x7f706013ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f7064858640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706006ea50 0x7f706013ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f7064858640 1 -- 172.21.15.60:0/287150855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70500094d0 con 0x7f7060160fb0 2023-11-18T20:44:38.344 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f7064858640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7060160fb0 0x7f7060158010 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f705800b9e0 tx=0x7f705800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:38.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f705cff9640 1 -- 172.21.15.60:0/287150855 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705800caa0 con 0x7f7060160fb0 2023-11-18T20:44:38.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f705cff9640 1 -- 172.21.15.60:0/287150855 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705800cc00 con 0x7f7060160fb0 2023-11-18T20:44:38.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f70662bb640 1 -- 172.21.15.60:0/287150855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70601e73e0 con 0x7f7060160fb0 2023-11-18T20:44:38.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f705cff9640 1 -- 172.21.15.60:0/287150855 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058014440 con 0x7f7060160fb0 2023-11-18T20:44:38.345 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.341+0000 7f70662bb640 1 -- 172.21.15.60:0/287150855 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70601e79a0 con 0x7f7060160fb0 2023-11-18T20:44:38.347 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.343+0000 7f705cff9640 1 -- 172.21.15.60:0/287150855 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7058007370 con 0x7f7060160fb0 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.343+0000 7f705cff9640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f704c06ae70 0x7f704c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.343+0000 7f705cff9640 1 -- 172.21.15.60:0/287150855 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70580894c0 con 0x7f7060160fb0 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.344+0000 7f705ffff640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f704c06ae70 0x7f704c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.344+0000 7f705ffff640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f704c06ae70 0x7f704c06d330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7050036000 tx=0x7f705003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.344+0000 7f7068e52740 1 Processor -- start 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.344+0000 7f7068e52740 1 -- start start 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f7068e52740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f7068e52740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514f70 0x55bc80517920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f7068e52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc804e8070 0x55bc80517e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f7068e52740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc804a88c0 con 0x55bc80514b90 2023-11-18T20:44:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f7068e52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8045ab90 con 0x55bc804e8070 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f7068e52740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc80458280 con 0x55bc80514f70 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f705ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f705ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804e7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50024/0 (socket says 172.21.15.60:50024) 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f705ffff640 1 -- 172.21.15.60:0/3153405184 learned_addr learned my addr 172.21.15.60:0/3153405184 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f705f7fe640 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514f70 0x55bc80517920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.345+0000 7f7064858640 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc804e8070 0x55bc80517e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f705ffff640 1 -- 172.21.15.60:0/3153405184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514f70 msgr2=0x55bc80517920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f705ffff640 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514f70 0x55bc80517920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f705ffff640 1 -- 172.21.15.60:0/3153405184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc804e8070 msgr2=0x55bc80517e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f705ffff640 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc804e8070 0x55bc80517e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f705ffff640 1 -- 172.21.15.60:0/3153405184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc80419500 con 0x55bc80514b90 2023-11-18T20:44:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f705f7fe640 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514f70 0x55bc80517920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7064858640 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc804e8070 0x55bc80517e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f705ffff640 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804e7b30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7050009990 tx=0x7f7050030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1526c934762c5e06 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7045ffb640 1 -- 172.21.15.60:0/3153405184 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705003a660 con 0x55bc80514b90 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7045ffb640 1 -- 172.21.15.60:0/3153405184 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705003a7c0 con 0x55bc80514b90 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7045ffb640 1 -- 172.21.15.60:0/3153405184 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705003a930 con 0x55bc80514b90 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7068e52740 1 -- 172.21.15.60:0/3153405184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 msgr2=0x55bc804e7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7068e52740 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804e7b30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7050009990 tx=0x7f7050030000 comp rx=0 tx=0).stop 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7068e52740 1 -- 172.21.15.60:0/3153405184 shutdown_connections 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7068e52740 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc804e8070 0x55bc80517e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7068e52740 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514f70 0x55bc80517920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7068e52740 1 --2- 172.21.15.60:0/3153405184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804e7b30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.346+0000 7f7068e52740 1 -- 172.21.15.60:0/3153405184 >> 172.21.15.60:0/3153405184 conn(0x55bc804e9da0 msgr2=0x55bc80532920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.347+0000 7f7068e52740 1 -- 172.21.15.60:0/3153405184 shutdown_connections 2023-11-18T20:44:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.347+0000 7f7068e52740 1 -- 172.21.15.60:0/3153405184 wait complete. 2023-11-18T20:44:38.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.347+0000 7f7068e52740 1 -- start start 2023-11-18T20:44:38.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.347+0000 7f7068e52740 1 Processor -- start 2023-11-18T20:44:38.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f7068e52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80514b90 0x55bc80519810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f7068e52740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80502860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f7068e52740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80533320 0x55bc80502da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f7068e52740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc80453b70 con 0x55bc80514f70 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f7068e52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8040de00 con 0x55bc80514b90 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f7068e52740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc8040daa0 con 0x55bc80533320 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f7068e52740 10 client.0 _ll_register_callbacks cb 0x55bc80165ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f705f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80502860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f705f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80502860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50026/0 (socket says 172.21.15.60:50026) 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f705f7fe640 1 -- 172.21.15.60:0/1575669330 learned_addr learned my addr 172.21.15.60:0/1575669330 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f705ffff640 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80514b90 0x55bc80519810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f7064858640 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80533320 0x55bc80502da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f705f7fe640 1 -- 172.21.15.60:0/1575669330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80533320 msgr2=0x55bc80502da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f705f7fe640 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80533320 0x55bc80502da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f705f7fe640 1 -- 172.21.15.60:0/1575669330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80514b90 msgr2=0x55bc80519810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.348+0000 7f705f7fe640 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80514b90 0x55bc80519810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f705f7fe640 1 -- 172.21.15.60:0/1575669330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706015e760 con 0x55bc80514f70 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f7064858640 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80533320 0x55bc80502da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f705f7fe640 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80502860 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f705400e280 tx=0x7f705400e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294141c85187096d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f703effd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f703effd640 1 -- 172.21.15.60:0/1575669330 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054013660 con 0x55bc80514f70 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f7068e52740 1 -- 172.21.15.60:0/1575669330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc8044cb50 con 0x55bc80514f70 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f703effd640 1 -- 172.21.15.60:0/1575669330 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70540137c0 con 0x55bc80514f70 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f703effd640 1 -- 172.21.15.60:0/1575669330 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7054013980 con 0x55bc80514f70 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f7068e52740 10 client.6024 fetch_fsmap learned FSMap version 21 2023-11-18T20:44:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f7068e52740 1 -- 172.21.15.60:0/1575669330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55bc804521f0 con 0x55bc80514f70 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f703effd640 1 -- 172.21.15.60:0/1575669330 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc8044cb50 con 0x55bc80514f70 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f703effd640 1 -- 172.21.15.60:0/1575669330 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7054011070 con 0x55bc80514f70 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f7068e52740 10 client.6024 fetch_fsmap finished waiting for FSMap version 21 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f7068e52740 10 client.6024 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f7068e52740 20 client.6024 populate_metadata read hostname 'smithi060' 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f7068e52740 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.60:6834/3098423505,v1:172.21.15.60:6835/3098423505] conn(0x55bc8054dfb0 0x55bc805503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f7068e52740 4 client.6024 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.349+0000 7f7068e52740 1 -- 172.21.15.60:0/1575669330 --> [v2:172.21.15.60:6834/3098423505,v1:172.21.15.60:6835/3098423505] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc8044c870 con 0x55bc8054dfb0 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.350+0000 7f705ffff640 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.60:6834/3098423505,v1:172.21.15.60:6835/3098423505] conn(0x55bc8054dfb0 0x55bc805503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.350+0000 7f705ffff640 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.60:6834/3098423505,v1:172.21.15.60:6835/3098423505] conn(0x55bc8054dfb0 0x55bc805503a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f0e1f743445ae6 server_cookie=46a21234da2b6fb9 in_seq=0 out_seq=0 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.350+0000 7f703effd640 10 client.6024 ms_handle_connect on v2:172.21.15.60:6834/3098423505 2023-11-18T20:44:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.350+0000 7f703effd640 1 -- 172.21.15.60:0/1575669330 <== mds.0 v2:172.21.15.60:6834/3098423505 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc8044c870 con 0x55bc8054dfb0 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.350+0000 7f703effd640 10 client.6024 handle_command_reply: tid=0 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.350+0000 7f7068e52740 1 client.6024 shutdown 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 1 -- 172.21.15.60:0/1575669330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 msgr2=0x55bc80502860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80502860 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f705400e280 tx=0x7f705400e740 comp rx=0 tx=0).stop 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 1 -- 172.21.15.60:0/1575669330 shutdown_connections 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.60:6834/3098423505,v1:172.21.15.60:6835/3098423505] conn(0x55bc8054dfb0 0x55bc805503a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80533320 0x55bc80502da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f703effd640 0 client.6024 ms_handle_reset on v2:172.21.15.60:6834/3098423505 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80502860 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 1 --2- 172.21.15.60:0/1575669330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80514b90 0x55bc80519810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 1 -- 172.21.15.60:0/1575669330 >> 172.21.15.60:0/1575669330 conn(0x55bc804e8070 msgr2=0x55bc804eba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 1 -- 172.21.15.60:0/1575669330 shutdown_connections 2023-11-18T20:44:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 1 -- 172.21.15.60:0/1575669330 wait complete. 2023-11-18T20:44:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.351+0000 7f7068e52740 20 client.6024 trim_cache size 0 max 16384 2023-11-18T20:44:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.366+0000 7f7068e52740 1 Processor -- start 2023-11-18T20:44:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.366+0000 7f7068e52740 1 -- start start 2023-11-18T20:44:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f7068e52740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804eb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f7068e52740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514f70 0x55bc80517a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f7068e52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80533320 0x55bc80512a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f7068e52740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc803d3270 con 0x55bc80514b90 2023-11-18T20:44:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f7068e52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc80517670 con 0x55bc80533320 2023-11-18T20:44:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f7068e52740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc8051a0f0 con 0x55bc80514f70 2023-11-18T20:44:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f705ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f705ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804eb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50034/0 (socket says 172.21.15.60:50034) 2023-11-18T20:44:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f705ffff640 1 -- 172.21.15.60:0/2076690220 learned_addr learned my addr 172.21.15.60:0/2076690220 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f705ffff640 1 -- 172.21.15.60:0/2076690220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514f70 msgr2=0x55bc80517a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f705ffff640 1 --2- 172.21.15.60:0/2076690220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514f70 0x55bc80517a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f705ffff640 1 -- 172.21.15.60:0/2076690220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80533320 msgr2=0x55bc80512a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.367+0000 7f705ffff640 1 --2- 172.21.15.60:0/2076690220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80533320 0x55bc80512a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f705ffff640 1 -- 172.21.15.60:0/2076690220 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc803efd00 con 0x55bc80514b90 2023-11-18T20:44:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f705ffff640 1 --2- 172.21.15.60:0/2076690220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804eb240 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7050046000 tx=0x7f7050009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e29845d56bd724d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f70467fc640 1 -- 172.21.15.60:0/2076690220 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705003a660 con 0x55bc80514b90 2023-11-18T20:44:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f70467fc640 1 -- 172.21.15.60:0/2076690220 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70500041c0 con 0x55bc80514b90 2023-11-18T20:44:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f70467fc640 1 -- 172.21.15.60:0/2076690220 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705003a930 con 0x55bc80514b90 2023-11-18T20:44:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f7068e52740 1 -- 172.21.15.60:0/2076690220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 msgr2=0x55bc804eb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f7068e52740 1 --2- 172.21.15.60:0/2076690220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804eb240 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7050046000 tx=0x7f7050009240 comp rx=0 tx=0).stop 2023-11-18T20:44:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f7068e52740 1 -- 172.21.15.60:0/2076690220 shutdown_connections 2023-11-18T20:44:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f7068e52740 1 --2- 172.21.15.60:0/2076690220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80533320 0x55bc80512a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f7068e52740 1 --2- 172.21.15.60:0/2076690220 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514f70 0x55bc80517a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f7068e52740 1 --2- 172.21.15.60:0/2076690220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514b90 0x55bc804eb240 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f7068e52740 1 -- 172.21.15.60:0/2076690220 >> 172.21.15.60:0/2076690220 conn(0x55bc804e8070 msgr2=0x55bc804e6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f7068e52740 1 -- 172.21.15.60:0/2076690220 shutdown_connections 2023-11-18T20:44:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.368+0000 7f7068e52740 1 -- 172.21.15.60:0/2076690220 wait complete. 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.369+0000 7f7068e52740 1 -- start start 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.369+0000 7f7068e52740 1 Processor -- start 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f7068e52740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514b90 0x55bc805970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f7068e52740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80552710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f7068e52740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80533320 0x55bc8054bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f7068e52740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc805133b0 con 0x55bc80514f70 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f7068e52740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc80501750 con 0x55bc80533320 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f7068e52740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc805346f0 con 0x55bc80514b90 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f7068e52740 10 client.0 _ll_register_callbacks cb 0x7f70500096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f705f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80552710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f705f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80552710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50042/0 (socket says 172.21.15.60:50042) 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f705f7fe640 1 -- 172.21.15.60:0/962705342 learned_addr learned my addr 172.21.15.60:0/962705342 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f7064858640 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80533320 0x55bc8054bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f705ffff640 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514b90 0x55bc805970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f705f7fe640 1 -- 172.21.15.60:0/962705342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514b90 msgr2=0x55bc805970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f705f7fe640 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514b90 0x55bc805970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f705f7fe640 1 -- 172.21.15.60:0/962705342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80533320 msgr2=0x55bc8054bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f705f7fe640 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80533320 0x55bc8054bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f705f7fe640 1 -- 172.21.15.60:0/962705342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc80419500 con 0x55bc80514f70 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f705f7fe640 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80552710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7054013a70 tx=0x7f70540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbf0af791d9869c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.370+0000 7f703effd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f703effd640 1 -- 172.21.15.60:0/962705342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400cb90 con 0x55bc80514f70 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f703effd640 1 -- 172.21.15.60:0/962705342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7054014600 con 0x55bc80514f70 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f7068e52740 1 -- 172.21.15.60:0/962705342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f705003a7c0 con 0x55bc80514f70 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f703effd640 1 -- 172.21.15.60:0/962705342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f705400ec40 con 0x55bc80514f70 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f7068e52740 10 client.6030 fetch_fsmap learned FSMap version 21 2023-11-18T20:44:38.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f7068e52740 1 -- 172.21.15.60:0/962705342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55bc804154f0 con 0x55bc80514f70 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f703effd640 1 -- 172.21.15.60:0/962705342 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705003a7c0 con 0x55bc80514f70 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f703effd640 1 -- 172.21.15.60:0/962705342 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f70540059c0 con 0x55bc80514f70 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f7068e52740 10 client.6030 fetch_fsmap finished waiting for FSMap version 21 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f7068e52740 10 client.6030 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f7068e52740 20 client.6030 populate_metadata read hostname 'smithi060' 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f7068e52740 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.60:6834/3098423505,v1:172.21.15.60:6835/3098423505] conn(0x55bc80545130 0x55bc8059d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f7068e52740 4 client.6030 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f7068e52740 1 -- 172.21.15.60:0/962705342 --> [v2:172.21.15.60:6834/3098423505,v1:172.21.15.60:6835/3098423505] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bc801546c0 con 0x55bc80545130 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.371+0000 7f705ffff640 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.60:6834/3098423505,v1:172.21.15.60:6835/3098423505] conn(0x55bc80545130 0x55bc8059d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f703effd640 10 client.6030 ms_handle_connect on v2:172.21.15.60:6834/3098423505 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f705ffff640 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.60:6834/3098423505,v1:172.21.15.60:6835/3098423505] conn(0x55bc80545130 0x55bc8059d030 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1752b1c80f17314 server_cookie=adf276eddaf4ea6a in_seq=0 out_seq=0 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f703effd640 1 -- 172.21.15.60:0/962705342 <== mds.0 v2:172.21.15.60:6834/3098423505 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bc801546c0 con 0x55bc80545130 2023-11-18T20:44:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f703effd640 10 client.6030 handle_command_reply: tid=0 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 client.6030 shutdown 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 -- 172.21.15.60:0/962705342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 msgr2=0x55bc80552710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80552710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7054013a70 tx=0x7f70540046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 -- 172.21.15.60:0/962705342 shutdown_connections 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.60:6834/3098423505,v1:172.21.15.60:6835/3098423505] conn(0x55bc80545130 0x55bc8059d030 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc80533320 0x55bc8054bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bc80514f70 0x55bc80552710 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f703effd640 0 client.6030 ms_handle_reset on v2:172.21.15.60:6834/3098423505 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 --2- 172.21.15.60:0/962705342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bc80514b90 0x55bc805970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 -- 172.21.15.60:0/962705342 >> 172.21.15.60:0/962705342 conn(0x55bc804e8070 msgr2=0x55bc80541720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 -- 172.21.15.60:0/962705342 shutdown_connections 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 1 -- 172.21.15.60:0/962705342 wait complete. 2023-11-18T20:44:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.372+0000 7f7068e52740 20 client.6030 trim_cache size 0 max 16384 2023-11-18T20:44:38.380 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:44:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "ae4682cb-f89a-4edd-b6d6-55eae03fd1b8", 2023-11-18T20:44:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2023-11-18T20:44:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5886, 2023-11-18T20:44:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2023-11-18T20:44:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2023-11-18T20:44:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2023-11-18T20:44:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.6787033830000002, 2023-11-18T20:44:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 21, 2023-11-18T20:44:38.380 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 49, 2023-11-18T20:44:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 49, 2023-11-18T20:44:38.381 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 9.6879797100000005 2023-11-18T20:44:38.381 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:44:38.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 -- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f704c06ae70 msgr2=0x7f704c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f704c06ae70 0x7f704c06d330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7050036000 tx=0x7f705003c040 comp rx=0 tx=0).stop 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 -- 172.21.15.60:0/287150855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7060160fb0 msgr2=0x7f7060158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7060160fb0 0x7f7060158010 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f705800b9e0 tx=0x7f705800bea0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 -- 172.21.15.60:0/287150855 shutdown_connections 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f704c06ae70 0x7f704c06d330 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7060160fb0 0x7f7060158010 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70600b3ff0 0x7f706015a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 --2- 172.21.15.60:0/287150855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706006ea50 0x7f706013ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 -- 172.21.15.60:0/287150855 >> 172.21.15.60:0/287150855 conn(0x7f706006a6b0 msgr2=0x7f706015ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 -- 172.21.15.60:0/287150855 shutdown_connections 2023-11-18T20:44:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.373+0000 7f70662bb640 1 -- 172.21.15.60:0/287150855 wait complete. 2023-11-18T20:44:38.385 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-18T20:44:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.490+0000 7f207e44b640 1 Processor -- start 2023-11-18T20:44:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.490+0000 7f207e44b640 1 -- start start 2023-11-18T20:44:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f207e44b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20781415f0 0x7f2078141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f207e44b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f207806bbf0 0x7f20780b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f207e44b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20780b2ef0 con 0x7f2078144a00 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f207e44b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20780b3b70 con 0x7f20781415f0 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f207e44b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20780b31d0 con 0x7f207806bbf0 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f20777fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2078144a00 0x7f207806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55432/0 (socket says 172.21.15.60:55432) 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f20777fe640 1 -- 172.21.15.60:0/1139961796 learned_addr learned my addr 172.21.15.60:0/1139961796 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f207c9e8640 1 --2- 172.21.15.60:0/1139961796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f207806bbf0 0x7f20780b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f2077fff640 1 --2- 172.21.15.60:0/1139961796 >> v2:172.21.15.100:3300/0 conn(0x7f20781415f0 0x7f2078141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f2076ffd640 1 -- 172.21.15.60:0/1139961796 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810254737 0 0) 0x7f20780b2ef0 con 0x7f2078144a00 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.491+0000 7f2076ffd640 1 -- 172.21.15.60:0/1139961796 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2060003520 con 0x7f2078144a00 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.492+0000 7f2077fff640 1 -- 172.21.15.60:0/1139961796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f207806bbf0 msgr2=0x7f20780b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.492+0000 7f2076ffd640 1 -- 172.21.15.60:0/1139961796 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1954665912 0 0) 0x7f2060003520 con 0x7f2078144a00 2023-11-18T20:44:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.492+0000 7f2077fff640 1 --2- 172.21.15.60:0/1139961796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f207806bbf0 0x7f20780b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.492+0000 7f2077fff640 1 -- 172.21.15.60:0/1139961796 >> v1:172.21.15.60:6789/0 conn(0x7f2078144a00 legacy=0x7f207806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.492+0000 7f2077fff640 1 -- 172.21.15.60:0/1139961796 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780b34b0 con 0x7f20781415f0 2023-11-18T20:44:38.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.492+0000 7f207c9e8640 1 --2- 172.21.15.60:0/1139961796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f207806bbf0 0x7f20780b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:38.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f2077fff640 1 --2- 172.21.15.60:0/1139961796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20781415f0 0x7f2078141e00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f206c0097a0 tx=0x7f206c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5c704d763057b09 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:38.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f2076ffd640 1 -- 172.21.15.60:0/1139961796 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068001670 con 0x7f20781415f0 2023-11-18T20:44:38.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f2076ffd640 1 -- 172.21.15.60:0/1139961796 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c02e050 con 0x7f20781415f0 2023-11-18T20:44:38.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f2076ffd640 1 -- 172.21.15.60:0/1139961796 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c030710 con 0x7f20781415f0 2023-11-18T20:44:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f207e44b640 1 -- 172.21.15.60:0/1139961796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20781415f0 msgr2=0x7f2078141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f207e44b640 1 --2- 172.21.15.60:0/1139961796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20781415f0 0x7f2078141e00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f206c0097a0 tx=0x7f206c02b790 comp rx=0 tx=0).stop 2023-11-18T20:44:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f207e44b640 1 -- 172.21.15.60:0/1139961796 shutdown_connections 2023-11-18T20:44:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f207e44b640 1 --2- 172.21.15.60:0/1139961796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f207806bbf0 0x7f20780b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f207e44b640 1 --2- 172.21.15.60:0/1139961796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20781415f0 0x7f2078141e00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f207e44b640 1 -- 172.21.15.60:0/1139961796 >> 172.21.15.60:0/1139961796 conn(0x7f2078070e70 msgr2=0x7f2078071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f207e44b640 1 -- 172.21.15.60:0/1139961796 shutdown_connections 2023-11-18T20:44:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.493+0000 7f207e44b640 1 -- 172.21.15.60:0/1139961796 wait complete. 2023-11-18T20:44:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.495+0000 7f207e44b640 1 Processor -- start 2023-11-18T20:44:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.495+0000 7f207e44b640 1 -- start start 2023-11-18T20:44:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.495+0000 7f207e44b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f207806bbf0 0x7f2078134820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.495+0000 7f207e44b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20781415f0 0x7f20781f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.495+0000 7f207e44b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2078144a00 0x7f20781f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.495+0000 7f207e44b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078134d60 con 0x7f207806bbf0 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.495+0000 7f207e44b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078155d50 con 0x7f2078144a00 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.495+0000 7f207e44b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2078156030 con 0x7f20781415f0 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2077fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f207806bbf0 0x7f2078134820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2077fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f207806bbf0 0x7f2078134820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50044/0 (socket says 172.21.15.60:50044) 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2077fff640 1 -- 172.21.15.60:0/2743555488 learned_addr learned my addr 172.21.15.60:0/2743555488 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f20777fe640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20781415f0 0x7f20781f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f207c9e8640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2078144a00 0x7f20781f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2077fff640 1 -- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20781415f0 msgr2=0x7f20781f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2077fff640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20781415f0 0x7f20781f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2077fff640 1 -- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2078144a00 msgr2=0x7f20781f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2077fff640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2078144a00 0x7f20781f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2077fff640 1 -- 172.21.15.60:0/2743555488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070009a70 con 0x7f207806bbf0 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2077fff640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f207806bbf0 0x7f2078134820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f206c000c00 tx=0x7f206c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2074ff9640 1 -- 172.21.15.60:0/2743555488 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c006e10 con 0x7f207806bbf0 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2074ff9640 1 -- 172.21.15.60:0/2743555488 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c03c620 con 0x7f207806bbf0 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f207e44b640 1 -- 172.21.15.60:0/2743555488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c009400 con 0x7f207806bbf0 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f2074ff9640 1 -- 172.21.15.60:0/2743555488 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c03c9b0 con 0x7f207806bbf0 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.496+0000 7f207e44b640 1 -- 172.21.15.60:0/2743555488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20781565f0 con 0x7f207806bbf0 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.497+0000 7f207e44b640 1 -- 172.21.15.60:0/2743555488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2078069400 con 0x7f207806bbf0 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.498+0000 7f2074ff9640 1 -- 172.21.15.60:0/2743555488 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f206c039070 con 0x7f207806bbf0 2023-11-18T20:44:38.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.498+0000 7f2074ff9640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f206406ae40 0x7f206406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:38.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.498+0000 7f2074ff9640 1 -- 172.21.15.60:0/2743555488 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f206c0aabd0 con 0x7f207806bbf0 2023-11-18T20:44:38.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.499+0000 7f20777fe640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f206406ae40 0x7f206406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:38.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.499+0000 7f20777fe640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f206406ae40 0x7f206406d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2068001300 tx=0x7f2068001290 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:38.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.503+0000 7f2074ff9640 1 -- 172.21.15.60:0/2743555488 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f206c075d20 con 0x7f207806bbf0 2023-11-18T20:44:38.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.716+0000 7f207e44b640 1 -- 172.21.15.60:0/2743555488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2078066760 con 0x7f207806bbf0 2023-11-18T20:44:38.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.717+0000 7f2074ff9640 1 -- 172.21.15.60:0/2743555488 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7f206c037070 con 0x7f207806bbf0 2023-11-18T20:44:38.718 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-18T20:44:38.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 -- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f206406ae40 msgr2=0x7f206406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f206406ae40 0x7f206406d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2068001300 tx=0x7f2068001290 comp rx=0 tx=0).stop 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 -- 172.21.15.60:0/2743555488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f207806bbf0 msgr2=0x7f2078134820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f207806bbf0 0x7f2078134820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f206c000c00 tx=0x7f206c0046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 -- 172.21.15.60:0/2743555488 shutdown_connections 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f206406ae40 0x7f206406d300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2078144a00 0x7f20781f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20781415f0 0x7f20781f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 --2- 172.21.15.60:0/2743555488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f207806bbf0 0x7f2078134820 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 -- 172.21.15.60:0/2743555488 >> 172.21.15.60:0/2743555488 conn(0x7f2078070e70 msgr2=0x7f20781419d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 -- 172.21.15.60:0/2743555488 shutdown_connections 2023-11-18T20:44:38.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:38.719+0000 7f207e44b640 1 -- 172.21.15.60:0/2743555488 wait complete. 2023-11-18T20:44:38.731 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-18T20:44:38.731 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-18T20:44:38.731 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-11-18T20:44:38.731 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-18T20:44:38.731 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:38.731 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-18T20:44:38.732 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-18T20:44:38.732 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-18T20:44:38.732 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:38.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:38.732 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42577sec preferred_lft 42577sec 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:44:38.749 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::d896:cfff:fef3:88dc/64 scope link 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0a:c3:0c:c2:e6:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8c3:cff:fec2:e672/64 scope link 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:38.750 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::40c:deff:fe9d:c1f6/64 scope link 2023-11-18T20:44:38.751 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:38.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:38.751 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-18T20:44:38.817 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-11-18T20:44:38.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:38.819 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-11-18T20:44:38.875 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-11-18T20:44:38.875 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-18T20:44:38.876 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-18T20:44:38.876 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-18T20:44:38.876 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-18T20:44:38.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:38.876 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:44:38.876 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:44:38.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-18T20:44:38.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-18T20:44:38.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-18T20:44:38.877 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:44:38.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:38.982 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-11-18T20:44:38.997 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-18T20:44:38.997 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-18T20:44:38.998 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:38.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:38.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-11-18T20:44:39.068 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-18T20:44:39.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:44:39.069 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:39.124 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:39.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:44:39.124 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:39.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:44:39.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:44:39.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:44:39.250 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:44:39.327 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:44:39.327 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:44:39.328 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:44:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:44:39.330 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:44:39.330 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:44:39.330 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:44:39.330 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:39.330 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:39.330 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:39.330 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:39.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:39.331 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:44:39.382 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-18T20:44:39.383 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-18T20:44:39.425 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:44:39.455 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:44:39.511 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:44:39.511 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:44:39.511 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:44:39.512 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:44:39.512 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:44:39.512 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:44:39.512 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:44:39.512 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:44:39.512 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:44:39.512 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:44:39.512 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:44:39.512 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:44:39.512 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:44:39.513 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:44:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:44:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:44:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:44:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:44:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:44:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:44:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:39.514 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:39.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:39.515 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:44:39.517 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-11-18T20:44:39.516+0000 7fbbf7c0b480 -1 init, newargv = 0x559eb9675040 newargc=15 2023-11-18T20:44:39.517 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[106639]: starting ceph client 2023-11-18T20:44:39.522 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[106639]: starting fuse 2023-11-18T20:44:39.532 INFO:teuthology.orchestra.run.smithi060.stdout:41 2023-11-18T20:44:39.533 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-18T20:44:39.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2023-11-18T20:44:39.533 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-11-18T20:44:39.533 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-11-18T20:44:39.533 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-11-18T20:44:39.533 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-11-18T20:44:39.533 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:39.533 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-11-18T20:44:39.533 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-18T20:44:39.534 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-18T20:44:39.535 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-11-18T20:44:39.535 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-11-18T20:44:39.535 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-11-18T20:44:39.535 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-18T20:44:39.535 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:39.535 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-11-18T20:44:39.535 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-11-18T20:44:39.659 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.106639.asok 2023-11-18T20:44:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:44:39.670 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106639.asok 2023-11-18T20:44:39.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:39.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106639.asok status 2023-11-18T20:44:39.812 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:44:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-11-18T20:44:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe", 2023-11-18T20:44:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-7418-gb0f45a51 (b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe) reef (dev)", 2023-11-18T20:44:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-11-18T20:44:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-11-18T20:44:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-18T20:44:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "106639", 2023-11-18T20:44:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-11-18T20:44:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-11-18T20:44:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-11-18T20:44:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6045, 2023-11-18T20:44:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-11-18T20:44:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-11-18T20:44:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-11-18T20:44:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6045 2023-11-18T20:44:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:44:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:44:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 132026017 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 132026017 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6045 192.168.0.1:0/132026017", 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/132026017", 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-11-18T20:44:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 21, 2023-11-18T20:44:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 49, 2023-11-18T20:44:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-11-18T20:44:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-11-18T20:44:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-11-18T20:44:39.817 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:44:39.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:39.821 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-11-18T20:44:39.821 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:39.836 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-11-18T20:44:39.837 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:39.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:39.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:39.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:39.909 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2023-11-18T20:44:39.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2023-11-18T20:44:40.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.122+0000 7fe3cef97640 1 Processor -- start 2023-11-18T20:44:40.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.122+0000 7fe3cef97640 1 -- start start 2023-11-18T20:44:40.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.122+0000 7fe3cef97640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe3c815d1f0 0x7fe3c80b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.122+0000 7fe3cef97640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c80b1ae0 0x7fe3c806d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.122+0000 7fe3cef97640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3c806d6a0 0x7fe3c80c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.122+0000 7fe3cef97640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c806db30 con 0x7fe3c806d6a0 2023-11-18T20:44:40.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.122+0000 7fe3cef97640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c8161b40 con 0x7fe3c815d1f0 2023-11-18T20:44:40.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.122+0000 7fe3cef97640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8160f30 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3cd534640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3c806d6a0 0x7fe3c80c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3cd534640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3c806d6a0 0x7fe3c80c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50056/0 (socket says 172.21.15.60:50056) 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3cd534640 1 -- 172.21.15.60:0/1357923193 learned_addr learned my addr 172.21.15.60:0/1357923193 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3cd534640 1 -- 172.21.15.60:0/1357923193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c80b1ae0 msgr2=0x7fe3c806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3c7fff640 1 --2- 172.21.15.60:0/1357923193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c80b1ae0 0x7fe3c806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3ccd33640 1 --2- 172.21.15.60:0/1357923193 >> v2:172.21.15.100:3300/0 conn(0x7fe3c815d1f0 0x7fe3c80b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3cd534640 1 --2- 172.21.15.60:0/1357923193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c80b1ae0 0x7fe3c806d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3cd534640 1 -- 172.21.15.60:0/1357923193 >> v2:172.21.15.100:3300/0 conn(0x7fe3c815d1f0 msgr2=0x7fe3c80b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3cd534640 1 --2- 172.21.15.60:0/1357923193 >> v2:172.21.15.100:3300/0 conn(0x7fe3c815d1f0 0x7fe3c80b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3cd534640 1 -- 172.21.15.60:0/1357923193 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8161210 con 0x7fe3c806d6a0 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3c7fff640 1 --2- 172.21.15.60:0/1357923193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c80b1ae0 0x7fe3c806d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.123+0000 7fe3ccd33640 1 --2- 172.21.15.60:0/1357923193 >> v2:172.21.15.100:3300/0 conn(0x7fe3c815d1f0 0x7fe3c80b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:40.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3cd534640 1 --2- 172.21.15.60:0/1357923193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c806d6a0 0x7fe3c80c8510 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80d0290 tx=0x7fe3c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a47838bf12b50a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:40.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3c77fe640 1 -- 172.21.15.60:0/1357923193 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c002ca70 con 0x7fe3c806d6a0 2023-11-18T20:44:40.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3c77fe640 1 -- 172.21.15.60:0/1357923193 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c002cbd0 con 0x7fe3c806d6a0 2023-11-18T20:44:40.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3c77fe640 1 -- 172.21.15.60:0/1357923193 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00343f0 con 0x7fe3c806d6a0 2023-11-18T20:44:40.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3cef97640 1 -- 172.21.15.60:0/1357923193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c806d6a0 msgr2=0x7fe3c80c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3cef97640 1 --2- 172.21.15.60:0/1357923193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c806d6a0 0x7fe3c80c8510 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80d0290 tx=0x7fe3c002bed0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.129 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3cef97640 1 -- 172.21.15.60:0/1357923193 shutdown_connections 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3cef97640 1 --2- 172.21.15.60:0/1357923193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c806d6a0 0x7fe3c80c8510 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3cef97640 1 --2- 172.21.15.60:0/1357923193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c80b1ae0 0x7fe3c806d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3cef97640 1 --2- 172.21.15.60:0/1357923193 >> v2:172.21.15.100:3300/0 conn(0x7fe3c815d1f0 0x7fe3c80b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.124+0000 7fe3cef97640 1 -- 172.21.15.60:0/1357923193 >> 172.21.15.60:0/1357923193 conn(0x7fe3c8070d80 msgr2=0x7fe3c80b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.125+0000 7fe3cef97640 1 -- 172.21.15.60:0/1357923193 shutdown_connections 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.125+0000 7fe3cef97640 1 -- 172.21.15.60:0/1357923193 wait complete. 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.126+0000 7fe3cef97640 1 Processor -- start 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.126+0000 7fe3cef97640 1 -- start start 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3cef97640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c806d6a0 0x7fe3c814db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3cef97640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c80b1ae0 0x7fe3c815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.130 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3cef97640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c815d1f0 0x7fe3c80cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3cef97640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c815a280 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3cef97640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80ca8d0 con 0x7fe3c815d1f0 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3cef97640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c815bd60 con 0x7fe3c806d6a0 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c80b1ae0 0x7fe3c815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c80b1ae0 0x7fe3c815a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50058/0 (socket says 172.21.15.60:50058) 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c7fff640 1 -- 172.21.15.60:0/2018643439 learned_addr learned my addr 172.21.15.60:0/2018643439 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3ccd33640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c806d6a0 0x7fe3c814db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3cd534640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c815d1f0 0x7fe3c80cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c7fff640 1 -- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c806d6a0 msgr2=0x7fe3c814db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c7fff640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c806d6a0 0x7fe3c814db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c7fff640 1 -- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c815d1f0 msgr2=0x7fe3c80cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c7fff640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c815d1f0 0x7fe3c80cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c7fff640 1 -- 172.21.15.60:0/2018643439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c0009b30 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3cd534640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c815d1f0 0x7fe3c80cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:40.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c7fff640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c80b1ae0 0x7fe3c815a6a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400c7c0 tx=0x7fe3b400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:40.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c57fa640 1 -- 172.21.15.60:0/2018643439 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4007820 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c57fa640 1 -- 172.21.15.60:0/2018643439 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b4007980 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3cef97640 1 -- 172.21.15.60:0/2018643439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c8159c00 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3c57fa640 1 -- 172.21.15.60:0/2018643439 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4007d10 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.127+0000 7fe3cef97640 1 -- 172.21.15.60:0/2018643439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c8159ea0 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.129+0000 7fe3cef97640 1 -- 172.21.15.60:0/2018643439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c815a3c0 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.129+0000 7fe3c57fa640 1 -- 172.21.15.60:0/2018643439 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3b40113f0 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.129+0000 7fe3c57fa640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3ac06ae40 0x7fe3ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.129+0000 7fe3c57fa640 1 -- 172.21.15.60:0/2018643439 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3b4089600 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.129+0000 7fe3ccd33640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3ac06ae40 0x7fe3ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.130+0000 7fe3ccd33640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3ac06ae40 0x7fe3ac06d300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00aac0 tx=0x7fe3bc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:40.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.134+0000 7fe3c57fa640 1 -- 172.21.15.60:0/2018643439 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3c815a3c0 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.375+0000 7fe3cef97640 1 -- 172.21.15.60:0/2018643439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) v1 -- 0x7fe3c815a3c0 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.377+0000 7fe3c57fa640 1 -- 172.21.15.60:0/2018643439 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v18) v1 ==== 183+0+140 (secure 0 0 0) 0x7fe3c815a3c0 con 0x7fe3c80b1ae0 2023-11-18T20:44:40.378 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-11-18T20:44:40.378 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA== 2023-11-18T20:44:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-11-18T20:44:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-11-18T20:44:40.379 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow r pool=cephfs_data" 2023-11-18T20:44:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-11-18T20:44:40.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 -- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3ac06ae40 msgr2=0x7fe3ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3ac06ae40 0x7fe3ac06d300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00aac0 tx=0x7fe3bc008040 comp rx=0 tx=0).stop 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 -- 172.21.15.60:0/2018643439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c80b1ae0 msgr2=0x7fe3c815a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c80b1ae0 0x7fe3c815a6a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400c7c0 tx=0x7fe3b400cc80 comp rx=0 tx=0).stop 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 -- 172.21.15.60:0/2018643439 shutdown_connections 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3ac06ae40 0x7fe3ac06d300 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3c815d1f0 0x7fe3c80cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3c80b1ae0 0x7fe3c815a6a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 --2- 172.21.15.60:0/2018643439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3c806d6a0 0x7fe3c814db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 -- 172.21.15.60:0/2018643439 >> 172.21.15.60:0/2018643439 conn(0x7fe3c8070d80 msgr2=0x7fe3c80c8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 -- 172.21.15.60:0/2018643439 shutdown_connections 2023-11-18T20:44:40.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.379+0000 7fe3cef97640 1 -- 172.21.15.60:0/2018643439 wait complete. 2023-11-18T20:44:40.391 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:40.391 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:40.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:40.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.551+0000 7f42d4c90640 1 Processor -- start 2023-11-18T20:44:40.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.551+0000 7f42d4c90640 1 -- start start 2023-11-18T20:44:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.552+0000 7f42d4c90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d015d330 0x7f42d00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.552+0000 7f42d4c90640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42d00d2910 0x7f42d00d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.552+0000 7f42d4c90640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f42d00d2d90 0x7f42d00d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.552+0000 7f42d4c90640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d00b2b10 con 0x7f42d00d2910 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.552+0000 7f42d4c90640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d00d1590 con 0x7f42d00d2d90 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.552+0000 7f42d4c90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00d1870 con 0x7f42d015d330 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42cdd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42d00d2910 0x7f42d00d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42cdd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f42d00d2910 0x7f42d00d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50066/0 (socket says 172.21.15.60:50066) 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42cdd9b640 1 -- 172.21.15.60:0/3618963246 learned_addr learned my addr 172.21.15.60:0/3618963246 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42ce59c640 1 --2- 172.21.15.60:0/3618963246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d015d330 0x7f42d00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42ced9d640 1 --2- 172.21.15.60:0/3618963246 >> v2:172.21.15.100:3300/0 conn(0x7f42d00d2d90 0x7f42d00d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42cdd9b640 1 -- 172.21.15.60:0/3618963246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d015d330 msgr2=0x7f42d00b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42cdd9b640 1 --2- 172.21.15.60:0/3618963246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d015d330 0x7f42d00b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42cdd9b640 1 -- 172.21.15.60:0/3618963246 >> v2:172.21.15.100:3300/0 conn(0x7f42d00d2d90 msgr2=0x7f42d00d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42cdd9b640 1 --2- 172.21.15.60:0/3618963246 >> v2:172.21.15.100:3300/0 conn(0x7f42d00d2d90 0x7f42d00d3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42cdd9b640 1 -- 172.21.15.60:0/3618963246 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d00d1b50 con 0x7f42d00d2910 2023-11-18T20:44:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42ce59c640 1 --2- 172.21.15.60:0/3618963246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d015d330 0x7f42d00b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:40.556 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.553+0000 7f42ced9d640 1 --2- 172.21.15.60:0/3618963246 >> v2:172.21.15.100:3300/0 conn(0x7f42d00d2d90 0x7f42d00d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42cdd9b640 1 --2- 172.21.15.60:0/3618963246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d00d2910 0x7f42d00d31f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f42d00d1540 tx=0x7f42b802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6955545d1cd7efe server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42cd59a640 1 -- 172.21.15.60:0/3618963246 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b80317e0 con 0x7f42d00d2910 2023-11-18T20:44:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42cd59a640 1 -- 172.21.15.60:0/3618963246 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b8031940 con 0x7f42d00d2910 2023-11-18T20:44:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42cd59a640 1 -- 172.21.15.60:0/3618963246 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b8031cd0 con 0x7f42d00d2910 2023-11-18T20:44:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42d4c90640 1 -- 172.21.15.60:0/3618963246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d00d2910 msgr2=0x7f42d00d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42d4c90640 1 --2- 172.21.15.60:0/3618963246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d00d2910 0x7f42d00d31f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f42d00d1540 tx=0x7f42b802b880 comp rx=0 tx=0).stop 2023-11-18T20:44:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42d4c90640 1 -- 172.21.15.60:0/3618963246 shutdown_connections 2023-11-18T20:44:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42d4c90640 1 --2- 172.21.15.60:0/3618963246 >> v2:172.21.15.100:3300/0 conn(0x7f42d00d2d90 0x7f42d00d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42d4c90640 1 --2- 172.21.15.60:0/3618963246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d00d2910 0x7f42d00d31f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42d4c90640 1 --2- 172.21.15.60:0/3618963246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d015d330 0x7f42d00b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.554+0000 7f42d4c90640 1 -- 172.21.15.60:0/3618963246 >> 172.21.15.60:0/3618963246 conn(0x7f42d0072020 msgr2=0x7f42d0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.555+0000 7f42d4c90640 1 -- 172.21.15.60:0/3618963246 shutdown_connections 2023-11-18T20:44:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.555+0000 7f42d4c90640 1 -- 172.21.15.60:0/3618963246 wait complete. 2023-11-18T20:44:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.556+0000 7f42d4c90640 1 Processor -- start 2023-11-18T20:44:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.556+0000 7f42d4c90640 1 -- start start 2023-11-18T20:44:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.556+0000 7f42d4c90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d00d2910 0x7f42d00c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42d4c90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d00d2d90 0x7f42d00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42d4c90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d015d330 0x7f42d015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42d4c90640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00c7870 con 0x7f42d015d330 2023-11-18T20:44:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42d4c90640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00c7b50 con 0x7f42d00d2d90 2023-11-18T20:44:40.560 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42d4c90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00c7e30 con 0x7f42d00d2910 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42ced9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d015d330 0x7f42d015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42ced9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d015d330 0x7f42d015f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50072/0 (socket says 172.21.15.60:50072) 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42ced9d640 1 -- 172.21.15.60:0/2516278446 learned_addr learned my addr 172.21.15.60:0/2516278446 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42cdd9b640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d00d2d90 0x7f42d00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42ce59c640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d00d2910 0x7f42d00c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42ced9d640 1 -- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d00d2910 msgr2=0x7f42d00c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42ced9d640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d00d2910 0x7f42d00c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42ced9d640 1 -- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d00d2d90 msgr2=0x7f42d00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42ced9d640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d00d2d90 0x7f42d00c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42ced9d640 1 -- 172.21.15.60:0/2516278446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b80094d0 con 0x7f42d015d330 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42cdd9b640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d00d2d90 0x7f42d00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42ced9d640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d015d330 0x7f42d015f910 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f42c0007de0 tx=0x7f42c000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42c6ffd640 1 -- 172.21.15.60:0/2516278446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0003b90 con 0x7f42d015d330 2023-11-18T20:44:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42c6ffd640 1 -- 172.21.15.60:0/2516278446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c000f040 con 0x7f42d015d330 2023-11-18T20:44:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42d4c90640 1 -- 172.21.15.60:0/2516278446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d00c8110 con 0x7f42d015d330 2023-11-18T20:44:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42c6ffd640 1 -- 172.21.15.60:0/2516278446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c000de80 con 0x7f42d015d330 2023-11-18T20:44:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.557+0000 7f42d4c90640 1 -- 172.21.15.60:0/2516278446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d00c7580 con 0x7f42d015d330 2023-11-18T20:44:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.558+0000 7f42d4c90640 1 -- 172.21.15.60:0/2516278446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d0069400 con 0x7f42d015d330 2023-11-18T20:44:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.559+0000 7f42c6ffd640 1 -- 172.21.15.60:0/2516278446 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f42c0004030 con 0x7f42d015d330 2023-11-18T20:44:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.559+0000 7f42c6ffd640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b006ae70 0x7f42b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.559+0000 7f42c6ffd640 1 -- 172.21.15.60:0/2516278446 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42c0089a00 con 0x7f42d015d330 2023-11-18T20:44:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.560+0000 7f42ce59c640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b006ae70 0x7f42b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.560+0000 7f42ce59c640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b006ae70 0x7f42b006d330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f42c8006fd0 tx=0x7f42c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:40.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.564+0000 7f42c6ffd640 1 -- 172.21.15.60:0/2516278446 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42c0054a30 con 0x7f42d015d330 2023-11-18T20:44:40.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.743+0000 7f42d4c90640 1 -- 172.21.15.60:0/2516278446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42d00c79b0 con 0x7f42d015d330 2023-11-18T20:44:40.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.744+0000 7f42c6ffd640 1 -- 172.21.15.60:0/2516278446 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f42d00c79b0 con 0x7f42d015d330 2023-11-18T20:44:40.745 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:40.746 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:34.103787+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:40.747 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 -- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b006ae70 msgr2=0x7f42b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b006ae70 0x7f42b006d330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f42c8006fd0 tx=0x7f42c8008040 comp rx=0 tx=0).stop 2023-11-18T20:44:40.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 -- 172.21.15.60:0/2516278446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d015d330 msgr2=0x7f42d015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d015d330 0x7f42d015f910 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f42c0007de0 tx=0x7f42c000d4a0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.748 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 -- 172.21.15.60:0/2516278446 shutdown_connections 2023-11-18T20:44:40.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f42b006ae70 0x7f42b006d330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d015d330 0x7f42d015f910 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d00d2d90 0x7f42d00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 --2- 172.21.15.60:0/2516278446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d00d2910 0x7f42d00c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 -- 172.21.15.60:0/2516278446 >> 172.21.15.60:0/2516278446 conn(0x7f42d0072020 msgr2=0x7f42d00b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:40.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 -- 172.21.15.60:0/2516278446 shutdown_connections 2023-11-18T20:44:40.749 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.746+0000 7f42d4c90640 1 -- 172.21.15.60:0/2516278446 wait complete. 2023-11-18T20:44:40.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:44:31.358325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:44:32.036538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:44:40.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:44:40.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.866+0000 7f3e82e15640 1 Processor -- start 2023-11-18T20:44:40.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.866+0000 7f3e82e15640 1 -- start start 2023-11-18T20:44:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.867+0000 7f3e82e15640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3e7c06abe0 0x7f3e7c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.867+0000 7f3e82e15640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3e7c1409d0 0x7f3e7c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.867+0000 7f3e82e15640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c1406f0 con 0x7f3e7c140310 2023-11-18T20:44:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.867+0000 7f3e82e15640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c161e20 con 0x7f3e7c1409d0 2023-11-18T20:44:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.867+0000 7f3e82e15640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c143960 con 0x7f3e7c06abe0 2023-11-18T20:44:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.867+0000 7f3e813b2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3e7c1409d0 0x7f3e7c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.867+0000 7f3e813b2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3e7c1409d0 0x7f3e7c06efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48654/0 (socket says 172.21.15.60:48654) 2023-11-18T20:44:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.867+0000 7f3e813b2640 1 -- 172.21.15.60:0/3999709343 learned_addr learned my addr 172.21.15.60:0/3999709343 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.868+0000 7f3e7bfff640 1 --2- 172.21.15.60:0/3999709343 >> v2:172.21.15.100:3300/0 conn(0x7f3e7c06abe0 0x7f3e7c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.868+0000 7f3e813b2640 1 -- 172.21.15.60:0/3999709343 >> v2:172.21.15.100:3300/0 conn(0x7f3e7c06abe0 msgr2=0x7f3e7c1618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.868+0000 7f3e813b2640 1 --2- 172.21.15.60:0/3999709343 >> v2:172.21.15.100:3300/0 conn(0x7f3e7c06abe0 0x7f3e7c1618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.868+0000 7f3e813b2640 1 -- 172.21.15.60:0/3999709343 >> v1:172.21.15.100:6789/0 conn(0x7f3e7c140310 legacy=0x7f3e7c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.868+0000 7f3e813b2640 1 -- 172.21.15.60:0/3999709343 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c142de0 con 0x7f3e7c1409d0 2023-11-18T20:44:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e813b2640 1 --2- 172.21.15.60:0/3999709343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c1409d0 0x7f3e7c06efb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c06e720 tx=0x7f3e6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7245a7bdacfa32bc server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e7b7fe640 1 -- 172.21.15.60:0/3999709343 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c031870 con 0x7f3e7c1409d0 2023-11-18T20:44:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e7b7fe640 1 -- 172.21.15.60:0/3999709343 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c0319d0 con 0x7f3e7c1409d0 2023-11-18T20:44:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e7b7fe640 1 -- 172.21.15.60:0/3999709343 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c031d60 con 0x7f3e7c1409d0 2023-11-18T20:44:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e82e15640 1 -- 172.21.15.60:0/3999709343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c1409d0 msgr2=0x7f3e7c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e82e15640 1 --2- 172.21.15.60:0/3999709343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c1409d0 0x7f3e7c06efb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c06e720 tx=0x7f3e6c02b910 comp rx=0 tx=0).stop 2023-11-18T20:44:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e82e15640 1 -- 172.21.15.60:0/3999709343 shutdown_connections 2023-11-18T20:44:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e82e15640 1 --2- 172.21.15.60:0/3999709343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c1409d0 0x7f3e7c06efb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e82e15640 1 --2- 172.21.15.60:0/3999709343 >> v2:172.21.15.100:3300/0 conn(0x7f3e7c06abe0 0x7f3e7c1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e82e15640 1 -- 172.21.15.60:0/3999709343 >> 172.21.15.60:0/3999709343 conn(0x7f3e7c0b1e50 msgr2=0x7f3e7c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e82e15640 1 -- 172.21.15.60:0/3999709343 shutdown_connections 2023-11-18T20:44:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.869+0000 7f3e82e15640 1 -- 172.21.15.60:0/3999709343 wait complete. 2023-11-18T20:44:40.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.870+0000 7f3e82e15640 1 Processor -- start 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.870+0000 7f3e82e15640 1 -- start start 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.871+0000 7f3e82e15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e7c06abe0 0x7f3e7c15e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.871+0000 7f3e82e15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c140310 0x7f3e7c13b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.871+0000 7f3e82e15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e7c1409d0 0x7f3e7c155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.871+0000 7f3e82e15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c146140 con 0x7f3e7c140310 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.871+0000 7f3e82e15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c15e590 con 0x7f3e7c06abe0 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.871+0000 7f3e82e15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c13bac0 con 0x7f3e7c1409d0 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.871+0000 7f3e7bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c140310 0x7f3e7c13b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.871+0000 7f3e7bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c140310 0x7f3e7c13b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48670/0 (socket says 172.21.15.60:48670) 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.871+0000 7f3e7bfff640 1 -- 172.21.15.60:0/1105443875 learned_addr learned my addr 172.21.15.60:0/1105443875 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e813b2640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e7c1409d0 0x7f3e7c155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e80bb1640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e7c06abe0 0x7f3e7c15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e7bfff640 1 -- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e7c1409d0 msgr2=0x7f3e7c155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e7bfff640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e7c1409d0 0x7f3e7c155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e7bfff640 1 -- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e7c06abe0 msgr2=0x7f3e7c15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e7bfff640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e7c06abe0 0x7f3e7c15e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e7bfff640 1 -- 172.21.15.60:0/1105443875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0094d0 con 0x7f3e7c140310 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e813b2640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e7c1409d0 0x7f3e7c155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e80bb1640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e7c06abe0 0x7f3e7c15e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e7bfff640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c140310 0x7f3e7c13b580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000e8a0 tx=0x7f3e7000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e797fa640 1 -- 172.21.15.60:0/1105443875 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e700094f0 con 0x7f3e7c140310 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e797fa640 1 -- 172.21.15.60:0/1105443875 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7000c940 con 0x7f3e7c140310 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e82e15640 1 -- 172.21.15.60:0/1105443875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c145e60 con 0x7f3e7c140310 2023-11-18T20:44:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e797fa640 1 -- 172.21.15.60:0/1105443875 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000ccf0 con 0x7f3e7c140310 2023-11-18T20:44:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.872+0000 7f3e82e15640 1 -- 172.21.15.60:0/1105443875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c143fb0 con 0x7f3e7c140310 2023-11-18T20:44:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.873+0000 7f3e82e15640 1 -- 172.21.15.60:0/1105443875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c143b30 con 0x7f3e7c140310 2023-11-18T20:44:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.874+0000 7f3e797fa640 1 -- 172.21.15.60:0/1105443875 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e70013470 con 0x7f3e7c140310 2023-11-18T20:44:40.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.874+0000 7f3e797fa640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3e6806ae40 0x7f3e6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:40.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.874+0000 7f3e797fa640 1 -- 172.21.15.60:0/1105443875 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e700896b0 con 0x7f3e7c140310 2023-11-18T20:44:40.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.874+0000 7f3e80bb1640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3e6806ae40 0x7f3e6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:40.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.875+0000 7f3e80bb1640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3e6806ae40 0x7f3e6806d300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3e740012e0 tx=0x7f3e74009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:40.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:40.878+0000 7f3e797fa640 1 -- 172.21.15.60:0/1105443875 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e7c143b30 con 0x7f3e7c140310 2023-11-18T20:44:41.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.055+0000 7f3e82e15640 1 -- 172.21.15.60:0/1105443875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e7c143b30 con 0x7f3e7c140310 2023-11-18T20:44:41.056 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.055+0000 7f3e797fa640 1 -- 172.21.15.60:0/1105443875 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e7c143b30 con 0x7f3e7c140310 2023-11-18T20:44:41.056 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:44:41.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 -- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3e6806ae40 msgr2=0x7f3e6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3e6806ae40 0x7f3e6806d300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3e740012e0 tx=0x7f3e74009040 comp rx=0 tx=0).stop 2023-11-18T20:44:41.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 -- 172.21.15.60:0/1105443875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c140310 msgr2=0x7f3e7c13b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c140310 0x7f3e7c13b580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000e8a0 tx=0x7f3e7000ed60 comp rx=0 tx=0).stop 2023-11-18T20:44:41.058 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 -- 172.21.15.60:0/1105443875 shutdown_connections 2023-11-18T20:44:41.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f3e6806ae40 0x7f3e6806d300 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e7c1409d0 0x7f3e7c155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e7c140310 0x7f3e7c13b580 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 --2- 172.21.15.60:0/1105443875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e7c06abe0 0x7f3e7c15e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 -- 172.21.15.60:0/1105443875 >> 172.21.15.60:0/1105443875 conn(0x7f3e7c0b1e50 msgr2=0x7f3e7c0b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:41.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 -- 172.21.15.60:0/1105443875 shutdown_connections 2023-11-18T20:44:41.059 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.057+0000 7f3e82e15640 1 -- 172.21.15.60:0/1105443875 wait complete. 2023-11-18T20:44:41.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:44:41.180 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.179+0000 7fdda1ebb640 1 Processor -- start 2023-11-18T20:44:41.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.179+0000 7fdda1ebb640 1 -- start start 2023-11-18T20:44:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.180+0000 7fdda1ebb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdd9c06ea50 0x7fdd9c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.180+0000 7fdda1ebb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd9c160fb0 0x7fdd9c161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.180+0000 7fdda1ebb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9c161d20 con 0x7fdd9c0b3ff0 2023-11-18T20:44:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.180+0000 7fdda1ebb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9c0d1ee0 con 0x7fdd9c06ea50 2023-11-18T20:44:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.180+0000 7fdda1ebb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c06f370 con 0x7fdd9c160fb0 2023-11-18T20:44:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.180+0000 7fdd9bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdd9c0b3ff0 0x7fdd9c0b4480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43494/0 (socket says 172.21.15.60:43494) 2023-11-18T20:44:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.180+0000 7fdd9bfff640 1 -- 172.21.15.60:0/3288737644 learned_addr learned my addr 172.21.15.60:0/3288737644 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.180+0000 7fdd9b7fe640 1 --2- 172.21.15.60:0/3288737644 >> v2:172.21.15.100:3300/0 conn(0x7fdd9c06ea50 0x7fdd9c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.180+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598803614 0 0) 0x7fdd9c161d20 con 0x7fdd9c0b3ff0 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.180+0000 7fdd9affd640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd9c160fb0 0x7fdd9c161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd84003520 con 0x7fdd9c0b3ff0 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414092992 0 0) 0x7fdd84003520 con 0x7fdd9c0b3ff0 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd9c160fb0 msgr2=0x7fdd9c161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd9c160fb0 0x7fdd9c161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 >> v2:172.21.15.100:3300/0 conn(0x7fdd9c06ea50 msgr2=0x7fdd9c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 --2- 172.21.15.60:0/3288737644 >> v2:172.21.15.100:3300/0 conn(0x7fdd9c06ea50 0x7fdd9c06ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c15e760 con 0x7fdd9c0b3ff0 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fdd940019f0 con 0x7fdd9c0b3ff0 2023-11-18T20:44:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 >> v1:172.21.15.60:6789/0 conn(0x7fdd9c0b3ff0 legacy=0x7fdd9c0b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd84003970 0x7fdd84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd840047c0 0x7fdd84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd9c160fb0 0x7fdd84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c0d1ee0 con 0x7fdd84003970 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c06f370 con 0x7fdd840047c0 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8400b980 con 0x7fdd9c160fb0 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9b7fe640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd84003970 0x7fdd84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9bfff640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd9c160fb0 0x7fdd84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9affd640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd840047c0 0x7fdd84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9b7fe640 1 -- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd9c160fb0 msgr2=0x7fdd84005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9b7fe640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd9c160fb0 0x7fdd84005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9b7fe640 1 -- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd840047c0 msgr2=0x7fdd84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9b7fe640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd840047c0 0x7fdd84004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.181+0000 7fdd9b7fe640 1 -- 172.21.15.60:0/3288737644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84003520 con 0x7fdd84003970 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.182+0000 7fdd9b7fe640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd84003970 0x7fdd84003d90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009ac0 tx=0x7fdd840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bf3ca86f5321c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.182+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c02c710 con 0x7fdd84003970 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.182+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c02c870 con 0x7fdd84003970 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.182+0000 7fdd9a7fc640 1 -- 172.21.15.60:0/3288737644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c02cc00 con 0x7fdd84003970 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.183+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3288737644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd84003970 msgr2=0x7fdd84003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.183+0000 7fdda1ebb640 1 --2- 172.21.15.60:0/3288737644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd84003970 0x7fdd84003d90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009ac0 tx=0x7fdd840057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.183+0000 7fdd9bfff640 1 -- 172.21.15.60:0/3288737644 reap_dead start 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.183+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3288737644 shutdown_connections 2023-11-18T20:44:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.183+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3288737644 >> 172.21.15.60:0/3288737644 conn(0x7fdd9c06a6b0 msgr2=0x7fdd9c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:41.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.183+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3288737644 shutdown_connections 2023-11-18T20:44:41.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.183+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3288737644 wait complete. 2023-11-18T20:44:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.184+0000 7fdda1ebb640 1 Processor -- start 2023-11-18T20:44:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.184+0000 7fdda1ebb640 1 -- start start 2023-11-18T20:44:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdda1ebb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd84003970 0x7fdd9c1f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdda1ebb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd9c15fcc0 0x7fdd9c160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdda1ebb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd9c150910 0x7fdd9c150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdda1ebb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c160680 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdda1ebb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c150040 con 0x7fdd9c150910 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdda1ebb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c1512d0 con 0x7fdd84003970 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdd9affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd9c15fcc0 0x7fdd9c160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdd9affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd9c15fcc0 0x7fdd9c160140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48682/0 (socket says 172.21.15.60:48682) 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdd9affd640 1 -- 172.21.15.60:0/3812802790 learned_addr learned my addr 172.21.15.60:0/3812802790 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdd9affd640 1 -- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd84003970 msgr2=0x7fdd9c1f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdd9affd640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd84003970 0x7fdd9c1f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdd9affd640 1 -- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd9c150910 msgr2=0x7fdd9c150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdd9affd640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd9c150910 0x7fdd9c150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdd9affd640 1 -- 172.21.15.60:0/3812802790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c151590 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.185+0000 7fdd9affd640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd9c15fcc0 0x7fdd9c160140 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdd90011600 tx=0x7fdd90011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.186+0000 7fdd83fff640 1 -- 172.21.15.60:0/3812802790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000f920 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.186+0000 7fdd83fff640 1 -- 172.21.15.60:0/3812802790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9000fa80 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.186+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3812802790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9c151870 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.186+0000 7fdd83fff640 1 -- 172.21.15.60:0/3812802790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000fdf0 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.186+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3812802790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c151e30 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.187+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3812802790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9c069400 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.187+0000 7fdd83fff640 1 -- 172.21.15.60:0/3812802790 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd90015460 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.187+0000 7fdd83fff640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdd8806ae40 0x7fdd8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.188+0000 7fdd83fff640 1 -- 172.21.15.60:0/3812802790 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd90089e90 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.188+0000 7fdd9b7fe640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdd8806ae40 0x7fdd8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.188+0000 7fdd9b7fe640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdd8806ae40 0x7fdd8806d300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c036000 tx=0x7fdd8c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:41.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.192+0000 7fdd83fff640 1 -- 172.21.15.60:0/3812802790 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd90054f40 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.365+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3812802790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd9c070d30 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.366+0000 7fdd83fff640 1 -- 172.21.15.60:0/3812802790 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd90058560 con 0x7fdd9c15fcc0 2023-11-18T20:44:41.367 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:41.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdd8806ae40 msgr2=0x7fdd8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdd8806ae40 0x7fdd8806d300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c036000 tx=0x7fdd8c02e000 comp rx=0 tx=0).stop 2023-11-18T20:44:41.369 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3812802790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd9c15fcc0 msgr2=0x7fdd9c160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd9c15fcc0 0x7fdd9c160140 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdd90011600 tx=0x7fdd90011ac0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3812802790 shutdown_connections 2023-11-18T20:44:41.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdd8806ae40 0x7fdd8806d300 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd9c150910 0x7fdd9c150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd9c15fcc0 0x7fdd9c160140 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 --2- 172.21.15.60:0/3812802790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd84003970 0x7fdd9c1f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3812802790 >> 172.21.15.60:0/3812802790 conn(0x7fdd9c06a6b0 msgr2=0x7fdd9c15c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:41.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3812802790 shutdown_connections 2023-11-18T20:44:41.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.368+0000 7fdda1ebb640 1 -- 172.21.15.60:0/3812802790 wait complete. 2023-11-18T20:44:41.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:44:41.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.488+0000 7fda99a15640 1 Processor -- start 2023-11-18T20:44:41.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.488+0000 7fda99a15640 1 -- start start 2023-11-18T20:44:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda99a15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda940d2b00 0x7fda941682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda99a15640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda941614e0 con 0x7fda94160f70 2023-11-18T20:44:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda99a15640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda941617c0 con 0x7fda9415d2d0 2023-11-18T20:44:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda99a15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda94161aa0 con 0x7fda940d2b00 2023-11-18T20:44:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda8bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fda94160f70 0x7fda940cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43500/0 (socket says 172.21.15.60:43500) 2023-11-18T20:44:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda8bfff640 1 -- 172.21.15.60:0/3781465285 learned_addr learned my addr 172.21.15.60:0/3781465285 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda937fe640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda940d2b00 0x7fda941682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192860845 0 0) 0x7fda941614e0 con 0x7fda94160f70 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda80003520 con 0x7fda94160f70 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16038550 0 0) 0x7fda80003520 con 0x7fda94160f70 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda940d2b00 msgr2=0x7fda941682a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda940d2b00 0x7fda941682a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 >> v1:172.21.15.100:6789/0 conn(0x7fda9415d2d0 legacy=0x7fda940b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda94161d80 con 0x7fda94160f70 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fda7c001670 con 0x7fda94160f70 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 >> v1:172.21.15.60:6789/0 conn(0x7fda94160f70 legacy=0x7fda940cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda800039a0 0x7fda80003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 0x7fda80004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda940d2b00 0x7fda80005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda94161aa0 con 0x7fda80004350 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.489+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8000b980 con 0x7fda800039a0 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.490+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8000bb40 con 0x7fda940d2b00 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.490+0000 7fda8bfff640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 0x7fda80004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.490+0000 7fda92ffd640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda800039a0 0x7fda80003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.490+0000 7fda937fe640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda940d2b00 0x7fda80005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.490+0000 7fda8bfff640 1 -- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda940d2b00 msgr2=0x7fda80005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.490+0000 7fda8bfff640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda940d2b00 0x7fda80005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.490+0000 7fda8bfff640 1 -- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda800039a0 msgr2=0x7fda80003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.490+0000 7fda8bfff640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda800039a0 0x7fda80003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.490+0000 7fda8bfff640 1 -- 172.21.15.60:0/3781465285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda80003520 con 0x7fda80004350 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda8bfff640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 0x7fda80004820 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00c210 tx=0x7fda800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a11f95d3bce5084 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:41.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c005ad0 con 0x7fda80004350 2023-11-18T20:44:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c005c50 con 0x7fda80004350 2023-11-18T20:44:41.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda927fc640 1 -- 172.21.15.60:0/3781465285 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c02c8d0 con 0x7fda80004350 2023-11-18T20:44:41.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda99a15640 1 -- 172.21.15.60:0/3781465285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 msgr2=0x7fda80004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda99a15640 1 --2- 172.21.15.60:0/3781465285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 0x7fda80004820 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00c210 tx=0x7fda800057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda937fe640 1 -- 172.21.15.60:0/3781465285 reap_dead start 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda99a15640 1 -- 172.21.15.60:0/3781465285 shutdown_connections 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda99a15640 1 -- 172.21.15.60:0/3781465285 >> 172.21.15.60:0/3781465285 conn(0x7fda94072230 msgr2=0x7fda940cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda99a15640 1 -- 172.21.15.60:0/3781465285 shutdown_connections 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.491+0000 7fda99a15640 1 -- 172.21.15.60:0/3781465285 wait complete. 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda99a15640 1 Processor -- start 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda99a15640 1 -- start start 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda99a15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 0x7fda940b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda99a15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda941516f0 0x7fda94160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda99a15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda94152cf0 0x7fda94153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda99a15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda941608b0 con 0x7fda80004350 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda99a15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9415e480 con 0x7fda94152cf0 2023-11-18T20:44:41.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda99a15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda94151b70 con 0x7fda941516f0 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda92ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 0x7fda940b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda92ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 0x7fda940b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48698/0 (socket says 172.21.15.60:48698) 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.493+0000 7fda92ffd640 1 -- 172.21.15.60:0/4240194188 learned_addr learned my addr 172.21.15.60:0/4240194188 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda8bfff640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda941516f0 0x7fda94160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda937fe640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda94152cf0 0x7fda94153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda92ffd640 1 -- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda941516f0 msgr2=0x7fda94160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda92ffd640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda941516f0 0x7fda94160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda92ffd640 1 -- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda94152cf0 msgr2=0x7fda94153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda92ffd640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda94152cf0 0x7fda94153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda92ffd640 1 -- 172.21.15.60:0/4240194188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9415db50 con 0x7fda80004350 2023-11-18T20:44:41.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda92ffd640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 0x7fda940b3830 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fda8400f7f0 tx=0x7fda8400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda90ff9640 1 -- 172.21.15.60:0/4240194188 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8400fea0 con 0x7fda80004350 2023-11-18T20:44:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda90ff9640 1 -- 172.21.15.60:0/4240194188 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda840052e0 con 0x7fda80004350 2023-11-18T20:44:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda90ff9640 1 -- 172.21.15.60:0/4240194188 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8400d910 con 0x7fda80004350 2023-11-18T20:44:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda99a15640 1 -- 172.21.15.60:0/4240194188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9415de30 con 0x7fda80004350 2023-11-18T20:44:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.494+0000 7fda99a15640 1 -- 172.21.15.60:0/4240194188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda94160580 con 0x7fda80004350 2023-11-18T20:44:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.495+0000 7fda99a15640 1 -- 172.21.15.60:0/4240194188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda94069400 con 0x7fda80004350 2023-11-18T20:44:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.496+0000 7fda90ff9640 1 -- 172.21.15.60:0/4240194188 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda8400da70 con 0x7fda80004350 2023-11-18T20:44:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.496+0000 7fda90ff9640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fda7406ae40 0x7fda7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.496+0000 7fda90ff9640 1 -- 172.21.15.60:0/4240194188 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda840895f0 con 0x7fda80004350 2023-11-18T20:44:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.496+0000 7fda8bfff640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fda7406ae40 0x7fda7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.497+0000 7fda8bfff640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fda7406ae40 0x7fda7406d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00c210 tx=0x7fda7c009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:41.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.500+0000 7fda90ff9640 1 -- 172.21.15.60:0/4240194188 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda840546a0 con 0x7fda80004350 2023-11-18T20:44:41.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.675+0000 7fda99a15640 1 -- 172.21.15.60:0/4240194188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda940d0aa0 con 0x7fda80004350 2023-11-18T20:44:41.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.675+0000 7fda90ff9640 1 -- 172.21.15.60:0/4240194188 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda84057cc0 con 0x7fda80004350 2023-11-18T20:44:41.677 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 -- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fda7406ae40 msgr2=0x7fda7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fda7406ae40 0x7fda7406d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00c210 tx=0x7fda7c009030 comp rx=0 tx=0).stop 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 -- 172.21.15.60:0/4240194188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 msgr2=0x7fda940b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 0x7fda940b3830 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fda8400f7f0 tx=0x7fda8400fcb0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 -- 172.21.15.60:0/4240194188 shutdown_connections 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fda7406ae40 0x7fda7406d300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda94152cf0 0x7fda94153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda941516f0 0x7fda94160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 --2- 172.21.15.60:0/4240194188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda80004350 0x7fda940b3830 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 -- 172.21.15.60:0/4240194188 >> 172.21.15.60:0/4240194188 conn(0x7fda94072230 msgr2=0x7fda940b7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:41.679 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 -- 172.21.15.60:0/4240194188 shutdown_connections 2023-11-18T20:44:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.677+0000 7fda99a15640 1 -- 172.21.15.60:0/4240194188 wait complete. 2023-11-18T20:44:41.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:44:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.797+0000 7faa687c0640 1 Processor -- start 2023-11-18T20:44:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.797+0000 7faa687c0640 1 -- start start 2023-11-18T20:44:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.798+0000 7faa687c0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa6015d330 0x7faa600b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.798+0000 7faa687c0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa600b2b10 con 0x7faa600d2910 2023-11-18T20:44:41.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.798+0000 7faa687c0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa600d1590 con 0x7faa600d3900 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.798+0000 7faa687c0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa600d1870 con 0x7faa6015d330 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.798+0000 7faa65d5b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faa600d2910 0x7faa600d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43514/0 (socket says 172.21.15.60:43514) 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.798+0000 7faa65d5b640 1 -- 172.21.15.60:0/3643079092 learned_addr learned my addr 172.21.15.60:0/3643079092 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6655c640 1 --2- 172.21.15.60:0/3643079092 >> v2:172.21.15.100:3300/0 conn(0x7faa6015d330 0x7faa600b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066855118 0 0) 0x7faa600b2b10 con 0x7faa600d2910 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa48003520 con 0x7faa600d2910 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4181141049 0 0) 0x7faa48003520 con 0x7faa600d2910 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 >> v2:172.21.15.100:3300/0 conn(0x7faa6015d330 msgr2=0x7faa600b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 --2- 172.21.15.60:0/3643079092 >> v2:172.21.15.100:3300/0 conn(0x7faa6015d330 0x7faa600b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 >> v1:172.21.15.100:6789/0 conn(0x7faa600d3900 legacy=0x7faa600b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa600d1b50 con 0x7faa600d2910 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7faa54001670 con 0x7faa600d2910 2023-11-18T20:44:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 >> v1:172.21.15.60:6789/0 conn(0x7faa600d2910 legacy=0x7faa600d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa48003970 0x7faa48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa480047c0 0x7faa48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa48005180 0x7faa4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa600d1870 con 0x7faa48003970 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4800c120 con 0x7faa48005180 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4800c2e0 con 0x7faa480047c0 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6655c640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa48003970 0x7faa48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa66d5d640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa48005180 0x7faa4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa65d5b640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa480047c0 0x7faa48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6655c640 1 -- 172.21.15.60:0/3643079092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa480047c0 msgr2=0x7faa48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6655c640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa480047c0 0x7faa48004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.799+0000 7faa6655c640 1 -- 172.21.15.60:0/3643079092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa48005180 msgr2=0x7faa4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:41.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.800+0000 7faa6655c640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa48005180 0x7faa4800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.800+0000 7faa6655c640 1 -- 172.21.15.60:0/3643079092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa48003520 con 0x7faa48003970 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.800+0000 7faa66d5d640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa48005180 0x7faa4800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.800+0000 7faa66d5d640 1 -- 172.21.15.60:0/3643079092 reap_dead start 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.800+0000 7faa6655c640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa48003970 0x7faa48003d90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faa58009ac0 tx=0x7faa480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fc3a2756ad7cbf server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.800+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5802c6b0 con 0x7faa48003970 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.800+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5802c810 con 0x7faa48003970 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.800+0000 7faa6555a640 1 -- 172.21.15.60:0/3643079092 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5802cba0 con 0x7faa48003970 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.801+0000 7faa687c0640 1 -- 172.21.15.60:0/3643079092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa48003970 msgr2=0x7faa48003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.801+0000 7faa687c0640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa48003970 0x7faa48003d90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faa58009ac0 tx=0x7faa480057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.801+0000 7faa687c0640 1 -- 172.21.15.60:0/3643079092 shutdown_connections 2023-11-18T20:44:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.801+0000 7faa687c0640 1 --2- 172.21.15.60:0/3643079092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa48003970 0x7faa48003d90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.801+0000 7faa687c0640 1 -- 172.21.15.60:0/3643079092 >> 172.21.15.60:0/3643079092 conn(0x7faa60072020 msgr2=0x7faa60072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:41.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.801+0000 7faa687c0640 1 -- 172.21.15.60:0/3643079092 shutdown_connections 2023-11-18T20:44:41.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.801+0000 7faa687c0640 1 -- 172.21.15.60:0/3643079092 wait complete. 2023-11-18T20:44:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.802+0000 7faa687c0640 1 Processor -- start 2023-11-18T20:44:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.802+0000 7faa687c0640 1 -- start start 2023-11-18T20:44:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa687c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa48003970 0x7faa600c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa687c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa600c9320 0x7faa600ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa687c0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa600ca870 0x7faa600cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa687c0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa600cacf0 con 0x7faa600c9320 2023-11-18T20:44:41.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa687c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa600c8e90 con 0x7faa48003970 2023-11-18T20:44:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa687c0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa601f2e30 con 0x7faa600ca870 2023-11-18T20:44:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa65d5b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa600c9320 0x7faa600ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa65d5b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa600c9320 0x7faa600ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48716/0 (socket says 172.21.15.60:48716) 2023-11-18T20:44:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa65d5b640 1 -- 172.21.15.60:0/102663722 learned_addr learned my addr 172.21.15.60:0/102663722 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa65d5b640 1 -- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa600ca870 msgr2=0x7faa600cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa65d5b640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa600ca870 0x7faa600cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa65d5b640 1 -- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa48003970 msgr2=0x7faa600c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa65d5b640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa48003970 0x7faa600c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.803+0000 7faa65d5b640 1 -- 172.21.15.60:0/102663722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa600c8280 con 0x7faa600c9320 2023-11-18T20:44:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.804+0000 7faa65d5b640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa600c9320 0x7faa600ca330 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e7f0 tx=0x7faa5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.804+0000 7faa5effd640 1 -- 172.21.15.60:0/102663722 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5400c850 con 0x7faa600c9320 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.804+0000 7faa5effd640 1 -- 172.21.15.60:0/102663722 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5400c9d0 con 0x7faa600c9320 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.804+0000 7faa687c0640 1 -- 172.21.15.60:0/102663722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa600c8560 con 0x7faa600c9320 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.804+0000 7faa5effd640 1 -- 172.21.15.60:0/102663722 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5400cd80 con 0x7faa600c9320 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.804+0000 7faa687c0640 1 -- 172.21.15.60:0/102663722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa600c8b20 con 0x7faa600c9320 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.805+0000 7faa687c0640 1 -- 172.21.15.60:0/102663722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa60069400 con 0x7faa600c9320 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.805+0000 7faa5effd640 1 -- 172.21.15.60:0/102663722 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa5401d460 con 0x7faa600c9320 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.806+0000 7faa5effd640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa4c06ae70 0x7faa4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.806+0000 7faa5effd640 1 -- 172.21.15.60:0/102663722 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa540895f0 con 0x7faa600c9320 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.806+0000 7faa6655c640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa4c06ae70 0x7faa4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.806+0000 7faa6655c640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa4c06ae70 0x7faa4c06d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa58036000 tx=0x7faa5802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:41.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:41.810+0000 7faa5effd640 1 -- 172.21.15.60:0/102663722 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa54054610 con 0x7faa600c9320 2023-11-18T20:44:42.006 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.005+0000 7faa687c0640 1 -- 172.21.15.60:0/102663722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa600c8fd0 con 0x7faa600c9320 2023-11-18T20:44:42.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.005+0000 7faa5effd640 1 -- 172.21.15.60:0/102663722 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7faa54057c30 con 0x7faa600c9320 2023-11-18T20:44:42.007 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:44:42.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 -- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa4c06ae70 msgr2=0x7faa4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa4c06ae70 0x7faa4c06d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa58036000 tx=0x7faa5802e000 comp rx=0 tx=0).stop 2023-11-18T20:44:42.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 -- 172.21.15.60:0/102663722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa600c9320 msgr2=0x7faa600ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa600c9320 0x7faa600ca330 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e7f0 tx=0x7faa5400ecb0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.009 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 -- 172.21.15.60:0/102663722 shutdown_connections 2023-11-18T20:44:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7faa4c06ae70 0x7faa4c06d330 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa600ca870 0x7faa600cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa600c9320 0x7faa600ca330 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 --2- 172.21.15.60:0/102663722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa48003970 0x7faa600c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 -- 172.21.15.60:0/102663722 >> 172.21.15.60:0/102663722 conn(0x7faa60072020 msgr2=0x7faa600b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 -- 172.21.15.60:0/102663722 shutdown_connections 2023-11-18T20:44:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.008+0000 7faa687c0640 1 -- 172.21.15.60:0/102663722 wait complete. 2023-11-18T20:44:42.019 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:42.020 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:42.035 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.179+0000 7f68653dc640 1 Processor -- start 2023-11-18T20:44:42.181 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.179+0000 7f68653dc640 1 -- start start 2023-11-18T20:44:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.180+0000 7f68653dc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f68600d2910 0x7f68600d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.180+0000 7f68653dc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68600d3730 0x7f68600b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.180+0000 7f68653dc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68600b2b10 con 0x7f686015d330 2023-11-18T20:44:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.180+0000 7f68653dc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68600d3bc0 con 0x7f68600d2910 2023-11-18T20:44:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.180+0000 7f68653dc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68600d1590 con 0x7f68600d3730 2023-11-18T20:44:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.180+0000 7f685e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f68600d2910 0x7f68600d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.180+0000 7f685e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f68600d2910 0x7f68600d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48722/0 (socket says 172.21.15.60:48722) 2023-11-18T20:44:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.180+0000 7f685e7fc640 1 -- 172.21.15.60:0/2462354195 learned_addr learned my addr 172.21.15.60:0/2462354195 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:42.182 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.181+0000 7f685f7fe640 1 --2- 172.21.15.60:0/2462354195 >> v2:172.21.15.100:3300/0 conn(0x7f68600d3730 0x7f68600b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.181+0000 7f685e7fc640 1 -- 172.21.15.60:0/2462354195 >> v2:172.21.15.100:3300/0 conn(0x7f68600d3730 msgr2=0x7f68600b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:42.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.181+0000 7f685e7fc640 1 --2- 172.21.15.60:0/2462354195 >> v2:172.21.15.100:3300/0 conn(0x7f68600d3730 0x7f68600b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.181+0000 7f685e7fc640 1 -- 172.21.15.60:0/2462354195 >> v1:172.21.15.100:6789/0 conn(0x7f686015d330 legacy=0x7f68600b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:42.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.181+0000 7f685e7fc640 1 -- 172.21.15.60:0/2462354195 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600d1870 con 0x7f68600d2910 2023-11-18T20:44:42.183 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.181+0000 7f685f7fe640 1 --2- 172.21.15.60:0/2462354195 >> v2:172.21.15.100:3300/0 conn(0x7f68600d3730 0x7f68600b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.181+0000 7f685e7fc640 1 --2- 172.21.15.60:0/2462354195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68600d2910 0x7f68600d31f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f686006c500 tx=0x7f684802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79263269910b9d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f685dffb640 1 -- 172.21.15.60:0/2462354195 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68480317e0 con 0x7f68600d2910 2023-11-18T20:44:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f685dffb640 1 -- 172.21.15.60:0/2462354195 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6848031940 con 0x7f68600d2910 2023-11-18T20:44:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f685dffb640 1 -- 172.21.15.60:0/2462354195 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848031cd0 con 0x7f68600d2910 2023-11-18T20:44:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f68653dc640 1 -- 172.21.15.60:0/2462354195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68600d2910 msgr2=0x7f68600d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f68653dc640 1 --2- 172.21.15.60:0/2462354195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68600d2910 0x7f68600d31f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f686006c500 tx=0x7f684802b880 comp rx=0 tx=0).stop 2023-11-18T20:44:42.184 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f68653dc640 1 -- 172.21.15.60:0/2462354195 shutdown_connections 2023-11-18T20:44:42.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f68653dc640 1 --2- 172.21.15.60:0/2462354195 >> v2:172.21.15.100:3300/0 conn(0x7f68600d3730 0x7f68600b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f68653dc640 1 --2- 172.21.15.60:0/2462354195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68600d2910 0x7f68600d31f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f68653dc640 1 -- 172.21.15.60:0/2462354195 >> 172.21.15.60:0/2462354195 conn(0x7f6860072020 msgr2=0x7f6860072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:42.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f68653dc640 1 -- 172.21.15.60:0/2462354195 shutdown_connections 2023-11-18T20:44:42.185 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.182+0000 7f68653dc640 1 -- 172.21.15.60:0/2462354195 wait complete. 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.184+0000 7f68653dc640 1 Processor -- start 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.184+0000 7f68653dc640 1 -- start start 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.184+0000 7f68653dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68600d2910 0x7f68600cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.184+0000 7f68653dc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68600d3730 0x7f6860154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.184+0000 7f68653dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686015d330 0x7f68600cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.184+0000 7f68653dc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68600cc1f0 con 0x7f686015d330 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.184+0000 7f68653dc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686015fb10 con 0x7f68600d2910 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.184+0000 7f68653dc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686015fdf0 con 0x7f68600d3730 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686015d330 0x7f68600cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686015d330 0x7f68600cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48724/0 (socket says 172.21.15.60:48724) 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685f7fe640 1 -- 172.21.15.60:0/1201749324 learned_addr learned my addr 172.21.15.60:0/1201749324 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:42.187 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685e7fc640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68600d3730 0x7f6860154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685effd640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68600d2910 0x7f68600cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685f7fe640 1 -- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68600d3730 msgr2=0x7f6860154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685f7fe640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68600d3730 0x7f6860154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685f7fe640 1 -- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68600d2910 msgr2=0x7f68600cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685f7fe640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68600d2910 0x7f68600cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685f7fe640 1 -- 172.21.15.60:0/1201749324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68480094d0 con 0x7f686015d330 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f685f7fe640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686015d330 0x7f68600cca60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f685800c7f0 tx=0x7f685800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f68577fe640 1 -- 172.21.15.60:0/1201749324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858007820 con 0x7f686015d330 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f68577fe640 1 -- 172.21.15.60:0/1201749324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6858007980 con 0x7f686015d330 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f68653dc640 1 -- 172.21.15.60:0/1201749324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68600cb4b0 con 0x7f686015d330 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f68577fe640 1 -- 172.21.15.60:0/1201749324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858007d70 con 0x7f686015d330 2023-11-18T20:44:42.188 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.185+0000 7f68653dc640 1 -- 172.21.15.60:0/1201749324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68601600d0 con 0x7f686015d330 2023-11-18T20:44:42.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.186+0000 7f68653dc640 1 -- 172.21.15.60:0/1201749324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68600cb9d0 con 0x7f686015d330 2023-11-18T20:44:42.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.187+0000 7f68577fe640 1 -- 172.21.15.60:0/1201749324 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68580113f0 con 0x7f686015d330 2023-11-18T20:44:42.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.187+0000 7f68577fe640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f684006ae70 0x7f684006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.187+0000 7f68577fe640 1 -- 172.21.15.60:0/1201749324 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6858089890 con 0x7f686015d330 2023-11-18T20:44:42.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.188+0000 7f685effd640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f684006ae70 0x7f684006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.188+0000 7f685effd640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f684006ae70 0x7f684006d330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f685000b450 tx=0x7f685000d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:42.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.192+0000 7f68577fe640 1 -- 172.21.15.60:0/1201749324 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68600cb9d0 con 0x7f686015d330 2023-11-18T20:44:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.372+0000 7f68653dc640 1 -- 172.21.15.60:0/1201749324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68600cb9d0 con 0x7f686015d330 2023-11-18T20:44:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.373+0000 7f68577fe640 1 -- 172.21.15.60:0/1201749324 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f68600cb9d0 con 0x7f686015d330 2023-11-18T20:44:42.374 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:42.374 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:34.103787+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 -- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f684006ae70 msgr2=0x7f684006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f684006ae70 0x7f684006d330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f685000b450 tx=0x7f685000d040 comp rx=0 tx=0).stop 2023-11-18T20:44:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 -- 172.21.15.60:0/1201749324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686015d330 msgr2=0x7f68600cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686015d330 0x7f68600cca60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f685800c7f0 tx=0x7f685800ccb0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 -- 172.21.15.60:0/1201749324 shutdown_connections 2023-11-18T20:44:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f684006ae70 0x7f684006d330 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f686015d330 0x7f68600cca60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68600d3730 0x7f6860154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 --2- 172.21.15.60:0/1201749324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68600d2910 0x7f68600cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 -- 172.21.15.60:0/1201749324 >> 172.21.15.60:0/1201749324 conn(0x7f6860072020 msgr2=0x7f6860072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 -- 172.21.15.60:0/1201749324 shutdown_connections 2023-11-18T20:44:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.375+0000 7f68653dc640 1 -- 172.21.15.60:0/1201749324 wait complete. 2023-11-18T20:44:42.387 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:44:31.358325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:44:32.036538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:44:42.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:44:42.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.496+0000 7f7304905640 1 Processor -- start 2023-11-18T20:44:42.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.496+0000 7f7304905640 1 -- start start 2023-11-18T20:44:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f7304905640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7300143490 0x7f73000b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f7304905640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f730015abf0 0x7f73000b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f7304905640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730015e2a0 con 0x7f730015e730 2023-11-18T20:44:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f7304905640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730015b520 con 0x7f7300143490 2023-11-18T20:44:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f7304905640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7300143870 con 0x7f730015abf0 2023-11-18T20:44:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f72fe59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7300143490 0x7f73000b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f72fe59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7300143490 0x7f73000b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48738/0 (socket says 172.21.15.60:48738) 2023-11-18T20:44:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f72fe59c640 1 -- 172.21.15.60:0/1126277607 learned_addr learned my addr 172.21.15.60:0/1126277607 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f72fed9d640 1 --2- 172.21.15.60:0/1126277607 >> v2:172.21.15.100:3300/0 conn(0x7f730015abf0 0x7f73000b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f72fe59c640 1 -- 172.21.15.60:0/1126277607 >> v2:172.21.15.100:3300/0 conn(0x7f730015abf0 msgr2=0x7f73000b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f72fe59c640 1 --2- 172.21.15.60:0/1126277607 >> v2:172.21.15.100:3300/0 conn(0x7f730015abf0 0x7f73000b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f72fe59c640 1 -- 172.21.15.60:0/1126277607 >> v1:172.21.15.60:6789/0 conn(0x7f730015e730 legacy=0x7f730015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f72fd59a640 1 -- 172.21.15.60:0/1126277607 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096108883 0 0) 0x7f730015e2a0 con 0x7f730015e730 2023-11-18T20:44:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.497+0000 7f72fe59c640 1 -- 172.21.15.60:0/1126277607 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730015b240 con 0x7f7300143490 2023-11-18T20:44:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.498+0000 7f72fe59c640 1 --2- 172.21.15.60:0/1126277607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7300143490 0x7f73000b0cf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f73000b4ca0 tx=0x7f72f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbb6e9241c5bda8 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:42.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.498+0000 7f72fd59a640 1 -- 172.21.15.60:0/1126277607 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8030860 con 0x7f7300143490 2023-11-18T20:44:42.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.498+0000 7f72fd59a640 1 -- 172.21.15.60:0/1126277607 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f80309c0 con 0x7f7300143490 2023-11-18T20:44:42.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.498+0000 7f72fd59a640 1 -- 172.21.15.60:0/1126277607 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8030d50 con 0x7f7300143490 2023-11-18T20:44:42.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.499+0000 7f7304905640 1 -- 172.21.15.60:0/1126277607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7300143490 msgr2=0x7f73000b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.499+0000 7f7304905640 1 --2- 172.21.15.60:0/1126277607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7300143490 0x7f73000b0cf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f73000b4ca0 tx=0x7f72f802b9a0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.499+0000 7f7304905640 1 -- 172.21.15.60:0/1126277607 shutdown_connections 2023-11-18T20:44:42.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.499+0000 7f7304905640 1 --2- 172.21.15.60:0/1126277607 >> v2:172.21.15.100:3300/0 conn(0x7f730015abf0 0x7f73000b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.499+0000 7f7304905640 1 --2- 172.21.15.60:0/1126277607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7300143490 0x7f73000b0cf0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.499+0000 7f7304905640 1 -- 172.21.15.60:0/1126277607 >> 172.21.15.60:0/1126277607 conn(0x7f730006a4d0 msgr2=0x7f730006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:42.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.499+0000 7f7304905640 1 -- 172.21.15.60:0/1126277607 shutdown_connections 2023-11-18T20:44:42.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.499+0000 7f7304905640 1 -- 172.21.15.60:0/1126277607 wait complete. 2023-11-18T20:44:42.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.500+0000 7f7304905640 1 Processor -- start 2023-11-18T20:44:42.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.500+0000 7f7304905640 1 -- start start 2023-11-18T20:44:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f7304905640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7300143490 0x7f73001e5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f7304905640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f730015abf0 0x7f73001f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f7304905640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f730015e730 0x7f730012f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f7304905640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7300131a60 con 0x7f730015e730 2023-11-18T20:44:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f7304905640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7300131760 con 0x7f7300143490 2023-11-18T20:44:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f7304905640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73001f69d0 con 0x7f730015abf0 2023-11-18T20:44:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f72fed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f730015e730 0x7f730012f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f72fed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f730015e730 0x7f730012f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48750/0 (socket says 172.21.15.60:48750) 2023-11-18T20:44:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f72fed9d640 1 -- 172.21.15.60:0/1207385720 learned_addr learned my addr 172.21.15.60:0/1207385720 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f72fe59c640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7300143490 0x7f73001e5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.501+0000 7f72fdd9b640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f730015abf0 0x7f73001f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f72fed9d640 1 -- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f730015abf0 msgr2=0x7f73001f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f72fed9d640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f730015abf0 0x7f73001f6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f72fed9d640 1 -- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7300143490 msgr2=0x7f73001e5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f72fed9d640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7300143490 0x7f73001e5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f72fed9d640 1 -- 172.21.15.60:0/1207385720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f80094d0 con 0x7f730015e730 2023-11-18T20:44:42.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f72fe59c640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7300143490 0x7f73001e5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:42.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f72fed9d640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f730015e730 0x7f730012f750 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f72f000d830 tx=0x7f72f000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:42.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f72eeffd640 1 -- 172.21.15.60:0/1207385720 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f000b6c0 con 0x7f730015e730 2023-11-18T20:44:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f72eeffd640 1 -- 172.21.15.60:0/1207385720 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f000b840 con 0x7f730015e730 2023-11-18T20:44:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f7304905640 1 -- 172.21.15.60:0/1207385720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73001f6be0 con 0x7f730015e730 2023-11-18T20:44:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f72eeffd640 1 -- 172.21.15.60:0/1207385720 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f000bbf0 con 0x7f730015e730 2023-11-18T20:44:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.502+0000 7f7304905640 1 -- 172.21.15.60:0/1207385720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73001f71a0 con 0x7f730015e730 2023-11-18T20:44:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.503+0000 7f7304905640 1 -- 172.21.15.60:0/1207385720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7300131ba0 con 0x7f730015e730 2023-11-18T20:44:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.503+0000 7f72eeffd640 1 -- 172.21.15.60:0/1207385720 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f72f001e460 con 0x7f730015e730 2023-11-18T20:44:42.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.504+0000 7f72eeffd640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f72e006ae40 0x7f72e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.504+0000 7f72eeffd640 1 -- 172.21.15.60:0/1207385720 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72f0089e10 con 0x7f730015e730 2023-11-18T20:44:42.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.504+0000 7f72fe59c640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f72e006ae40 0x7f72e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.505+0000 7f72fe59c640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f72e006ae40 0x7f72e006d300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f72f8036000 tx=0x7f72f8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:42.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.508+0000 7f72eeffd640 1 -- 172.21.15.60:0/1207385720 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7300131ba0 con 0x7f730015e730 2023-11-18T20:44:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.682+0000 7f7304905640 1 -- 172.21.15.60:0/1207385720 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7300131ba0 con 0x7f730015e730 2023-11-18T20:44:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.682+0000 7f72eeffd640 1 -- 172.21.15.60:0/1207385720 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7300131ba0 con 0x7f730015e730 2023-11-18T20:44:42.684 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:44:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 -- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f72e006ae40 msgr2=0x7f72e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f72e006ae40 0x7f72e006d300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f72f8036000 tx=0x7f72f8008030 comp rx=0 tx=0).stop 2023-11-18T20:44:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 -- 172.21.15.60:0/1207385720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f730015e730 msgr2=0x7f730012f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f730015e730 0x7f730012f750 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f72f000d830 tx=0x7f72f000dcf0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 -- 172.21.15.60:0/1207385720 shutdown_connections 2023-11-18T20:44:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f72e006ae40 0x7f72e006d300 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f730015e730 0x7f730012f750 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f730015abf0 0x7f73001f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 --2- 172.21.15.60:0/1207385720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7300143490 0x7f73001e5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 -- 172.21.15.60:0/1207385720 >> 172.21.15.60:0/1207385720 conn(0x7f730006a4d0 msgr2=0x7f73000ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 -- 172.21.15.60:0/1207385720 shutdown_connections 2023-11-18T20:44:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.684+0000 7f7304905640 1 -- 172.21.15.60:0/1207385720 wait complete. 2023-11-18T20:44:42.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:44:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.804+0000 7ff78d415640 1 Processor -- start 2023-11-18T20:44:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.804+0000 7ff78d415640 1 -- start start 2023-11-18T20:44:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.805+0000 7ff78d415640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7880b4370 0x7ff7880b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.805+0000 7ff78d415640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff788071770 con 0x7ff788144980 2023-11-18T20:44:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.805+0000 7ff78d415640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7880ccb60 con 0x7ff7880ceb70 2023-11-18T20:44:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.805+0000 7ff78d415640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78815ced0 con 0x7ff7880b4370 2023-11-18T20:44:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.805+0000 7ff786ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff788144980 0x7ff78815c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43530/0 (socket says 172.21.15.60:43530) 2023-11-18T20:44:42.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.805+0000 7ff786ffd640 1 -- 172.21.15.60:0/3795828345 learned_addr learned my addr 172.21.15.60:0/3795828345 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff7877fe640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7880b4370 0x7ff7880b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569477654 0 0) 0x7ff788071770 con 0x7ff788144980 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff770003520 con 0x7ff788144980 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198600484 0 0) 0x7ff7880ccb60 con 0x7ff7880ceb70 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff788071770 con 0x7ff7880ceb70 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3494689295 0 0) 0x7ff770003520 con 0x7ff788144980 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7880b4370 msgr2=0x7ff7880b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7880b4370 0x7ff7880b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 >> v1:172.21.15.100:6789/0 conn(0x7ff7880ceb70 legacy=0x7ff7880cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7880b4ce0 con 0x7ff788144980 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7ff780001670 con 0x7ff788144980 2023-11-18T20:44:42.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 >> v1:172.21.15.60:6789/0 conn(0x7ff788144980 legacy=0x7ff78815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff770003b30 0x7ff770003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7700047c0 0x7ff770004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7880b4370 0x7ff770005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78815ced0 con 0x7ff770003b30 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77000ba60 con 0x7ff7700047c0 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77000bc20 con 0x7ff7880b4370 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff786ffd640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff770003b30 0x7ff770003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff7877fe640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7880b4370 0x7ff770005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff7867fc640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7700047c0 0x7ff770004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff786ffd640 1 -- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7880b4370 msgr2=0x7ff770005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.806+0000 7ff786ffd640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7880b4370 0x7ff770005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.807+0000 7ff786ffd640 1 -- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7700047c0 msgr2=0x7ff770004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.807+0000 7ff786ffd640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7700047c0 0x7ff770004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.807+0000 7ff786ffd640 1 -- 172.21.15.60:0/3795828345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770003520 con 0x7ff770003b30 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.807+0000 7ff7867fc640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7700047c0 0x7ff770004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.807+0000 7ff786ffd640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff770003b30 0x7ff770003f50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff780001eb0 tx=0x7ff7700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667ad16d78ad4384 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.807+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780005ad0 con 0x7ff770003b30 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.807+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff780005c30 con 0x7ff770003b30 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.807+0000 7ff785ffb640 1 -- 172.21.15.60:0/3795828345 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780007070 con 0x7ff770003b30 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.808+0000 7ff78d415640 1 -- 172.21.15.60:0/3795828345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff770003b30 msgr2=0x7ff770003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.808+0000 7ff78d415640 1 --2- 172.21.15.60:0/3795828345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff770003b30 0x7ff770003f50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff780001eb0 tx=0x7ff7700057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.808+0000 7ff7877fe640 1 -- 172.21.15.60:0/3795828345 reap_dead start 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.808+0000 7ff78d415640 1 -- 172.21.15.60:0/3795828345 shutdown_connections 2023-11-18T20:44:42.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.808+0000 7ff78d415640 1 -- 172.21.15.60:0/3795828345 >> 172.21.15.60:0/3795828345 conn(0x7ff78806a4d0 msgr2=0x7ff78806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.808+0000 7ff78d415640 1 -- 172.21.15.60:0/3795828345 shutdown_connections 2023-11-18T20:44:42.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.808+0000 7ff78d415640 1 -- 172.21.15.60:0/3795828345 wait complete. 2023-11-18T20:44:42.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.809+0000 7ff78d415640 1 Processor -- start 2023-11-18T20:44:42.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.809+0000 7ff78d415640 1 -- start start 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff78d415640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff770003b30 0x7ff7881f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff78d415640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7880b3090 0x7ff7880b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff78d415640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7881564a0 0x7ff788156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff78d415640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff788157d70 con 0x7ff7881564a0 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff78d415640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880c9720 con 0x7ff7880b3090 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff78d415640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880b3a30 con 0x7ff770003b30 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff7877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7881564a0 0x7ff788156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff7877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7881564a0 0x7ff788156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48764/0 (socket says 172.21.15.60:48764) 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff7877fe640 1 -- 172.21.15.60:0/760099209 learned_addr learned my addr 172.21.15.60:0/760099209 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff786ffd640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff770003b30 0x7ff7881f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff7867fc640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7880b3090 0x7ff7880b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff7877fe640 1 -- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff770003b30 msgr2=0x7ff7881f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff7877fe640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff770003b30 0x7ff7881f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff7877fe640 1 -- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7880b3090 msgr2=0x7ff7880b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.810+0000 7ff7877fe640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7880b3090 0x7ff7880b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.811+0000 7ff7877fe640 1 -- 172.21.15.60:0/760099209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff788156e60 con 0x7ff7881564a0 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.811+0000 7ff7867fc640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7880b3090 0x7ff7880b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.811+0000 7ff7877fe640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7881564a0 0x7ff788156920 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff788071100 tx=0x7ff77800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.811+0000 7ff76f7fe640 1 -- 172.21.15.60:0/760099209 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77800e040 con 0x7ff7881564a0 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.811+0000 7ff76f7fe640 1 -- 172.21.15.60:0/760099209 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77800f040 con 0x7ff7881564a0 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.811+0000 7ff78d415640 1 -- 172.21.15.60:0/760099209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7881570f0 con 0x7ff7881564a0 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.811+0000 7ff76f7fe640 1 -- 172.21.15.60:0/760099209 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778015670 con 0x7ff7881564a0 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.811+0000 7ff78d415640 1 -- 172.21.15.60:0/760099209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7881576b0 con 0x7ff7881564a0 2023-11-18T20:44:42.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.812+0000 7ff78d415640 1 -- 172.21.15.60:0/760099209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff788069400 con 0x7ff7881564a0 2023-11-18T20:44:42.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.812+0000 7ff76f7fe640 1 -- 172.21.15.60:0/760099209 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff7780157d0 con 0x7ff7881564a0 2023-11-18T20:44:42.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.813+0000 7ff76f7fe640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff77406ae70 0x7ff77406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:42.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.813+0000 7ff76f7fe640 1 -- 172.21.15.60:0/760099209 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff778089cf0 con 0x7ff7881564a0 2023-11-18T20:44:42.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.813+0000 7ff786ffd640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff77406ae70 0x7ff77406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:42.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.813+0000 7ff786ffd640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff77406ae70 0x7ff77406d330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff780034000 tx=0x7ff780009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:42.818 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.817+0000 7ff76f7fe640 1 -- 172.21.15.60:0/760099209 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff77808e070 con 0x7ff7881564a0 2023-11-18T20:44:42.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.992+0000 7ff78d415640 1 -- 172.21.15.60:0/760099209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff788159d30 con 0x7ff7881564a0 2023-11-18T20:44:42.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.993+0000 7ff76f7fe640 1 -- 172.21.15.60:0/760099209 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff778054da0 con 0x7ff7881564a0 2023-11-18T20:44:42.994 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.994+0000 7ff78d415640 1 -- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff77406ae70 msgr2=0x7ff77406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.994+0000 7ff78d415640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff77406ae70 0x7ff77406d330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff780034000 tx=0x7ff780009030 comp rx=0 tx=0).stop 2023-11-18T20:44:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.994+0000 7ff78d415640 1 -- 172.21.15.60:0/760099209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7881564a0 msgr2=0x7ff788156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.994+0000 7ff78d415640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7881564a0 0x7ff788156920 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff788071100 tx=0x7ff77800f580 comp rx=0 tx=0).stop 2023-11-18T20:44:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.994+0000 7ff78d415640 1 -- 172.21.15.60:0/760099209 shutdown_connections 2023-11-18T20:44:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.994+0000 7ff78d415640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff77406ae70 0x7ff77406d330 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.994+0000 7ff78d415640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7881564a0 0x7ff788156920 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.994+0000 7ff78d415640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7880b3090 0x7ff7880b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.994+0000 7ff78d415640 1 --2- 172.21.15.60:0/760099209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff770003b30 0x7ff7881f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:42.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.994+0000 7ff78d415640 1 -- 172.21.15.60:0/760099209 >> 172.21.15.60:0/760099209 conn(0x7ff78806a4d0 msgr2=0x7ff78806c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:42.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.995+0000 7ff78d415640 1 -- 172.21.15.60:0/760099209 shutdown_connections 2023-11-18T20:44:42.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:42.995+0000 7ff78d415640 1 -- 172.21.15.60:0/760099209 wait complete. 2023-11-18T20:44:43.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:44:43.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.115+0000 7f20429a9640 1 Processor -- start 2023-11-18T20:44:43.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.115+0000 7f20429a9640 1 -- start start 2023-11-18T20:44:43.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.115+0000 7f20429a9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f203c15d2d0 0x7f203c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.115+0000 7f20429a9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f203c160f70 0x7f203c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.115+0000 7f20429a9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c1614e0 con 0x7f203c0ce480 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.115+0000 7f20429a9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c1617c0 con 0x7f203c15d2d0 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.115+0000 7f20429a9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c161aa0 con 0x7f203c160f70 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.116+0000 7f203bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f203c15d2d0 0x7f203c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.116+0000 7f2040f46640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f203c0ce480 0x7f203c06b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43544/0 (socket says 172.21.15.60:43544) 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.116+0000 7f2040f46640 1 -- 172.21.15.60:0/2621177913 learned_addr learned my addr 172.21.15.60:0/2621177913 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.116+0000 7f203bfff640 1 -- 172.21.15.60:0/2621177913 >> v2:172.21.15.100:3300/0 conn(0x7f203c160f70 msgr2=0x7f203c0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.116+0000 7f203b7fe640 1 --2- 172.21.15.60:0/2621177913 >> v2:172.21.15.100:3300/0 conn(0x7f203c160f70 0x7f203c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.116+0000 7f203affd640 1 -- 172.21.15.60:0/2621177913 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125261674 0 0) 0x7f203c1614e0 con 0x7f203c0ce480 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.116+0000 7f203bfff640 1 --2- 172.21.15.60:0/2621177913 >> v2:172.21.15.100:3300/0 conn(0x7f203c160f70 0x7f203c0cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.116+0000 7f203bfff640 1 -- 172.21.15.60:0/2621177913 >> v1:172.21.15.60:6789/0 conn(0x7f203c0ce480 legacy=0x7f203c06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.116+0000 7f203bfff640 1 -- 172.21.15.60:0/2621177913 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c161d80 con 0x7f203c15d2d0 2023-11-18T20:44:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f203bfff640 1 --2- 172.21.15.60:0/2621177913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c15d2d0 0x7f203c0b8b90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f203c0cdb80 tx=0x7f202402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cfc3bccd39b589 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f203affd640 1 -- 172.21.15.60:0/2621177913 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024031830 con 0x7f203c15d2d0 2023-11-18T20:44:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f203affd640 1 -- 172.21.15.60:0/2621177913 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2024031990 con 0x7f203c15d2d0 2023-11-18T20:44:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f203affd640 1 -- 172.21.15.60:0/2621177913 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024031d20 con 0x7f203c15d2d0 2023-11-18T20:44:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f20429a9640 1 -- 172.21.15.60:0/2621177913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c15d2d0 msgr2=0x7f203c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f20429a9640 1 --2- 172.21.15.60:0/2621177913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c15d2d0 0x7f203c0b8b90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f203c0cdb80 tx=0x7f202402b8d0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f20429a9640 1 -- 172.21.15.60:0/2621177913 shutdown_connections 2023-11-18T20:44:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f20429a9640 1 --2- 172.21.15.60:0/2621177913 >> v2:172.21.15.100:3300/0 conn(0x7f203c160f70 0x7f203c0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f20429a9640 1 --2- 172.21.15.60:0/2621177913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c15d2d0 0x7f203c0b8b90 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f20429a9640 1 -- 172.21.15.60:0/2621177913 >> 172.21.15.60:0/2621177913 conn(0x7f203c072230 msgr2=0x7f203c0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f20429a9640 1 -- 172.21.15.60:0/2621177913 shutdown_connections 2023-11-18T20:44:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.117+0000 7f20429a9640 1 -- 172.21.15.60:0/2621177913 wait complete. 2023-11-18T20:44:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.119+0000 7f20429a9640 1 Processor -- start 2023-11-18T20:44:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.119+0000 7f20429a9640 1 -- start start 2023-11-18T20:44:43.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.119+0000 7f20429a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c0ce480 0x7f203c0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.119+0000 7f20429a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f203c15d2d0 0x7f203c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.123 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.119+0000 7f20429a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f203c160f70 0x7f203c15dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.119+0000 7f20429a9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c159be0 con 0x7f203c0ce480 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f20429a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c159e30 con 0x7f203c15d2d0 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f20429a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c15a110 con 0x7f203c160f70 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f203bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c0ce480 0x7f203c0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f203bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c0ce480 0x7f203c0c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48778/0 (socket says 172.21.15.60:48778) 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f203bfff640 1 -- 172.21.15.60:0/2463449074 learned_addr learned my addr 172.21.15.60:0/2463449074 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f203b7fe640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f203c15d2d0 0x7f203c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f2040f46640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f203c160f70 0x7f203c15dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f203bfff640 1 -- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f203c160f70 msgr2=0x7f203c15dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f203bfff640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f203c160f70 0x7f203c15dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f203bfff640 1 -- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f203c15d2d0 msgr2=0x7f203c0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f203bfff640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f203c15d2d0 0x7f203c0cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f203bfff640 1 -- 172.21.15.60:0/2463449074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2024009400 con 0x7f203c0ce480 2023-11-18T20:44:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f203bfff640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c0ce480 0x7f203c0c92f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2024009890 tx=0x7f20240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f2038ff9640 1 -- 172.21.15.60:0/2463449074 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202403ba70 con 0x7f203c0ce480 2023-11-18T20:44:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f2038ff9640 1 -- 172.21.15.60:0/2463449074 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202403bbf0 con 0x7f203c0ce480 2023-11-18T20:44:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f2038ff9640 1 -- 172.21.15.60:0/2463449074 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024045400 con 0x7f203c0ce480 2023-11-18T20:44:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f20429a9640 1 -- 172.21.15.60:0/2463449074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c15a3f0 con 0x7f203c0ce480 2023-11-18T20:44:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.120+0000 7f20429a9640 1 -- 172.21.15.60:0/2463449074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c15a9b0 con 0x7f203c0ce480 2023-11-18T20:44:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.122+0000 7f20429a9640 1 -- 172.21.15.60:0/2463449074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c069400 con 0x7f203c0ce480 2023-11-18T20:44:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.122+0000 7f2038ff9640 1 -- 172.21.15.60:0/2463449074 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f202404e440 con 0x7f203c0ce480 2023-11-18T20:44:43.125 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.122+0000 7f2038ff9640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f201c06ae40 0x7f201c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.122+0000 7f2038ff9640 1 -- 172.21.15.60:0/2463449074 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20240ab840 con 0x7f203c0ce480 2023-11-18T20:44:43.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.122+0000 7f203b7fe640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f201c06ae40 0x7f201c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.123+0000 7f203b7fe640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f201c06ae40 0x7f201c06d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f203c160a50 tx=0x7f202c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:43.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.127+0000 7f2038ff9640 1 -- 172.21.15.60:0/2463449074 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20240769a0 con 0x7f203c0ce480 2023-11-18T20:44:43.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.302+0000 7f20429a9640 1 -- 172.21.15.60:0/2463449074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f203c0d0aa0 con 0x7f203c0ce480 2023-11-18T20:44:43.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.303+0000 7f2038ff9640 1 -- 172.21.15.60:0/2463449074 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20240350a0 con 0x7f203c0ce480 2023-11-18T20:44:43.304 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:44:43.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 -- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f201c06ae40 msgr2=0x7f201c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f201c06ae40 0x7f201c06d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f203c160a50 tx=0x7f202c008040 comp rx=0 tx=0).stop 2023-11-18T20:44:43.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 -- 172.21.15.60:0/2463449074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c0ce480 msgr2=0x7f203c0c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c0ce480 0x7f203c0c92f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2024009890 tx=0x7f20240046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 -- 172.21.15.60:0/2463449074 shutdown_connections 2023-11-18T20:44:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f201c06ae40 0x7f201c06d300 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f203c160f70 0x7f203c15dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f203c15d2d0 0x7f203c0cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 --2- 172.21.15.60:0/2463449074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f203c0ce480 0x7f203c0c92f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 -- 172.21.15.60:0/2463449074 >> 172.21.15.60:0/2463449074 conn(0x7f203c072230 msgr2=0x7f203c0b8660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:43.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 -- 172.21.15.60:0/2463449074 shutdown_connections 2023-11-18T20:44:43.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.305+0000 7f20429a9640 1 -- 172.21.15.60:0/2463449074 wait complete. 2023-11-18T20:44:43.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:44:43.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.422+0000 7fd6d2f1d640 1 Processor -- start 2023-11-18T20:44:43.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.422+0000 7fd6d2f1d640 1 -- start start 2023-11-18T20:44:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.423+0000 7fd6d2f1d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd6cc0d2910 0x7fd6cc0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.423+0000 7fd6d2f1d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0b2b10 con 0x7fd6cc15d330 2023-11-18T20:44:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.423+0000 7fd6d2f1d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0d1590 con 0x7fd6cc0d3730 2023-11-18T20:44:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.423+0000 7fd6d2f1d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6cc0d3bc0 con 0x7fd6cc0d2910 2023-11-18T20:44:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.423+0000 7fd6d0cb9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd6cc15d330 0x7fd6cc0b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43550/0 (socket says 172.21.15.60:43550) 2023-11-18T20:44:43.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.423+0000 7fd6d0cb9640 1 -- 172.21.15.60:0/3556078792 learned_addr learned my addr 172.21.15.60:0/3556078792 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cbfff640 1 --2- 172.21.15.60:0/3556078792 >> v2:172.21.15.100:3300/0 conn(0x7fd6cc0d2910 0x7fd6cc0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374519050 0 0) 0x7fd6cc0b2b10 con 0x7fd6cc15d330 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b4003520 con 0x7fd6cc15d330 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260867459 0 0) 0x7fd6b4003520 con 0x7fd6cc15d330 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 >> v2:172.21.15.100:3300/0 conn(0x7fd6cc0d2910 msgr2=0x7fd6cc0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 --2- 172.21.15.60:0/3556078792 >> v2:172.21.15.100:3300/0 conn(0x7fd6cc0d2910 0x7fd6cc0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 >> v1:172.21.15.100:6789/0 conn(0x7fd6cc0d3730 legacy=0x7fd6cc0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0d1870 con 0x7fd6cc15d330 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fd6bc001670 con 0x7fd6cc15d330 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cbfff640 1 --2- 172.21.15.60:0/3556078792 >> v2:172.21.15.100:3300/0 conn(0x7fd6cc0d2910 0x7fd6cc0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 >> v1:172.21.15.60:6789/0 conn(0x7fd6cc15d330 legacy=0x7fd6cc0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6b4003970 0x7fd6b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b40047c0 0x7fd6b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b4005180 0x7fd6b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0d3bc0 con 0x7fd6b40047c0 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b400c120 con 0x7fd6b4003970 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b400c2e0 con 0x7fd6b4005180 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cbfff640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b40047c0 0x7fd6b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6d0cb9640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6b4003970 0x7fd6b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6d14ba640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b4005180 0x7fd6b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cbfff640 1 -- 172.21.15.60:0/3556078792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b4005180 msgr2=0x7fd6b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cbfff640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b4005180 0x7fd6b400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cbfff640 1 -- 172.21.15.60:0/3556078792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6b4003970 msgr2=0x7fd6b4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:43.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cbfff640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6b4003970 0x7fd6b4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6cbfff640 1 -- 172.21.15.60:0/3556078792 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4003520 con 0x7fd6b40047c0 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6d14ba640 1 -- 172.21.15.60:0/3556078792 reap_dead start 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.424+0000 7fd6d0cb9640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6b4003970 0x7fd6b4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.425+0000 7fd6cbfff640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b40047c0 0x7fd6b4004c40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009c90 tx=0x7fd6b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1d5d9dc56b106 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.425+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c002c8f0 con 0x7fd6b40047c0 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.425+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c002ca50 con 0x7fd6b40047c0 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.425+0000 7fd6cb7fe640 1 -- 172.21.15.60:0/3556078792 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c002ce40 con 0x7fd6b40047c0 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.426+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/3556078792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b40047c0 msgr2=0x7fd6b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.426+0000 7fd6d2f1d640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b40047c0 0x7fd6b4004c40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009c90 tx=0x7fd6b40057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.426+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/3556078792 shutdown_connections 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.426+0000 7fd6d2f1d640 1 --2- 172.21.15.60:0/3556078792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b40047c0 0x7fd6b4004c40 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.426+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/3556078792 >> 172.21.15.60:0/3556078792 conn(0x7fd6cc072020 msgr2=0x7fd6cc072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:43.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.426+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/3556078792 shutdown_connections 2023-11-18T20:44:43.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.426+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/3556078792 wait complete. 2023-11-18T20:44:43.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.427+0000 7fd6d2f1d640 1 Processor -- start 2023-11-18T20:44:43.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.427+0000 7fd6d2f1d640 1 -- start start 2023-11-18T20:44:43.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d2f1d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b40047c0 0x7fd6cc0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d2f1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc0c8140 0x7fd6cc0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d2f1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6cc0b42c0 0x7fd6cc0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d2f1d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0cd190 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d2f1d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0b2d20 con 0x7fd6cc0c8140 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d2f1d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0b2fc0 con 0x7fd6b40047c0 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d14ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6cc0b42c0 0x7fd6cc0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d14ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6cc0b42c0 0x7fd6cc0cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48786/0 (socket says 172.21.15.60:48786) 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d14ba640 1 -- 172.21.15.60:0/719926657 learned_addr learned my addr 172.21.15.60:0/719926657 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6cbfff640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc0c8140 0x7fd6cc0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d14ba640 1 -- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b40047c0 msgr2=0x7fd6cc0c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d14ba640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b40047c0 0x7fd6cc0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d14ba640 1 -- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc0c8140 msgr2=0x7fd6cc0b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.434 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d14ba640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc0c8140 0x7fd6cc0b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d14ba640 1 -- 172.21.15.60:0/719926657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0b32a0 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.428+0000 7fd6d14ba640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6cc0b42c0 0x7fd6cc0cad40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4007fd0 tx=0x7fd6c400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.429+0000 7fd6c97fa640 1 -- 172.21.15.60:0/719926657 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4003970 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.429+0000 7fd6c97fa640 1 -- 172.21.15.60:0/719926657 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4003af0 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.429+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/719926657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc0b3580 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.429+0000 7fd6c97fa640 1 -- 172.21.15.60:0/719926657 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4003ea0 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.429+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/719926657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc0cb280 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.430+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/719926657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6cc0b3aa0 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.430+0000 7fd6c97fa640 1 -- 172.21.15.60:0/719926657 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6c40050d0 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.431+0000 7fd6c97fa640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd6b806ae70 0x7fd6b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.431+0000 7fd6c97fa640 1 -- 172.21.15.60:0/719926657 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6c40893b0 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.431+0000 7fd6d0cb9640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd6b806ae70 0x7fd6b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.431+0000 7fd6d0cb9640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd6b806ae70 0x7fd6b806d330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc000f60 tx=0x7fd6bc001b20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:43.436 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.435+0000 7fd6c97fa640 1 -- 172.21.15.60:0/719926657 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6cc0b3aa0 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.623+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/719926657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd6cc0b3aa0 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.625 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.623+0000 7fd6c97fa640 1 -- 172.21.15.60:0/719926657 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd6cc0b3aa0 con 0x7fd6cc0b42c0 2023-11-18T20:44:43.625 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:44:43.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.625+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd6b806ae70 msgr2=0x7fd6b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.625+0000 7fd6d2f1d640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd6b806ae70 0x7fd6b806d330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc000f60 tx=0x7fd6bc001b20 comp rx=0 tx=0).stop 2023-11-18T20:44:43.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.625+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/719926657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6cc0b42c0 msgr2=0x7fd6cc0cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.625+0000 7fd6d2f1d640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6cc0b42c0 0x7fd6cc0cad40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4007fd0 tx=0x7fd6c400ec10 comp rx=0 tx=0).stop 2023-11-18T20:44:43.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.625+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/719926657 shutdown_connections 2023-11-18T20:44:43.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.625+0000 7fd6d2f1d640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd6b806ae70 0x7fd6b806d330 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.627 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.625+0000 7fd6d2f1d640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6cc0b42c0 0x7fd6cc0cad40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.625+0000 7fd6d2f1d640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6cc0c8140 0x7fd6cc0b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.625+0000 7fd6d2f1d640 1 --2- 172.21.15.60:0/719926657 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b40047c0 0x7fd6cc0c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.625+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/719926657 >> 172.21.15.60:0/719926657 conn(0x7fd6cc072020 msgr2=0x7fd6cc0b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:43.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.626+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/719926657 shutdown_connections 2023-11-18T20:44:43.628 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.626+0000 7fd6d2f1d640 1 -- 172.21.15.60:0/719926657 wait complete. 2023-11-18T20:44:43.637 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-11-18T20:44:43.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:43.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:43.666 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[106639]: fuse finished with error 0 and tester_r 0 2023-11-18T20:44:43.668 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:44:43.712 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:44:43.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:43.712 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-18T20:44:43.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-18T20:44:43.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.873+0000 7f4dbaf05640 1 Processor -- start 2023-11-18T20:44:43.874 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.873+0000 7f4dbaf05640 1 -- start start 2023-11-18T20:44:43.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.873+0000 7f4dbaf05640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4db406ea50 0x7f4db406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.873+0000 7f4dbaf05640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4db4160fb0 0x7f4db4161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4dbaf05640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db4161d20 con 0x7f4db40b3ff0 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4dbaf05640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db40d1ee0 con 0x7f4db406ea50 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4dbaf05640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db406f370 con 0x7f4db4160fb0 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4db8ca1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4db406ea50 0x7f4db406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4db94a2640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4db40b3ff0 0x7f4db40b4480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43552/0 (socket says 172.21.15.60:43552) 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4db94a2640 1 -- 172.21.15.60:0/598222939 learned_addr learned my addr 172.21.15.60:0/598222939 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4db3fff640 1 --2- 172.21.15.60:0/598222939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4db4160fb0 0x7f4db4161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4db8ca1640 1 -- 172.21.15.60:0/598222939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4db4160fb0 msgr2=0x7f4db4161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4db8ca1640 1 --2- 172.21.15.60:0/598222939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4db4160fb0 0x7f4db4161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4db8ca1640 1 -- 172.21.15.60:0/598222939 >> v1:172.21.15.60:6789/0 conn(0x7f4db40b3ff0 legacy=0x7f4db40b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4db37fe640 1 -- 172.21.15.60:0/598222939 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354709343 0 0) 0x7f4db4161d20 con 0x7f4db40b3ff0 2023-11-18T20:44:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.874+0000 7f4db8ca1640 1 -- 172.21.15.60:0/598222939 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db415e760 con 0x7f4db406ea50 2023-11-18T20:44:43.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.875+0000 7f4db8ca1640 1 --2- 172.21.15.60:0/598222939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db406ea50 0x7f4db406ee30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4da80097a0 tx=0x7f4da802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f83278af933eb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:43.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.875+0000 7f4db37fe640 1 -- 172.21.15.60:0/598222939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8031830 con 0x7f4db406ea50 2023-11-18T20:44:43.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.875+0000 7f4db37fe640 1 -- 172.21.15.60:0/598222939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da8031990 con 0x7f4db406ea50 2023-11-18T20:44:43.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.875+0000 7f4db37fe640 1 -- 172.21.15.60:0/598222939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8031d20 con 0x7f4db406ea50 2023-11-18T20:44:43.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.875+0000 7f4dbaf05640 1 -- 172.21.15.60:0/598222939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db406ea50 msgr2=0x7f4db406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.875+0000 7f4dbaf05640 1 --2- 172.21.15.60:0/598222939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db406ea50 0x7f4db406ee30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4da80097a0 tx=0x7f4da802b8d0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.875+0000 7f4dbaf05640 1 -- 172.21.15.60:0/598222939 shutdown_connections 2023-11-18T20:44:43.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.875+0000 7f4dbaf05640 1 --2- 172.21.15.60:0/598222939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4db4160fb0 0x7f4db4161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.875+0000 7f4dbaf05640 1 --2- 172.21.15.60:0/598222939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db406ea50 0x7f4db406ee30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.875+0000 7f4dbaf05640 1 -- 172.21.15.60:0/598222939 >> 172.21.15.60:0/598222939 conn(0x7f4db406a6b0 msgr2=0x7f4db406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.876+0000 7f4dbaf05640 1 -- 172.21.15.60:0/598222939 shutdown_connections 2023-11-18T20:44:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.876+0000 7f4dbaf05640 1 -- 172.21.15.60:0/598222939 wait complete. 2023-11-18T20:44:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.877+0000 7f4dbaf05640 1 Processor -- start 2023-11-18T20:44:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.877+0000 7f4dbaf05640 1 -- start start 2023-11-18T20:44:43.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4dbaf05640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db406ea50 0x7f4db413f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4dbaf05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db40b3ff0 0x7f4db41e8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4dbaf05640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4db4160fb0 0x7f4db4159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4dbaf05640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41e98d0 con 0x7f4db40b3ff0 2023-11-18T20:44:43.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4dbaf05640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41e8830 con 0x7f4db406ea50 2023-11-18T20:44:43.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4dbaf05640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41e8b10 con 0x7f4db4160fb0 2023-11-18T20:44:43.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db40b3ff0 0x7f4db41e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db40b3ff0 0x7f4db41e8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48796/0 (socket says 172.21.15.60:48796) 2023-11-18T20:44:43.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db3fff640 1 -- 172.21.15.60:0/2172182903 learned_addr learned my addr 172.21.15.60:0/2172182903 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db8ca1640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db406ea50 0x7f4db413f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db94a2640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4db4160fb0 0x7f4db4159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db3fff640 1 -- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4db4160fb0 msgr2=0x7f4db4159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db3fff640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4db4160fb0 0x7f4db4159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db3fff640 1 -- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db406ea50 msgr2=0x7f4db413f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db3fff640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db406ea50 0x7f4db413f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db3fff640 1 -- 172.21.15.60:0/2172182903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da8009400 con 0x7f4db40b3ff0 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db8ca1640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db406ea50 0x7f4db413f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db3fff640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db40b3ff0 0x7f4db41e8150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4da000c7f0 tx=0x7f4da000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db17fa640 1 -- 172.21.15.60:0/2172182903 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0004220 con 0x7f4db40b3ff0 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db17fa640 1 -- 172.21.15.60:0/2172182903 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da00043a0 con 0x7f4db40b3ff0 2023-11-18T20:44:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4db17fa640 1 -- 172.21.15.60:0/2172182903 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0007860 con 0x7f4db40b3ff0 2023-11-18T20:44:43.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4dbaf05640 1 -- 172.21.15.60:0/2172182903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db41e8df0 con 0x7f4db40b3ff0 2023-11-18T20:44:43.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.878+0000 7f4dbaf05640 1 -- 172.21.15.60:0/2172182903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db41e93b0 con 0x7f4db40b3ff0 2023-11-18T20:44:43.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.879+0000 7f4dbaf05640 1 -- 172.21.15.60:0/2172182903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db41e95a0 con 0x7f4db40b3ff0 2023-11-18T20:44:43.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.880+0000 7f4db17fa640 1 -- 172.21.15.60:0/2172182903 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4da0007bd0 con 0x7f4db40b3ff0 2023-11-18T20:44:43.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.880+0000 7f4db17fa640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d9806ae70 0x7f4d9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:43.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.880+0000 7f4db17fa640 1 -- 172.21.15.60:0/2172182903 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4da00895d0 con 0x7f4db40b3ff0 2023-11-18T20:44:43.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.880+0000 7f4db8ca1640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d9806ae70 0x7f4d9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:43.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.881+0000 7f4db8ca1640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d9806ae70 0x7f4d9806d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4da8036000 tx=0x7f4da803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:43.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:43.885+0000 7f4db17fa640 1 -- 172.21.15.60:0/2172182903 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4da0054680 con 0x7f4db40b3ff0 2023-11-18T20:44:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.099+0000 7f4dbaf05640 1 -- 172.21.15.60:0/2172182903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4db4069400 con 0x7f4db40b3ff0 2023-11-18T20:44:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.100+0000 7f4db17fa640 1 -- 172.21.15.60:0/2172182903 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4da0057ca0 con 0x7f4db40b3ff0 2023-11-18T20:44:44.101 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-18T20:44:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.101+0000 7f4dbaf05640 1 -- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d9806ae70 msgr2=0x7f4d9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.101+0000 7f4dbaf05640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d9806ae70 0x7f4d9806d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4da8036000 tx=0x7f4da803c040 comp rx=0 tx=0).stop 2023-11-18T20:44:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.101+0000 7f4dbaf05640 1 -- 172.21.15.60:0/2172182903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db40b3ff0 msgr2=0x7f4db41e8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.101+0000 7f4dbaf05640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db40b3ff0 0x7f4db41e8150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4da000c7f0 tx=0x7f4da000ccb0 comp rx=0 tx=0).stop 2023-11-18T20:44:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.102+0000 7f4dbaf05640 1 -- 172.21.15.60:0/2172182903 shutdown_connections 2023-11-18T20:44:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.102+0000 7f4dbaf05640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4d9806ae70 0x7f4d9806d330 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.102+0000 7f4dbaf05640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4db4160fb0 0x7f4db4159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.102+0000 7f4dbaf05640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4db40b3ff0 0x7f4db41e8150 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:44.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.102+0000 7f4dbaf05640 1 --2- 172.21.15.60:0/2172182903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4db406ea50 0x7f4db413f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:44.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.102+0000 7f4dbaf05640 1 -- 172.21.15.60:0/2172182903 >> 172.21.15.60:0/2172182903 conn(0x7f4db406a6b0 msgr2=0x7f4db415ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:44.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.102+0000 7f4dbaf05640 1 -- 172.21.15.60:0/2172182903 shutdown_connections 2023-11-18T20:44:44.104 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:44.102+0000 7f4dbaf05640 1 -- 172.21.15.60:0/2172182903 wait complete. 2023-11-18T20:44:44.113 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-18T20:44:44.114 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-18T20:44:44.114 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-11-18T20:44:44.114 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-18T20:44:44.114 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:44.114 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-18T20:44:44.114 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-18T20:44:44.114 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-18T20:44:44.114 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:44.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:44.114 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-11-18T20:44:44.130 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:44:44.130 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:44:44.130 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:44.131 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42572sec preferred_lft 42572sec 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:44.132 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::d896:cfff:fef3:88dc/64 scope link 2023-11-18T20:44:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:44.133 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:44:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0a:c3:0c:c2:e6:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8c3:cff:fec2:e672/64 scope link 2023-11-18T20:44:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:44.133 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:44:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::40c:deff:fe9d:c1f6/64 scope link 2023-11-18T20:44:44.133 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:44.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:44.134 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-18T20:44:44.202 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-11-18T20:44:44.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:44.203 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-11-18T20:44:44.259 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-11-18T20:44:44.259 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-18T20:44:44.259 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-18T20:44:44.259 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-18T20:44:44.259 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-18T20:44:44.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:44.260 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:44:44.260 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:44:44.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-18T20:44:44.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-18T20:44:44.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-18T20:44:44.260 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:44:44.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:44.366 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-11-18T20:44:44.381 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-18T20:44:44.381 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-18T20:44:44.381 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:44.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:44.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-11-18T20:44:44.452 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-18T20:44:44.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:44:44.452 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:44.507 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:44.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:44:44.507 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:44.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:44:44.634 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:44:44.690 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:44:44.690 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:44:44.690 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:44:44.691 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:44:44.691 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:44:44.691 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:44:44.691 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:44:44.691 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:44:44.691 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:44:44.691 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:44:44.692 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:44:44.692 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:44:44.692 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:44:44.692 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:44:44.692 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:44:44.692 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:44:44.692 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:44:44.693 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:44:44.693 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:44:44.693 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:44:44.693 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:44:44.693 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:44:44.693 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:44:44.694 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:44:44.694 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:44:44.694 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:44:44.694 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:44:44.694 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:44:44.694 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:44:44.694 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:44:44.695 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:44.695 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:44.695 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:44.695 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:44.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:44.696 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:44:44.746 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-18T20:44:44.746 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-18T20:44:44.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:44:44.818 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:44:44.875 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:44:44.875 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:44:44.875 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:44:44.875 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:44:44.876 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:44:44.877 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:44:44.878 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:44:44.878 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:44:44.878 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:44.878 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:44.878 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:44.878 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:44.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:44.878 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:44:44.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-11-18T20:44:44.880+0000 7f145b733480 -1 init, newargv = 0x563fc99f7910 newargc=15 2023-11-18T20:44:44.882 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[107601]: starting ceph client 2023-11-18T20:44:44.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[107601]: starting fuse 2023-11-18T20:44:44.897 INFO:teuthology.orchestra.run.smithi060.stdout:41 2023-11-18T20:44:44.898 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-18T20:44:44.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2023-11-18T20:44:44.898 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-11-18T20:44:44.898 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-11-18T20:44:44.898 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-11-18T20:44:44.898 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-11-18T20:44:44.898 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:44.898 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-18T20:44:44.899 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-18T20:44:44.900 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-18T20:44:44.900 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-11-18T20:44:44.900 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-11-18T20:44:44.900 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-11-18T20:44:44.900 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-18T20:44:44.900 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:44.900 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-11-18T20:44:44.900 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-11-18T20:44:45.022 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.107601.asok 2023-11-18T20:44:45.025 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:44:45.033 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.107601.asok 2023-11-18T20:44:45.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:45.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.107601.asok status 2023-11-18T20:44:45.174 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:44:45.174 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-11-18T20:44:45.175 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe", 2023-11-18T20:44:45.175 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-7418-gb0f45a51 (b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe) reef (dev)", 2023-11-18T20:44:45.175 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-11-18T20:44:45.175 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-11-18T20:44:45.175 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-18T20:44:45.175 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "107601", 2023-11-18T20:44:45.175 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-11-18T20:44:45.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:45.175 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-11-18T20:44:45.176 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-11-18T20:44:45.176 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6132, 2023-11-18T20:44:45.176 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-11-18T20:44:45.176 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-11-18T20:44:45.176 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-11-18T20:44:45.176 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6132 2023-11-18T20:44:45.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3952659651 2023-11-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3952659651 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6132 192.168.0.1:0/3952659651", 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/3952659651", 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 21, 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 49, 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-11-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:44:45.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:45.184 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-11-18T20:44:45.184 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:45.199 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-11-18T20:44:45.199 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:45.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:45.200 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:45.270 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2023-11-18T20:44:45.270 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-11-18T20:44:45.270 DEBUG:teuthology.orchestra.run.smithi060:> import errno 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> ret = os.read(fd, 1024) 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> else: 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> os.write(fd, b'"'"'content'"'"') 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> except OSError as e: 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> if e.errno != errno.EPERM: 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> raise 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> else: 2023-11-18T20:44:45.271 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("client does not check permission of data pool") 2023-11-18T20:44:45.272 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-11-18T20:44:45.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:44:45.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2023-11-18T20:44:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.540+0000 7fa20483f640 1 Processor -- start 2023-11-18T20:44:45.542 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.540+0000 7fa20483f640 1 -- start start 2023-11-18T20:44:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.540+0000 7fa20483f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa20015d4d0 0x7fa2000d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.541+0000 7fa20483f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20015d8b0 con 0x7fa2000b4cd0 2023-11-18T20:44:45.543 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.541+0000 7fa20483f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000d1ae0 con 0x7fa2000b8ec0 2023-11-18T20:44:45.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.541+0000 7fa20483f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2000d1dc0 con 0x7fa20015d4d0 2023-11-18T20:44:45.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.541+0000 7fa1fe59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa20015d4d0 0x7fa2000d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:45.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.541+0000 7fa1fed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa2000b4cd0 0x7fa2000b6380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43554/0 (socket says 172.21.15.60:43554) 2023-11-18T20:44:45.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.541+0000 7fa1fed9d640 1 -- 172.21.15.60:0/1732091763 learned_addr learned my addr 172.21.15.60:0/1732091763 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:45.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.541+0000 7fa1fe59c640 1 -- 172.21.15.60:0/1732091763 >> v1:172.21.15.100:6789/0 conn(0x7fa2000b8ec0 legacy=0x7fa2000d1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.541+0000 7fa1fe59c640 1 -- 172.21.15.60:0/1732091763 >> v1:172.21.15.60:6789/0 conn(0x7fa2000b4cd0 legacy=0x7fa2000b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.541+0000 7fa1fe59c640 1 -- 172.21.15.60:0/1732091763 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000d20a0 con 0x7fa20015d4d0 2023-11-18T20:44:45.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.541+0000 7fa1fdd9b640 1 -- 172.21.15.60:0/1732091763 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847353119 0 0) 0x7fa20015d8b0 con 0x7fa2000b4cd0 2023-11-18T20:44:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.542+0000 7fa1fe59c640 1 --2- 172.21.15.60:0/1732091763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 0x7fa2000d2b80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00097a0 tx=0x7fa1f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4f54d7535a4070 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.542+0000 7fa1fdd9b640 1 -- 172.21.15.60:0/1732091763 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0030610 con 0x7fa20015d4d0 2023-11-18T20:44:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.542+0000 7fa1fdd9b640 1 -- 172.21.15.60:0/1732091763 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0030770 con 0x7fa20015d4d0 2023-11-18T20:44:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.542+0000 7fa1fdd9b640 1 -- 172.21.15.60:0/1732091763 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0030b00 con 0x7fa20015d4d0 2023-11-18T20:44:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.542+0000 7fa20483f640 1 -- 172.21.15.60:0/1732091763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 msgr2=0x7fa2000d2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.542+0000 7fa20483f640 1 --2- 172.21.15.60:0/1732091763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 0x7fa2000d2b80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00097a0 tx=0x7fa1f002b960 comp rx=0 tx=0).stop 2023-11-18T20:44:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.543+0000 7fa20483f640 1 -- 172.21.15.60:0/1732091763 shutdown_connections 2023-11-18T20:44:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.543+0000 7fa20483f640 1 --2- 172.21.15.60:0/1732091763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 0x7fa2000d2b80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.543+0000 7fa20483f640 1 -- 172.21.15.60:0/1732091763 >> 172.21.15.60:0/1732091763 conn(0x7fa200072230 msgr2=0x7fa2000cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.543+0000 7fa20483f640 1 -- 172.21.15.60:0/1732091763 shutdown_connections 2023-11-18T20:44:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.543+0000 7fa20483f640 1 -- 172.21.15.60:0/1732091763 wait complete. 2023-11-18T20:44:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.544+0000 7fa20483f640 1 Processor -- start 2023-11-18T20:44:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.544+0000 7fa20483f640 1 -- start start 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa20483f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2000b4cd0 0x7fa20014ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa20483f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2000b8ec0 0x7fa2000b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa20483f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 0x7fa20014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa20483f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20015fad0 con 0x7fa20015d4d0 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa20483f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20015ec40 con 0x7fa2000b4cd0 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa20483f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20015ee50 con 0x7fa2000b8ec0 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1fed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 0x7fa20014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1fed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 0x7fa20014b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48804/0 (socket says 172.21.15.60:48804) 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1fed9d640 1 -- 172.21.15.60:0/2837427891 learned_addr learned my addr 172.21.15.60:0/2837427891 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1fe59c640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2000b4cd0 0x7fa20014ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1f7fff640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2000b8ec0 0x7fa2000b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1fed9d640 1 -- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2000b8ec0 msgr2=0x7fa2000b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1fed9d640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2000b8ec0 0x7fa2000b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1fed9d640 1 -- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2000b4cd0 msgr2=0x7fa20014ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1fed9d640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2000b4cd0 0x7fa20014ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1fed9d640 1 -- 172.21.15.60:0/2837427891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0009400 con 0x7fa20015d4d0 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1fed9d640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 0x7fa20014b2b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c7e0 tx=0x7fa1f800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1f6ffd640 1 -- 172.21.15.60:0/2837427891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800a640 con 0x7fa20015d4d0 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1f6ffd640 1 -- 172.21.15.60:0/2837427891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f800a7c0 con 0x7fa20015d4d0 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa1f6ffd640 1 -- 172.21.15.60:0/2837427891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800ab70 con 0x7fa20015d4d0 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa20483f640 1 -- 172.21.15.60:0/2837427891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20015f130 con 0x7fa20015d4d0 2023-11-18T20:44:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.545+0000 7fa20483f640 1 -- 172.21.15.60:0/2837427891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20015f6f0 con 0x7fa20015d4d0 2023-11-18T20:44:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.546+0000 7fa20483f640 1 -- 172.21.15.60:0/2837427891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa200069400 con 0x7fa20015d4d0 2023-11-18T20:44:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.547+0000 7fa1f6ffd640 1 -- 172.21.15.60:0/2837427891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1f800acd0 con 0x7fa20015d4d0 2023-11-18T20:44:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.547+0000 7fa1f6ffd640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa1e006ae40 0x7fa1e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.547+0000 7fa1f6ffd640 1 -- 172.21.15.60:0/2837427891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1f8088b40 con 0x7fa20015d4d0 2023-11-18T20:44:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.547+0000 7fa1fe59c640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa1e006ae40 0x7fa1e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.548+0000 7fa1fe59c640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa1e006ae40 0x7fa1e006d300 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0036000 tx=0x7fa1f003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.551+0000 7fa1f6ffd640 1 -- 172.21.15.60:0/2837427891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1f8053ca0 con 0x7fa20015d4d0 2023-11-18T20:44:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.795+0000 7fa20483f640 1 -- 172.21.15.60:0/2837427891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) v1 -- 0x7fa20015a6b0 con 0x7fa20015d4d0 2023-11-18T20:44:45.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.797+0000 7fa1f6ffd640 1 -- 172.21.15.60:0/2837427891 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v19) v1 ==== 183+0+140 (secure 0 0 0) 0x7fa1f8018070 con 0x7fa20015d4d0 2023-11-18T20:44:45.799 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-11-18T20:44:45.800 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA== 2023-11-18T20:44:45.800 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-11-18T20:44:45.800 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-11-18T20:44:45.800 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow w pool=cephfs_data" 2023-11-18T20:44:45.800 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-11-18T20:44:45.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 -- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa1e006ae40 msgr2=0x7fa1e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa1e006ae40 0x7fa1e006d300 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0036000 tx=0x7fa1f003b040 comp rx=0 tx=0).stop 2023-11-18T20:44:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 -- 172.21.15.60:0/2837427891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 msgr2=0x7fa20014b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 0x7fa20014b2b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c7e0 tx=0x7fa1f800cca0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 -- 172.21.15.60:0/2837427891 shutdown_connections 2023-11-18T20:44:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa1e006ae40 0x7fa1e006d300 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa20015d4d0 0x7fa20014b2b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2000b8ec0 0x7fa2000b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 --2- 172.21.15.60:0/2837427891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2000b4cd0 0x7fa20014ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 -- 172.21.15.60:0/2837427891 >> 172.21.15.60:0/2837427891 conn(0x7fa200072230 msgr2=0x7fa200154870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:45.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 -- 172.21.15.60:0/2837427891 shutdown_connections 2023-11-18T20:44:45.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.800+0000 7fa20483f640 1 -- 172.21.15.60:0/2837427891 wait complete. 2023-11-18T20:44:45.813 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:45.813 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:45.828 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.972+0000 7f7c7b79e640 1 Processor -- start 2023-11-18T20:44:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.973+0000 7f7c7b79e640 1 -- start start 2023-11-18T20:44:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.973+0000 7f7c7b79e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7c740d2910 0x7f7c740d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.973+0000 7f7c7b79e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740b2b10 con 0x7f7c7415d330 2023-11-18T20:44:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.973+0000 7f7c7b79e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740d1590 con 0x7f7c740d3730 2023-11-18T20:44:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.973+0000 7f7c7b79e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740d3bc0 con 0x7f7c740d2910 2023-11-18T20:44:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.973+0000 7f7c78d39640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7c740d2910 0x7f7c740d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.974+0000 7f7c7953a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7c7415d330 0x7f7c740b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43568/0 (socket says 172.21.15.60:43568) 2023-11-18T20:44:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.974+0000 7f7c78d39640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7c740d2910 0x7f7c740d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48820/0 (socket says 172.21.15.60:48820) 2023-11-18T20:44:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.974+0000 7f7c7953a640 1 -- 172.21.15.60:0/1684484337 learned_addr learned my addr 172.21.15.60:0/1684484337 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.974+0000 7f7c78d39640 1 -- 172.21.15.60:0/1684484337 >> v1:172.21.15.100:6789/0 conn(0x7f7c740d3730 legacy=0x7f7c740b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.974+0000 7f7c78d39640 1 -- 172.21.15.60:0/1684484337 >> v1:172.21.15.60:6789/0 conn(0x7f7c7415d330 legacy=0x7f7c740b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.974+0000 7f7c78d39640 1 -- 172.21.15.60:0/1684484337 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740d1870 con 0x7f7c740d2910 2023-11-18T20:44:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.974+0000 7f7c6bfff640 1 -- 172.21.15.60:0/1684484337 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634448914 0 0) 0x7f7c740b2b10 con 0x7f7c7415d330 2023-11-18T20:44:45.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c78d39640 1 --2- 172.21.15.60:0/1684484337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c740d2910 0x7f7c740d31f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7c640097a0 tx=0x7f7c6402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a7c1bf82a5cb7b server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:45.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c6bfff640 1 -- 172.21.15.60:0/1684484337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6402e070 con 0x7f7c740d2910 2023-11-18T20:44:45.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c6bfff640 1 -- 172.21.15.60:0/1684484337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c640315a0 con 0x7f7c740d2910 2023-11-18T20:44:45.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c6bfff640 1 -- 172.21.15.60:0/1684484337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64031930 con 0x7f7c740d2910 2023-11-18T20:44:45.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1684484337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c740d2910 msgr2=0x7f7c740d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.978 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c7b79e640 1 --2- 172.21.15.60:0/1684484337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c740d2910 0x7f7c740d31f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7c640097a0 tx=0x7f7c6402b840 comp rx=0 tx=0).stop 2023-11-18T20:44:45.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1684484337 shutdown_connections 2023-11-18T20:44:45.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c7b79e640 1 --2- 172.21.15.60:0/1684484337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c740d2910 0x7f7c740d31f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1684484337 >> 172.21.15.60:0/1684484337 conn(0x7f7c74072020 msgr2=0x7f7c74072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:45.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1684484337 shutdown_connections 2023-11-18T20:44:45.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.975+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1684484337 wait complete. 2023-11-18T20:44:45.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c7b79e640 1 Processor -- start 2023-11-18T20:44:45.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c7b79e640 1 -- start start 2023-11-18T20:44:45.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c7b79e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c740d2910 0x7f7c741543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:45.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c7b79e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740d3730 0x7f7c740b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:45.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c7b79e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c7415d330 0x7f7c740b8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:45.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c7b79e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74154930 con 0x7f7c7415d330 2023-11-18T20:44:45.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c7b79e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740cd120 con 0x7f7c740d3730 2023-11-18T20:44:45.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c7b79e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7415d8d0 con 0x7f7c740d2910 2023-11-18T20:44:45.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c79d3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c7415d330 0x7f7c740b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:45.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c79d3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c7415d330 0x7f7c740b8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48826/0 (socket says 172.21.15.60:48826) 2023-11-18T20:44:45.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.977+0000 7f7c79d3b640 1 -- 172.21.15.60:0/1335788510 learned_addr learned my addr 172.21.15.60:0/1335788510 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:45.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c78d39640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740d3730 0x7f7c740b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:45.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c7953a640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c740d2910 0x7f7c741543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:45.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c79d3b640 1 -- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c740d2910 msgr2=0x7f7c741543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c79d3b640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c740d2910 0x7f7c741543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c79d3b640 1 -- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740d3730 msgr2=0x7f7c740b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:45.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c79d3b640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740d3730 0x7f7c740b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c79d3b640 1 -- 172.21.15.60:0/1335788510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c64009400 con 0x7f7c7415d330 2023-11-18T20:44:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c7953a640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c740d2910 0x7f7c741543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c79d3b640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c7415d330 0x7f7c740b8740 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c780 tx=0x7f7c7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c69ffb640 1 -- 172.21.15.60:0/1335788510 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70004b60 con 0x7f7c7415d330 2023-11-18T20:44:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c69ffb640 1 -- 172.21.15.60:0/1335788510 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7000ce90 con 0x7f7c7415d330 2023-11-18T20:44:45.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c69ffb640 1 -- 172.21.15.60:0/1335788510 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7000a850 con 0x7f7c7415d330 2023-11-18T20:44:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1335788510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c740cc880 con 0x7f7c7415d330 2023-11-18T20:44:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.978+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1335788510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c740cce40 con 0x7f7c7415d330 2023-11-18T20:44:45.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.979+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1335788510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7415da40 con 0x7f7c7415d330 2023-11-18T20:44:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.980+0000 7f7c69ffb640 1 -- 172.21.15.60:0/1335788510 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c7000a9b0 con 0x7f7c7415d330 2023-11-18T20:44:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.980+0000 7f7c69ffb640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7c5806ae40 0x7f7c5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.980+0000 7f7c69ffb640 1 -- 172.21.15.60:0/1335788510 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c70089760 con 0x7f7c7415d330 2023-11-18T20:44:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.980+0000 7f7c7953a640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7c5806ae40 0x7f7c5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:45.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.981+0000 7f7c7953a640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7c5806ae40 0x7f7c5806d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c000f60 tx=0x7f7c6c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:45.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:45.984+0000 7f7c69ffb640 1 -- 172.21.15.60:0/1335788510 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c700548a0 con 0x7f7c7415d330 2023-11-18T20:44:46.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.160+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1335788510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c74069400 con 0x7f7c7415d330 2023-11-18T20:44:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.161+0000 7f7c69ffb640 1 -- 172.21.15.60:0/1335788510 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f7c70057ec0 con 0x7f7c7415d330 2023-11-18T20:44:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:46.163 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:34.103787+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7c5806ae40 msgr2=0x7f7c5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7c5806ae40 0x7f7c5806d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c000f60 tx=0x7f7c6c009040 comp rx=0 tx=0).stop 2023-11-18T20:44:46.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1335788510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c7415d330 msgr2=0x7f7c740b8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c7415d330 0x7f7c740b8740 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c780 tx=0x7f7c7000cc40 comp rx=0 tx=0).stop 2023-11-18T20:44:46.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1335788510 shutdown_connections 2023-11-18T20:44:46.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f7c5806ae40 0x7f7c5806d300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c7415d330 0x7f7c740b8740 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c740d3730 0x7f7c740b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 --2- 172.21.15.60:0/1335788510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c740d2910 0x7f7c741543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1335788510 >> 172.21.15.60:0/1335788510 conn(0x7f7c74072020 msgr2=0x7f7c74072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:46.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1335788510 shutdown_connections 2023-11-18T20:44:46.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.163+0000 7f7c7b79e640 1 -- 172.21.15.60:0/1335788510 wait complete. 2023-11-18T20:44:46.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:44:31.358325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:44:32.036538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:44:46.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:44:46.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.283+0000 7fba028af640 1 Processor -- start 2023-11-18T20:44:46.286 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.284+0000 7fba028af640 1 -- start start 2023-11-18T20:44:46.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.284+0000 7fba028af640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc0d2910 0x7fb9fc0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.284+0000 7fba028af640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc0b2b10 con 0x7fb9fc0d3730 2023-11-18T20:44:46.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.284+0000 7fba028af640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc0d3b50 con 0x7fb9fc15d330 2023-11-18T20:44:46.287 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.284+0000 7fba028af640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0d1590 con 0x7fb9fc0d2910 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.284+0000 7fba00e4c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb9fc0d3730 0x7fb9fc0b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43584/0 (socket says 172.21.15.60:43584) 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.285+0000 7fba00e4c640 1 -- 172.21.15.60:0/1901089606 learned_addr learned my addr 172.21.15.60:0/1901089606 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.285+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/1901089606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc0d2910 0x7fb9fc0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.285+0000 7fb9faffd640 1 -- 172.21.15.60:0/1901089606 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516909267 0 0) 0x7fb9fc0b2b10 con 0x7fb9fc0d3730 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.285+0000 7fb9faffd640 1 -- 172.21.15.60:0/1901089606 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9e4003520 con 0x7fb9fc0d3730 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.285+0000 7fb9faffd640 1 -- 172.21.15.60:0/1901089606 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198546407 0 0) 0x7fb9fc0d3b50 con 0x7fb9fc15d330 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.285+0000 7fb9faffd640 1 -- 172.21.15.60:0/1901089606 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9fc0b2b10 con 0x7fb9fc15d330 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.285+0000 7fb9faffd640 1 -- 172.21.15.60:0/1901089606 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2173863168 0 0) 0x7fb9e4003520 con 0x7fb9fc0d3730 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.285+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/1901089606 >> v1:172.21.15.100:6789/0 conn(0x7fb9fc15d330 legacy=0x7fb9fc0b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.285+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/1901089606 >> v1:172.21.15.60:6789/0 conn(0x7fb9fc0d3730 legacy=0x7fb9fc0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.285+0000 7fb9fb7fe640 1 -- 172.21.15.60:0/1901089606 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0d1870 con 0x7fb9fc0d2910 2023-11-18T20:44:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.286+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/1901089606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc0d2910 0x7fb9fc0d31f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00097d0 tx=0x7fb9f002b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd14fdb0e5e5b74e server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.286+0000 7fb9faffd640 1 -- 172.21.15.60:0/1901089606 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f40019f0 con 0x7fb9fc0d2910 2023-11-18T20:44:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.286+0000 7fb9faffd640 1 -- 172.21.15.60:0/1901089606 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f002e050 con 0x7fb9fc0d2910 2023-11-18T20:44:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.286+0000 7fb9faffd640 1 -- 172.21.15.60:0/1901089606 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0030730 con 0x7fb9fc0d2910 2023-11-18T20:44:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.287+0000 7fba028af640 1 -- 172.21.15.60:0/1901089606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc0d2910 msgr2=0x7fb9fc0d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.287+0000 7fba028af640 1 --2- 172.21.15.60:0/1901089606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc0d2910 0x7fb9fc0d31f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00097d0 tx=0x7fb9f002b770 comp rx=0 tx=0).stop 2023-11-18T20:44:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.287+0000 7fba028af640 1 -- 172.21.15.60:0/1901089606 shutdown_connections 2023-11-18T20:44:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.287+0000 7fba028af640 1 --2- 172.21.15.60:0/1901089606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc0d2910 0x7fb9fc0d31f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.287+0000 7fba028af640 1 -- 172.21.15.60:0/1901089606 >> 172.21.15.60:0/1901089606 conn(0x7fb9fc072020 msgr2=0x7fb9fc072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.287+0000 7fba028af640 1 -- 172.21.15.60:0/1901089606 shutdown_connections 2023-11-18T20:44:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.287+0000 7fba028af640 1 -- 172.21.15.60:0/1901089606 wait complete. 2023-11-18T20:44:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.288+0000 7fba028af640 1 Processor -- start 2023-11-18T20:44:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.288+0000 7fba028af640 1 -- start start 2023-11-18T20:44:46.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fba028af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc0d2910 0x7fb9fc15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fba028af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9fc0d3730 0x7fb9fc0b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.293 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fba028af640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc15d330 0x7fb9fc0b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fba028af640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc154ab0 con 0x7fb9fc0d2910 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fba028af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc156f30 con 0x7fb9fc0d3730 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fba028af640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc157210 con 0x7fb9fc15d330 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9fbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc0d2910 0x7fb9fc15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9fbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc0d2910 0x7fb9fc15f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48842/0 (socket says 172.21.15.60:48842) 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9fbfff640 1 -- 172.21.15.60:0/2998047737 learned_addr learned my addr 172.21.15.60:0/2998047737 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9fc0d3730 0x7fb9fc0b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fba00e4c640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc15d330 0x7fb9fc0b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9fbfff640 1 -- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc15d330 msgr2=0x7fb9fc0b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9fbfff640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc15d330 0x7fb9fc0b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9fbfff640 1 -- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9fc0d3730 msgr2=0x7fb9fc0b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9fbfff640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9fc0d3730 0x7fb9fc0b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9fbfff640 1 -- 172.21.15.60:0/2998047737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f0009400 con 0x7fb9fc0d2910 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fba00e4c640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc15d330 0x7fb9fc0b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9fbfff640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc0d2910 0x7fb9fc15f820 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c430 tx=0x7fb9ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/2998047737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0052d0 con 0x7fb9fc0d2910 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/2998047737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec005450 con 0x7fb9fc0d2910 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fba028af640 1 -- 172.21.15.60:0/2998047737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc1574f0 con 0x7fb9fc0d2910 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.289+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/2998047737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec003830 con 0x7fb9fc0d2910 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.290+0000 7fba028af640 1 -- 172.21.15.60:0/2998047737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc157ab0 con 0x7fb9fc0d2910 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.291+0000 7fba028af640 1 -- 172.21.15.60:0/2998047737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc069400 con 0x7fb9fc0d2910 2023-11-18T20:44:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.291+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/2998047737 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9ec003990 con 0x7fb9fc0d2910 2023-11-18T20:44:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.291+0000 7fb9f8ff9640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae70 0x7fb9e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.292+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/2998047737 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9ec088bf0 con 0x7fb9fc0d2910 2023-11-18T20:44:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.292+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae70 0x7fb9e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.292+0000 7fb9fb7fe640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae70 0x7fb9e806d330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0034000 tx=0x7fb9f0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:46.297 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.296+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/2998047737 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9ec053cd0 con 0x7fb9fc0d2910 2023-11-18T20:44:46.475 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.473+0000 7fba028af640 1 -- 172.21.15.60:0/2998047737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9fc0d11f0 con 0x7fb9fc0d2910 2023-11-18T20:44:46.476 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.473+0000 7fb9f8ff9640 1 -- 172.21.15.60:0/2998047737 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9ec0572f0 con 0x7fb9fc0d2910 2023-11-18T20:44:46.476 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:44:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 -- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae70 msgr2=0x7fb9e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae70 0x7fb9e806d330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0034000 tx=0x7fb9f0008030 comp rx=0 tx=0).stop 2023-11-18T20:44:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 -- 172.21.15.60:0/2998047737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc0d2910 msgr2=0x7fb9fc15f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc0d2910 0x7fb9fc15f820 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c430 tx=0x7fb9ec00c8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 -- 172.21.15.60:0/2998047737 shutdown_connections 2023-11-18T20:44:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb9e806ae70 0x7fb9e806d330 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9fc15d330 0x7fb9fc0b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9fc0d3730 0x7fb9fc0b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 --2- 172.21.15.60:0/2998047737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9fc0d2910 0x7fb9fc15f820 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 -- 172.21.15.60:0/2998047737 >> 172.21.15.60:0/2998047737 conn(0x7fb9fc072020 msgr2=0x7fb9fc0b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 -- 172.21.15.60:0/2998047737 shutdown_connections 2023-11-18T20:44:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.476+0000 7fba028af640 1 -- 172.21.15.60:0/2998047737 wait complete. 2023-11-18T20:44:46.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:44:46.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.595+0000 7ff87d67c640 1 Processor -- start 2023-11-18T20:44:46.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.595+0000 7ff87d67c640 1 -- start start 2023-11-18T20:44:46.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.596+0000 7ff87d67c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87815d270 0x7ff87806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.596+0000 7ff87d67c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8780709c0 0x7ff8780cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.596+0000 7ff87d67c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8780d0c40 con 0x7ff8780cdd10 2023-11-18T20:44:46.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.596+0000 7ff87d67c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8780ce1a0 con 0x7ff8780709c0 2023-11-18T20:44:46.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.596+0000 7ff87d67c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780ced90 con 0x7ff87815d270 2023-11-18T20:44:46.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff8767fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8780709c0 0x7ff8780cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff876ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87815d270 0x7ff87806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff8777fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff8780cdd10 0x7ff8780cc6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:33564/0 (socket says 172.21.15.60:33564) 2023-11-18T20:44:46.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff8777fe640 1 -- 172.21.15.60:0/270055392 learned_addr learned my addr 172.21.15.60:0/270055392 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:46.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff8767fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8780709c0 0x7ff8780cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53948/0 (socket says 172.21.15.60:53948) 2023-11-18T20:44:46.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff875ffb640 1 -- 172.21.15.60:0/270055392 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664040816 0 0) 0x7ff8780d0c40 con 0x7ff8780cdd10 2023-11-18T20:44:46.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff875ffb640 1 -- 172.21.15.60:0/270055392 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff860003520 con 0x7ff8780cdd10 2023-11-18T20:44:46.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff8767fc640 1 -- 172.21.15.60:0/270055392 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87815d270 msgr2=0x7ff87806ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:46.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff8767fc640 1 --2- 172.21.15.60:0/270055392 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87815d270 0x7ff87806ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff8767fc640 1 -- 172.21.15.60:0/270055392 >> v1:172.21.15.100:6789/0 conn(0x7ff8780cdd10 legacy=0x7ff8780cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.597+0000 7ff8767fc640 1 -- 172.21.15.60:0/270055392 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780cf070 con 0x7ff8780709c0 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff8767fc640 1 --2- 172.21.15.60:0/270055392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8780709c0 0x7ff8780cd7d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0097a0 tx=0x7ff86c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=734deaecb3beee07 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff875ffb640 1 -- 172.21.15.60:0/270055392 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c02e070 con 0x7ff8780709c0 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff875ffb640 1 -- 172.21.15.60:0/270055392 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c0315a0 con 0x7ff8780709c0 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff875ffb640 1 -- 172.21.15.60:0/270055392 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c031910 con 0x7ff8780709c0 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff87d67c640 1 -- 172.21.15.60:0/270055392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8780709c0 msgr2=0x7ff8780cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff87d67c640 1 --2- 172.21.15.60:0/270055392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8780709c0 0x7ff8780cd7d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0097a0 tx=0x7ff86c02b840 comp rx=0 tx=0).stop 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff87d67c640 1 -- 172.21.15.60:0/270055392 shutdown_connections 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff87d67c640 1 --2- 172.21.15.60:0/270055392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8780709c0 0x7ff8780cd7d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff87d67c640 1 --2- 172.21.15.60:0/270055392 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87815d270 0x7ff87806ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff87d67c640 1 -- 172.21.15.60:0/270055392 >> 172.21.15.60:0/270055392 conn(0x7ff8780b2c10 msgr2=0x7ff8780b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:46.606 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.598+0000 7ff87d67c640 1 -- 172.21.15.60:0/270055392 shutdown_connections 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.599+0000 7ff87d67c640 1 -- 172.21.15.60:0/270055392 wait complete. 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.600+0000 7ff87d67c640 1 Processor -- start 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.600+0000 7ff87d67c640 1 -- start start 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.600+0000 7ff87d67c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8780709c0 0x7ff8780ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.600+0000 7ff87d67c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8780cdd10 0x7ff8780c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff87d67c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87815d270 0x7ff8780c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff87d67c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780b5610 con 0x7ff8780709c0 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff87d67c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780c9730 con 0x7ff8780cdd10 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff87d67c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780c9a10 con 0x7ff87815d270 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff876ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8780709c0 0x7ff8780ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff876ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8780709c0 0x7ff8780ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48846/0 (socket says 172.21.15.60:48846) 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff876ffd640 1 -- 172.21.15.60:0/1416902862 learned_addr learned my addr 172.21.15.60:0/1416902862 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff8767fc640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8780cdd10 0x7ff8780c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff8777fe640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87815d270 0x7ff8780c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff876ffd640 1 -- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87815d270 msgr2=0x7ff8780c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff876ffd640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87815d270 0x7ff8780c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff876ffd640 1 -- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8780cdd10 msgr2=0x7ff8780c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff876ffd640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8780cdd10 0x7ff8780c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff876ffd640 1 -- 172.21.15.60:0/1416902862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c009400 con 0x7ff8780709c0 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff876ffd640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8780709c0 0x7ff8780ca7d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff87000e9b0 tx=0x7ff87000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff85f7fe640 1 -- 172.21.15.60:0/1416902862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870009670 con 0x7ff8780709c0 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff85f7fe640 1 -- 172.21.15.60:0/1416902862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87000c9a0 con 0x7ff8780709c0 2023-11-18T20:44:46.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff87d67c640 1 -- 172.21.15.60:0/1416902862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8780c9cf0 con 0x7ff8780709c0 2023-11-18T20:44:46.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff85f7fe640 1 -- 172.21.15.60:0/1416902862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87000cd50 con 0x7ff8780709c0 2023-11-18T20:44:46.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.601+0000 7ff87d67c640 1 -- 172.21.15.60:0/1416902862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8780ca2b0 con 0x7ff8780709c0 2023-11-18T20:44:46.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.602+0000 7ff87d67c640 1 -- 172.21.15.60:0/1416902862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff878067fc0 con 0x7ff8780709c0 2023-11-18T20:44:46.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.603+0000 7ff85f7fe640 1 -- 172.21.15.60:0/1416902862 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff870013470 con 0x7ff8780709c0 2023-11-18T20:44:46.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.603+0000 7ff85f7fe640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86406ae40 0x7ff86406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.604+0000 7ff85f7fe640 1 -- 172.21.15.60:0/1416902862 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8700898b0 con 0x7ff8780709c0 2023-11-18T20:44:46.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.604+0000 7ff8767fc640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86406ae40 0x7ff86406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.604+0000 7ff8767fc640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86406ae40 0x7ff86406d300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff86c036000 tx=0x7ff86c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:46.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.608+0000 7ff85f7fe640 1 -- 172.21.15.60:0/1416902862 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff870054970 con 0x7ff8780709c0 2023-11-18T20:44:46.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.782+0000 7ff87d67c640 1 -- 172.21.15.60:0/1416902862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff878067420 con 0x7ff8780709c0 2023-11-18T20:44:46.784 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.782+0000 7ff85f7fe640 1 -- 172.21.15.60:0/1416902862 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff870017020 con 0x7ff8780709c0 2023-11-18T20:44:46.785 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:46.786 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 -- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86406ae40 msgr2=0x7ff86406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86406ae40 0x7ff86406d300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff86c036000 tx=0x7ff86c03d040 comp rx=0 tx=0).stop 2023-11-18T20:44:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 -- 172.21.15.60:0/1416902862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8780709c0 msgr2=0x7ff8780ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8780709c0 0x7ff8780ca7d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff87000e9b0 tx=0x7ff87000ee70 comp rx=0 tx=0).stop 2023-11-18T20:44:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 -- 172.21.15.60:0/1416902862 shutdown_connections 2023-11-18T20:44:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86406ae40 0x7ff86406d300 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87815d270 0x7ff8780c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8780cdd10 0x7ff8780c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 --2- 172.21.15.60:0/1416902862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8780709c0 0x7ff8780ca7d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 -- 172.21.15.60:0/1416902862 >> 172.21.15.60:0/1416902862 conn(0x7ff8780b2c10 msgr2=0x7ff8780b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:46.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 -- 172.21.15.60:0/1416902862 shutdown_connections 2023-11-18T20:44:46.788 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.784+0000 7ff87d67c640 1 -- 172.21.15.60:0/1416902862 wait complete. 2023-11-18T20:44:46.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:44:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.904+0000 7f376b3ce640 1 Processor -- start 2023-11-18T20:44:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.904+0000 7f376b3ce640 1 -- start start 2023-11-18T20:44:46.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376b3ce640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3764144980 0x7f376415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376b3ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37640ceb70 0x7f37640cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376b3ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f37640b4370 0x7f37640b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376b3ce640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37640b4d20 con 0x7f3764144980 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376b3ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3764071770 con 0x7f37640b4370 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376b3ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37640ccdc0 con 0x7f37640ceb70 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376916a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3764144980 0x7f376415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376916a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3764144980 0x7f376415c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48850/0 (socket says 172.21.15.60:48850) 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376916a640 1 -- 172.21.15.60:0/1613088839 learned_addr learned my addr 172.21.15.60:0/1613088839 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376996b640 1 --2- 172.21.15.60:0/1613088839 >> v2:172.21.15.100:3300/0 conn(0x7f37640b4370 0x7f37640b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f376916a640 1 -- 172.21.15.60:0/1613088839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37640ceb70 msgr2=0x7f37640cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.905+0000 7f3768969640 1 --2- 172.21.15.60:0/1613088839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37640ceb70 0x7f37640cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.906+0000 7f376916a640 1 --2- 172.21.15.60:0/1613088839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37640ceb70 0x7f37640cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.906+0000 7f376916a640 1 -- 172.21.15.60:0/1613088839 >> v2:172.21.15.100:3300/0 conn(0x7f37640b4370 msgr2=0x7f37640b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:46.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.906+0000 7f376916a640 1 --2- 172.21.15.60:0/1613088839 >> v2:172.21.15.100:3300/0 conn(0x7f37640b4370 0x7f37640b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.906+0000 7f376916a640 1 -- 172.21.15.60:0/1613088839 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37640cd0a0 con 0x7f3764144980 2023-11-18T20:44:46.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.906+0000 7f376996b640 1 --2- 172.21.15.60:0/1613088839 >> v2:172.21.15.100:3300/0 conn(0x7f37640b4370 0x7f37640b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:46.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.906+0000 7f376916a640 1 --2- 172.21.15.60:0/1613088839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764144980 0x7f376415c7c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f37640b2110 tx=0x7f375c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41cbc59c4b345f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:46.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.906+0000 7f3763fff640 1 -- 172.21.15.60:0/1613088839 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c031870 con 0x7f3764144980 2023-11-18T20:44:46.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f3763fff640 1 -- 172.21.15.60:0/1613088839 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375c0319d0 con 0x7f3764144980 2023-11-18T20:44:46.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f3763fff640 1 -- 172.21.15.60:0/1613088839 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c031d60 con 0x7f3764144980 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f376b3ce640 1 -- 172.21.15.60:0/1613088839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764144980 msgr2=0x7f376415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f376b3ce640 1 --2- 172.21.15.60:0/1613088839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764144980 0x7f376415c7c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f37640b2110 tx=0x7f375c02b910 comp rx=0 tx=0).stop 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f376b3ce640 1 -- 172.21.15.60:0/1613088839 shutdown_connections 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f376b3ce640 1 --2- 172.21.15.60:0/1613088839 >> v2:172.21.15.100:3300/0 conn(0x7f37640b4370 0x7f37640b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f376b3ce640 1 --2- 172.21.15.60:0/1613088839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37640ceb70 0x7f37640cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f376b3ce640 1 --2- 172.21.15.60:0/1613088839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764144980 0x7f376415c7c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f376b3ce640 1 -- 172.21.15.60:0/1613088839 >> 172.21.15.60:0/1613088839 conn(0x7f376406a4d0 msgr2=0x7f376406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f376b3ce640 1 -- 172.21.15.60:0/1613088839 shutdown_connections 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.907+0000 7f376b3ce640 1 -- 172.21.15.60:0/1613088839 wait complete. 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.908+0000 7f376b3ce640 1 Processor -- start 2023-11-18T20:44:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.908+0000 7f376b3ce640 1 -- start start 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.909+0000 7f376b3ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37640b4370 0x7f37640ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.909+0000 7f376b3ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37640ceb70 0x7f376413ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.909+0000 7f376b3ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3764144980 0x7f37640a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.909+0000 7f376b3ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37640acdd0 con 0x7f37640b4370 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.909+0000 7f376b3ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37640a98c0 con 0x7f3764144980 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.909+0000 7f376b3ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3764159250 con 0x7f37640ceb70 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.909+0000 7f376916a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37640b4370 0x7f37640ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.909+0000 7f376916a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37640b4370 0x7f37640ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48866/0 (socket says 172.21.15.60:48866) 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.909+0000 7f376916a640 1 -- 172.21.15.60:0/553685689 learned_addr learned my addr 172.21.15.60:0/553685689 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f376996b640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3764144980 0x7f37640a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f376916a640 1 -- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37640ceb70 msgr2=0x7f376413ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f376916a640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37640ceb70 0x7f376413ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f376916a640 1 -- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3764144980 msgr2=0x7f37640a8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:46.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f376916a640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3764144980 0x7f37640a8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:46.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f376916a640 1 -- 172.21.15.60:0/553685689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c0094d0 con 0x7f37640b4370 2023-11-18T20:44:46.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f376996b640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3764144980 0x7f37640a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:46.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f376916a640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37640b4370 0x7f37640ac890 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f375c02b950 tx=0x7f375c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:46.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f3761ffb640 1 -- 172.21.15.60:0/553685689 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c031860 con 0x7f37640b4370 2023-11-18T20:44:46.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f3761ffb640 1 -- 172.21.15.60:0/553685689 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375c0443f0 con 0x7f37640b4370 2023-11-18T20:44:46.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f376b3ce640 1 -- 172.21.15.60:0/553685689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3764159530 con 0x7f37640b4370 2023-11-18T20:44:46.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f3761ffb640 1 -- 172.21.15.60:0/553685689 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c0447a0 con 0x7f37640b4370 2023-11-18T20:44:46.915 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.910+0000 7f376b3ce640 1 -- 172.21.15.60:0/553685689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376413cfd0 con 0x7f37640b4370 2023-11-18T20:44:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.911+0000 7f376b3ce640 1 -- 172.21.15.60:0/553685689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37640cb8b0 con 0x7f37640b4370 2023-11-18T20:44:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.912+0000 7f3761ffb640 1 -- 172.21.15.60:0/553685689 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f375c04d450 con 0x7f37640b4370 2023-11-18T20:44:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.912+0000 7f3761ffb640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f375006ae70 0x7f375006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.912+0000 7f3761ffb640 1 -- 172.21.15.60:0/553685689 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f375c02d040 con 0x7f37640b4370 2023-11-18T20:44:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.912+0000 7f3768969640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f375006ae70 0x7f375006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.913+0000 7f3768969640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f375006ae70 0x7f375006d330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3758006b10 tx=0x7f3758008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:46.917+0000 7f3761ffb640 1 -- 172.21.15.60:0/553685689 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f375c076dd0 con 0x7f37640b4370 2023-11-18T20:44:47.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.095+0000 7f376b3ce640 1 -- 172.21.15.60:0/553685689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37641590b0 con 0x7f37640b4370 2023-11-18T20:44:47.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.095+0000 7f3761ffb640 1 -- 172.21.15.60:0/553685689 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37641590b0 con 0x7f37640b4370 2023-11-18T20:44:47.097 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:44:47.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 -- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f375006ae70 msgr2=0x7f375006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f375006ae70 0x7f375006d330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3758006b10 tx=0x7f3758008040 comp rx=0 tx=0).stop 2023-11-18T20:44:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 -- 172.21.15.60:0/553685689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37640b4370 msgr2=0x7f37640ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37640b4370 0x7f37640ac890 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f375c02b950 tx=0x7f375c0046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 -- 172.21.15.60:0/553685689 shutdown_connections 2023-11-18T20:44:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f375006ae70 0x7f375006d330 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3764144980 0x7f37640a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37640ceb70 0x7f376413ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 --2- 172.21.15.60:0/553685689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37640b4370 0x7f37640ac890 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 -- 172.21.15.60:0/553685689 >> 172.21.15.60:0/553685689 conn(0x7f376406a4d0 msgr2=0x7f376406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:47.100 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 -- 172.21.15.60:0/553685689 shutdown_connections 2023-11-18T20:44:47.101 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.097+0000 7f376b3ce640 1 -- 172.21.15.60:0/553685689 wait complete. 2023-11-18T20:44:47.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:44:47.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.215+0000 7f5040ae5640 1 Processor -- start 2023-11-18T20:44:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.215+0000 7f5040ae5640 1 -- start start 2023-11-18T20:44:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.216+0000 7f5040ae5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f503c15d330 0x7f503c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.216+0000 7f5040ae5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c0b2b10 con 0x7f503c0d3900 2023-11-18T20:44:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.216+0000 7f5040ae5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c0d1590 con 0x7f503c0d2910 2023-11-18T20:44:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.216+0000 7f5040ae5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503c0d1870 con 0x7f503c15d330 2023-11-18T20:44:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.217+0000 7f503a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f503c15d330 0x7f503c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.217+0000 7f503a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f503c15d330 0x7f503c0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48880/0 (socket says 172.21.15.60:48880) 2023-11-18T20:44:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.217+0000 7f503ad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f503c0d3900 0x7f503c0b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43598/0 (socket says 172.21.15.60:43598) 2023-11-18T20:44:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.217+0000 7f503a59c640 1 -- 172.21.15.60:0/2742579993 learned_addr learned my addr 172.21.15.60:0/2742579993 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:47.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.217+0000 7f503a59c640 1 -- 172.21.15.60:0/2742579993 >> v1:172.21.15.100:6789/0 conn(0x7f503c0d2910 legacy=0x7f503c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:47.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.217+0000 7f503a59c640 1 -- 172.21.15.60:0/2742579993 >> v1:172.21.15.60:6789/0 conn(0x7f503c0d3900 legacy=0x7f503c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.217+0000 7f503a59c640 1 -- 172.21.15.60:0/2742579993 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c0d1b50 con 0x7f503c15d330 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f503a59c640 1 --2- 172.21.15.60:0/2742579993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 0x7f503c0b8b80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5034009870 tx=0x7f503402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b74ff2f6e68c8b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f503959a640 1 -- 172.21.15.60:0/2742579993 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034031870 con 0x7f503c15d330 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f503959a640 1 -- 172.21.15.60:0/2742579993 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50340319d0 con 0x7f503c15d330 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f503959a640 1 -- 172.21.15.60:0/2742579993 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034031d60 con 0x7f503c15d330 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f5040ae5640 1 -- 172.21.15.60:0/2742579993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 msgr2=0x7f503c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f5040ae5640 1 --2- 172.21.15.60:0/2742579993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 0x7f503c0b8b80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5034009870 tx=0x7f503402b910 comp rx=0 tx=0).stop 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f5040ae5640 1 -- 172.21.15.60:0/2742579993 shutdown_connections 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f5040ae5640 1 --2- 172.21.15.60:0/2742579993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 0x7f503c0b8b80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f5040ae5640 1 -- 172.21.15.60:0/2742579993 >> 172.21.15.60:0/2742579993 conn(0x7f503c072020 msgr2=0x7f503c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f5040ae5640 1 -- 172.21.15.60:0/2742579993 shutdown_connections 2023-11-18T20:44:47.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.218+0000 7f5040ae5640 1 -- 172.21.15.60:0/2742579993 wait complete. 2023-11-18T20:44:47.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.220+0000 7f5040ae5640 1 Processor -- start 2023-11-18T20:44:47.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.220+0000 7f5040ae5640 1 -- start start 2023-11-18T20:44:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.220+0000 7f5040ae5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f503c0d2910 0x7f503c14c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.220+0000 7f5040ae5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f503c0d3900 0x7f503c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5040ae5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 0x7f503c0ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5040ae5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c14cb20 con 0x7f503c15d330 2023-11-18T20:44:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5040ae5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c14ce00 con 0x7f503c0d2910 2023-11-18T20:44:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5040ae5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503c14d0e0 con 0x7f503c0d3900 2023-11-18T20:44:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f503ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 0x7f503c0ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f503ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 0x7f503c0ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48884/0 (socket says 172.21.15.60:48884) 2023-11-18T20:44:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f503ad9d640 1 -- 172.21.15.60:0/1504019708 learned_addr learned my addr 172.21.15.60:0/1504019708 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f503a59c640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f503c0d2910 0x7f503c14c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5039d9b640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f503c0d3900 0x7f503c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f503ad9d640 1 -- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f503c0d3900 msgr2=0x7f503c0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f503ad9d640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f503c0d3900 0x7f503c0cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f503ad9d640 1 -- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f503c0d2910 msgr2=0x7f503c14c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f503ad9d640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f503c0d2910 0x7f503c14c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f503ad9d640 1 -- 172.21.15.60:0/1504019708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50340094d0 con 0x7f503c15d330 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5039d9b640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f503c0d3900 0x7f503c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f503ad9d640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 0x7f503c0ca690 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c7a0 tx=0x7f502c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5032ffd640 1 -- 172.21.15.60:0/1504019708 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c00a640 con 0x7f503c15d330 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5032ffd640 1 -- 172.21.15.60:0/1504019708 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c00a7a0 con 0x7f503c15d330 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5040ae5640 1 -- 172.21.15.60:0/1504019708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503c14d3c0 con 0x7f503c15d330 2023-11-18T20:44:47.225 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5032ffd640 1 -- 172.21.15.60:0/1504019708 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c00ab10 con 0x7f503c15d330 2023-11-18T20:44:47.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.221+0000 7f5040ae5640 1 -- 172.21.15.60:0/1504019708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c14d980 con 0x7f503c15d330 2023-11-18T20:44:47.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.222+0000 7f5040ae5640 1 -- 172.21.15.60:0/1504019708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503c0b78f0 con 0x7f503c15d330 2023-11-18T20:44:47.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.223+0000 7f5032ffd640 1 -- 172.21.15.60:0/1504019708 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f502c00ac70 con 0x7f503c15d330 2023-11-18T20:44:47.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.223+0000 7f5032ffd640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f501c06ae40 0x7f501c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.223+0000 7f5032ffd640 1 -- 172.21.15.60:0/1504019708 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f502c088cf0 con 0x7f503c15d330 2023-11-18T20:44:47.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.223+0000 7f503a59c640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f501c06ae40 0x7f501c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.224+0000 7f503a59c640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f501c06ae40 0x7f501c06d300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5034036000 tx=0x7f503403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:47.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.228+0000 7f5032ffd640 1 -- 172.21.15.60:0/1504019708 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f502c08d070 con 0x7f503c15d330 2023-11-18T20:44:47.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.416+0000 7f5040ae5640 1 -- 172.21.15.60:0/1504019708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f503c0b9170 con 0x7f503c15d330 2023-11-18T20:44:47.419 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.417+0000 7f5032ffd640 1 -- 172.21.15.60:0/1504019708 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7f502c017080 con 0x7f503c15d330 2023-11-18T20:44:47.419 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:44:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.418+0000 7f5040ae5640 1 -- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f501c06ae40 msgr2=0x7f501c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.418+0000 7f5040ae5640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f501c06ae40 0x7f501c06d300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5034036000 tx=0x7f503403c040 comp rx=0 tx=0).stop 2023-11-18T20:44:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.418+0000 7f5040ae5640 1 -- 172.21.15.60:0/1504019708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 msgr2=0x7f503c0ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.418+0000 7f5040ae5640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 0x7f503c0ca690 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c7a0 tx=0x7f502c00cc60 comp rx=0 tx=0).stop 2023-11-18T20:44:47.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.418+0000 7f5040ae5640 1 -- 172.21.15.60:0/1504019708 shutdown_connections 2023-11-18T20:44:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.418+0000 7f5040ae5640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f501c06ae40 0x7f501c06d300 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.418+0000 7f5040ae5640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f503c15d330 0x7f503c0ca690 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.418+0000 7f5040ae5640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f503c0d3900 0x7f503c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.418+0000 7f5040ae5640 1 --2- 172.21.15.60:0/1504019708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f503c0d2910 0x7f503c14c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.418+0000 7f5040ae5640 1 -- 172.21.15.60:0/1504019708 >> 172.21.15.60:0/1504019708 conn(0x7f503c072020 msgr2=0x7f503c072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.419+0000 7f5040ae5640 1 -- 172.21.15.60:0/1504019708 shutdown_connections 2023-11-18T20:44:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.419+0000 7f5040ae5640 1 -- 172.21.15.60:0/1504019708 wait complete. 2023-11-18T20:44:47.431 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:47.432 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:47.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:47.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.592+0000 7f4fd6716640 1 Processor -- start 2023-11-18T20:44:47.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.592+0000 7f4fd6716640 1 -- start start 2023-11-18T20:44:47.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.593+0000 7f4fd6716640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0140310 0x7f4fd01618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.593+0000 7f4fd6716640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd01406f0 con 0x7f4fd006abe0 2023-11-18T20:44:47.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.593+0000 7f4fd6716640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd0143960 con 0x7f4fd01409d0 2023-11-18T20:44:47.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.593+0000 7f4fd6716640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0161e20 con 0x7f4fd0140310 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.593+0000 7f4fd4cb3640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4fd006abe0 0x7f4fd006efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43614/0 (socket says 172.21.15.60:43614) 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.593+0000 7f4fd4cb3640 1 -- 172.21.15.60:0/3807085263 learned_addr learned my addr 172.21.15.60:0/3807085263 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.593+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203636105 0 0) 0x7f4fd01406f0 con 0x7f4fd006abe0 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.593+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fbc003520 con 0x7f4fd006abe0 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.593+0000 7f4fcffff640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0140310 0x7f4fd01618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3025593180 0 0) 0x7f4fbc003520 con 0x7f4fd006abe0 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0140310 msgr2=0x7f4fd01618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0140310 0x7f4fd01618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 >> v1:172.21.15.100:6789/0 conn(0x7f4fd01409d0 legacy=0x7f4fd006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd0142de0 con 0x7f4fd006abe0 2023-11-18T20:44:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f4fc80019f0 con 0x7f4fd006abe0 2023-11-18T20:44:47.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 >> v1:172.21.15.60:6789/0 conn(0x7f4fd006abe0 legacy=0x7f4fd006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc0039a0 0x7f4fbc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fbc0047c0 0x7f4fbc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0140310 0x7f4fbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0161e20 con 0x7f4fbc0047c0 2023-11-18T20:44:47.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc00bab0 con 0x7f4fbc0039a0 2023-11-18T20:44:47.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc00bc70 con 0x7f4fd0140310 2023-11-18T20:44:47.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fcf7fe640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fbc0047c0 0x7f4fbc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fcffff640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc0039a0 0x7f4fbc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fd4cb3640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0140310 0x7f4fbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fcffff640 1 -- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0140310 msgr2=0x7f4fbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:47.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fcffff640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0140310 0x7f4fbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fcffff640 1 -- 172.21.15.60:0/3807085263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fbc0047c0 msgr2=0x7f4fbc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:47.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fcffff640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fbc0047c0 0x7f4fbc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fcffff640 1 -- 172.21.15.60:0/3807085263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc003520 con 0x7f4fbc0039a0 2023-11-18T20:44:47.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.594+0000 7f4fcf7fe640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fbc0047c0 0x7f4fbc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:47.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.595+0000 7f4fcffff640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc0039a0 0x7f4fbc003dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009ac0 tx=0x7f4fbc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f50d2a48928cb7a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:47.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.595+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb802c6b0 con 0x7f4fbc0039a0 2023-11-18T20:44:47.601 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.595+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb802c810 con 0x7f4fbc0039a0 2023-11-18T20:44:47.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.595+0000 7f4fceffd640 1 -- 172.21.15.60:0/3807085263 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb802cba0 con 0x7f4fbc0039a0 2023-11-18T20:44:47.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.596+0000 7f4fd6716640 1 -- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc0039a0 msgr2=0x7f4fbc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.596+0000 7f4fd6716640 1 --2- 172.21.15.60:0/3807085263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc0039a0 0x7f4fbc003dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009ac0 tx=0x7f4fbc0057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.602 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.596+0000 7f4fd4cb3640 1 -- 172.21.15.60:0/3807085263 reap_dead start 2023-11-18T20:44:47.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.596+0000 7f4fd6716640 1 -- 172.21.15.60:0/3807085263 shutdown_connections 2023-11-18T20:44:47.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.596+0000 7f4fd6716640 1 -- 172.21.15.60:0/3807085263 >> 172.21.15.60:0/3807085263 conn(0x7f4fd00b1e50 msgr2=0x7f4fd00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:47.603 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.596+0000 7f4fd6716640 1 -- 172.21.15.60:0/3807085263 shutdown_connections 2023-11-18T20:44:47.604 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.596+0000 7f4fd6716640 1 -- 172.21.15.60:0/3807085263 wait complete. 2023-11-18T20:44:47.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.597+0000 7f4fd6716640 1 Processor -- start 2023-11-18T20:44:47.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.597+0000 7f4fd6716640 1 -- start start 2023-11-18T20:44:47.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd6716640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc0039a0 0x7f4fd00aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.608 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd6716640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0154950 0x7f4fd01459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd6716640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd01f6a10 0x7f4fd01f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd6716640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd01f73d0 con 0x7f4fd01f6a10 2023-11-18T20:44:47.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd6716640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0146850 con 0x7f4fbc0039a0 2023-11-18T20:44:47.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd6716640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd0145530 con 0x7f4fd0154950 2023-11-18T20:44:47.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd4cb3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd01f6a10 0x7f4fd01f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd4cb3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd01f6a10 0x7f4fd01f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48912/0 (socket says 172.21.15.60:48912) 2023-11-18T20:44:47.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd4cb3640 1 -- 172.21.15.60:0/3234672687 learned_addr learned my addr 172.21.15.60:0/3234672687 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:47.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fcffff640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc0039a0 0x7f4fd00aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fcf7fe640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0154950 0x7f4fd01459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd4cb3640 1 -- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0154950 msgr2=0x7f4fd01459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd4cb3640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0154950 0x7f4fd01459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.611 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd4cb3640 1 -- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc0039a0 msgr2=0x7f4fd00aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd4cb3640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc0039a0 0x7f4fd00aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.598+0000 7f4fd4cb3640 1 -- 172.21.15.60:0/3234672687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd01444a0 con 0x7f4fd01f6a10 2023-11-18T20:44:47.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.599+0000 7f4fd4cb3640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd01f6a10 0x7f4fd01f6e90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80012e0 tx=0x7f4fc800a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:47.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.599+0000 7f4fccff9640 1 -- 172.21.15.60:0/3234672687 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800ace0 con 0x7f4fd01f6a10 2023-11-18T20:44:47.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.599+0000 7f4fccff9640 1 -- 172.21.15.60:0/3234672687 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc800ae40 con 0x7f4fd01f6a10 2023-11-18T20:44:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.599+0000 7f4fd6716640 1 -- 172.21.15.60:0/3234672687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd0146aa0 con 0x7f4fd01f6a10 2023-11-18T20:44:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.599+0000 7f4fccff9640 1 -- 172.21.15.60:0/3234672687 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc80135f0 con 0x7f4fd01f6a10 2023-11-18T20:44:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.599+0000 7f4fd6716640 1 -- 172.21.15.60:0/3234672687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd013fab0 con 0x7f4fd01f6a10 2023-11-18T20:44:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.600+0000 7f4fd6716640 1 -- 172.21.15.60:0/3234672687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd01456a0 con 0x7f4fd01f6a10 2023-11-18T20:44:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.600+0000 7f4fccff9640 1 -- 172.21.15.60:0/3234672687 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4fc8013750 con 0x7f4fd01f6a10 2023-11-18T20:44:47.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.601+0000 7f4fccff9640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4fb006ae70 0x7f4fb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.601+0000 7f4fccff9640 1 -- 172.21.15.60:0/3234672687 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fc8089c70 con 0x7f4fd01f6a10 2023-11-18T20:44:47.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.601+0000 7f4fcffff640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4fb006ae70 0x7f4fb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.601+0000 7f4fcffff640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4fb006ae70 0x7f4fb006d330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8036000 tx=0x7f4fb8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:47.614 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.605+0000 7f4fccff9640 1 -- 172.21.15.60:0/3234672687 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4fc8054ca0 con 0x7f4fd01f6a10 2023-11-18T20:44:47.785 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.783+0000 7f4fd6716640 1 -- 172.21.15.60:0/3234672687 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fd0069400 con 0x7f4fd01f6a10 2023-11-18T20:44:47.787 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.784+0000 7f4fccff9640 1 -- 172.21.15.60:0/3234672687 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f4fc80582c0 con 0x7f4fd01f6a10 2023-11-18T20:44:47.787 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:47.787 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:34.103787+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:47.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 -- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4fb006ae70 msgr2=0x7f4fb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4fb006ae70 0x7f4fb006d330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8036000 tx=0x7f4fb8030000 comp rx=0 tx=0).stop 2023-11-18T20:44:47.789 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 -- 172.21.15.60:0/3234672687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd01f6a10 msgr2=0x7f4fd01f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd01f6a10 0x7f4fd01f6e90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80012e0 tx=0x7f4fc800a6e0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 -- 172.21.15.60:0/3234672687 shutdown_connections 2023-11-18T20:44:47.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f4fb006ae70 0x7f4fb006d330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.790 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd01f6a10 0x7f4fd01f6e90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd0154950 0x7f4fd01459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 --2- 172.21.15.60:0/3234672687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fbc0039a0 0x7f4fd00aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 -- 172.21.15.60:0/3234672687 >> 172.21.15.60:0/3234672687 conn(0x7f4fd00b1e50 msgr2=0x7f4fd015fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:47.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 -- 172.21.15.60:0/3234672687 shutdown_connections 2023-11-18T20:44:47.791 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.787+0000 7f4fd6716640 1 -- 172.21.15.60:0/3234672687 wait complete. 2023-11-18T20:44:47.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:44:31.358325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:44:32.036538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:44:47.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:44:47.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.907+0000 7fc0fd3dc640 1 Processor -- start 2023-11-18T20:44:47.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.907+0000 7fc0fd3dc640 1 -- start start 2023-11-18T20:44:47.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.908+0000 7fc0fd3dc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc0f815d6c0 0x7fc0f806b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.908+0000 7fc0fd3dc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f80d1d50 0x7fc0f806ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.908+0000 7fc0fd3dc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f806ce00 con 0x7fc0f815d2e0 2023-11-18T20:44:47.911 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.908+0000 7fc0fd3dc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f806bf40 con 0x7fc0f815d6c0 2023-11-18T20:44:47.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.908+0000 7fc0fd3dc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f806c1f0 con 0x7fc0f80d1d50 2023-11-18T20:44:47.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.909+0000 7fc0f77fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f80d1d50 0x7fc0f806ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.909+0000 7fc0f67fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc0f815d6c0 0x7fc0f806b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.909+0000 7fc0f6ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc0f815d2e0 0x7fc0f80d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:33592/0 (socket says 172.21.15.60:33592) 2023-11-18T20:44:47.912 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.909+0000 7fc0f6ffd640 1 -- 172.21.15.60:0/2279552269 learned_addr learned my addr 172.21.15.60:0/2279552269 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.909+0000 7fc0f67fc640 1 -- 172.21.15.60:0/2279552269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f80d1d50 msgr2=0x7fc0f806ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.909+0000 7fc0f67fc640 1 --2- 172.21.15.60:0/2279552269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f80d1d50 0x7fc0f806ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.909+0000 7fc0f67fc640 1 -- 172.21.15.60:0/2279552269 >> v1:172.21.15.100:6789/0 conn(0x7fc0f815d2e0 legacy=0x7fc0f80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.909+0000 7fc0f67fc640 1 -- 172.21.15.60:0/2279552269 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f806c4d0 con 0x7fc0f815d6c0 2023-11-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.909+0000 7fc0f77fe640 1 --2- 172.21.15.60:0/2279552269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f80d1d50 0x7fc0f806ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.910+0000 7fc0f67fc640 1 --2- 172.21.15.60:0/2279552269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f815d6c0 0x7fc0f806b4c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00097a0 tx=0x7fc0e002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae0d872cbc179816 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:47.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.910+0000 7fc0f5ffb640 1 -- 172.21.15.60:0/2279552269 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e002e070 con 0x7fc0f815d6c0 2023-11-18T20:44:47.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.910+0000 7fc0f5ffb640 1 -- 172.21.15.60:0/2279552269 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e00315a0 con 0x7fc0f815d6c0 2023-11-18T20:44:47.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.910+0000 7fc0f5ffb640 1 -- 172.21.15.60:0/2279552269 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0031930 con 0x7fc0f815d6c0 2023-11-18T20:44:47.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.911+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2279552269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f815d6c0 msgr2=0x7fc0f806b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.911+0000 7fc0fd3dc640 1 --2- 172.21.15.60:0/2279552269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f815d6c0 0x7fc0f806b4c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00097a0 tx=0x7fc0e002b840 comp rx=0 tx=0).stop 2023-11-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.911+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2279552269 shutdown_connections 2023-11-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.911+0000 7fc0fd3dc640 1 --2- 172.21.15.60:0/2279552269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f815d6c0 0x7fc0f806b4c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.911+0000 7fc0fd3dc640 1 --2- 172.21.15.60:0/2279552269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f80d1d50 0x7fc0f806ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.911+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2279552269 >> 172.21.15.60:0/2279552269 conn(0x7fc0f809c580 msgr2=0x7fc0f80d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.911+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2279552269 shutdown_connections 2023-11-18T20:44:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.911+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2279552269 wait complete. 2023-11-18T20:44:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.912+0000 7fc0fd3dc640 1 Processor -- start 2023-11-18T20:44:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.912+0000 7fc0fd3dc640 1 -- start start 2023-11-18T20:44:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0fd3dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f80d1d50 0x7fc0f80b8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0fd3dc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f815d2e0 0x7fc0f80c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0fd3dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f815d6c0 0x7fc0f80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0fd3dc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f8160970 con 0x7fc0f80d1d50 2023-11-18T20:44:47.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0fd3dc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f815ec20 con 0x7fc0f815d6c0 2023-11-18T20:44:47.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0fd3dc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f8151720 con 0x7fc0f815d2e0 2023-11-18T20:44:47.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f80d1d50 0x7fc0f80b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f80d1d50 0x7fc0f80b8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48920/0 (socket says 172.21.15.60:48920) 2023-11-18T20:44:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f6ffd640 1 -- 172.21.15.60:0/2361565321 learned_addr learned my addr 172.21.15.60:0/2361565321 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f77fe640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f815d6c0 0x7fc0f80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f67fc640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f815d2e0 0x7fc0f80c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f6ffd640 1 -- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f815d2e0 msgr2=0x7fc0f80c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f6ffd640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f815d2e0 0x7fc0f80c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f6ffd640 1 -- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f815d6c0 msgr2=0x7fc0f80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f6ffd640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f815d6c0 0x7fc0f80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f6ffd640 1 -- 172.21.15.60:0/2361565321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f0009400 con 0x7fc0f80d1d50 2023-11-18T20:44:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f67fc640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f815d2e0 0x7fc0f80c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.913+0000 7fc0f6ffd640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f80d1d50 0x7fc0f80b8320 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800cbd0 tx=0x7fc0e800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.914+0000 7fc0ef7fe640 1 -- 172.21.15.60:0/2361565321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e800ac70 con 0x7fc0f80d1d50 2023-11-18T20:44:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.914+0000 7fc0ef7fe640 1 -- 172.21.15.60:0/2361565321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e800add0 con 0x7fc0f80d1d50 2023-11-18T20:44:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.914+0000 7fc0ef7fe640 1 -- 172.21.15.60:0/2361565321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8012600 con 0x7fc0f80d1d50 2023-11-18T20:44:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.914+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2361565321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e0009400 con 0x7fc0f80d1d50 2023-11-18T20:44:47.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.914+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2361565321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f80b8860 con 0x7fc0f80d1d50 2023-11-18T20:44:47.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.915+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2361565321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f8067420 con 0x7fc0f80d1d50 2023-11-18T20:44:47.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.915+0000 7fc0ef7fe640 1 -- 172.21.15.60:0/2361565321 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0e8007070 con 0x7fc0f80d1d50 2023-11-18T20:44:47.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.916+0000 7fc0ef7fe640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0d806ae40 0x7fc0d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:47.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.916+0000 7fc0ef7fe640 1 -- 172.21.15.60:0/2361565321 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0e8089260 con 0x7fc0f80d1d50 2023-11-18T20:44:47.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.916+0000 7fc0f67fc640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0d806ae40 0x7fc0d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.916+0000 7fc0f67fc640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0d806ae40 0x7fc0d806d300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0036000 tx=0x7fc0e003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:47.920+0000 7fc0ef7fe640 1 -- 172.21.15.60:0/2361565321 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0e80543d0 con 0x7fc0f80d1d50 2023-11-18T20:44:48.095 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.093+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2361565321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0f80b8860 con 0x7fc0f80d1d50 2023-11-18T20:44:48.096 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.093+0000 7fc0ef7fe640 1 -- 172.21.15.60:0/2361565321 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0f80b8860 con 0x7fc0f80d1d50 2023-11-18T20:44:48.096 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:44:48.097 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0d806ae40 msgr2=0x7fc0d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0d806ae40 0x7fc0d806d300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0036000 tx=0x7fc0e003d040 comp rx=0 tx=0).stop 2023-11-18T20:44:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2361565321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f80d1d50 msgr2=0x7fc0f80b8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f80d1d50 0x7fc0f80b8320 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800cbd0 tx=0x7fc0e800a060 comp rx=0 tx=0).stop 2023-11-18T20:44:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2361565321 shutdown_connections 2023-11-18T20:44:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc0d806ae40 0x7fc0d806d300 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0f815d6c0 0x7fc0f80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0f815d2e0 0x7fc0f80c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 --2- 172.21.15.60:0/2361565321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc0f80d1d50 0x7fc0f80b8320 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2361565321 >> 172.21.15.60:0/2361565321 conn(0x7fc0f809c580 msgr2=0x7fc0f8151e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:48.098 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2361565321 shutdown_connections 2023-11-18T20:44:48.099 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.095+0000 7fc0fd3dc640 1 -- 172.21.15.60:0/2361565321 wait complete. 2023-11-18T20:44:48.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:44:48.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.216+0000 7f5a93215640 1 Processor -- start 2023-11-18T20:44:48.218 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.216+0000 7f5a93215640 1 -- start start 2023-11-18T20:44:48.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.216+0000 7f5a93215640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5a8c15d330 0x7f5a8c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.216+0000 7f5a93215640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8c0b2b10 con 0x7f5a8c0d3900 2023-11-18T20:44:48.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.216+0000 7f5a93215640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8c0d1590 con 0x7f5a8c0d2910 2023-11-18T20:44:48.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.216+0000 7f5a93215640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8c0d1870 con 0x7f5a8c15d330 2023-11-18T20:44:48.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.216+0000 7f5a917b2640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5a8c0d3900 0x7f5a8c0b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43616/0 (socket says 172.21.15.60:43616) 2023-11-18T20:44:48.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.217+0000 7f5a917b2640 1 -- 172.21.15.60:0/68404026 learned_addr learned my addr 172.21.15.60:0/68404026 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:48.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.217+0000 7f5a90fb1640 1 --2- 172.21.15.60:0/68404026 >> v2:172.21.15.100:3300/0 conn(0x7f5a8c15d330 0x7f5a8c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.217+0000 7f5a8b7fe640 1 -- 172.21.15.60:0/68404026 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951622745 0 0) 0x7f5a8c0b2b10 con 0x7f5a8c0d3900 2023-11-18T20:44:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.217+0000 7f5a8b7fe640 1 -- 172.21.15.60:0/68404026 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a78003520 con 0x7f5a8c0d3900 2023-11-18T20:44:48.222 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.217+0000 7f5a8b7fe640 1 -- 172.21.15.60:0/68404026 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 758321053 0 0) 0x7f5a78003520 con 0x7f5a8c0d3900 2023-11-18T20:44:48.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.217+0000 7f5a90fb1640 1 -- 172.21.15.60:0/68404026 >> v1:172.21.15.100:6789/0 conn(0x7f5a8c0d2910 legacy=0x7f5a8c0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.217+0000 7f5a90fb1640 1 -- 172.21.15.60:0/68404026 >> v1:172.21.15.60:6789/0 conn(0x7f5a8c0d3900 legacy=0x7f5a8c0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.217+0000 7f5a90fb1640 1 -- 172.21.15.60:0/68404026 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0d1b50 con 0x7f5a8c15d330 2023-11-18T20:44:48.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.218+0000 7f5a90fb1640 1 --2- 172.21.15.60:0/68404026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c15d330 0x7f5a8c0b8b80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a740097a0 tx=0x7f5a7402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3407049ea196a83 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:48.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.218+0000 7f5a8b7fe640 1 -- 172.21.15.60:0/68404026 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a840019f0 con 0x7f5a8c15d330 2023-11-18T20:44:48.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.218+0000 7f5a8b7fe640 1 -- 172.21.15.60:0/68404026 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7402e050 con 0x7f5a8c15d330 2023-11-18T20:44:48.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.218+0000 7f5a8b7fe640 1 -- 172.21.15.60:0/68404026 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74030710 con 0x7f5a8c15d330 2023-11-18T20:44:48.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.219+0000 7f5a93215640 1 -- 172.21.15.60:0/68404026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c15d330 msgr2=0x7f5a8c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.219+0000 7f5a93215640 1 --2- 172.21.15.60:0/68404026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c15d330 0x7f5a8c0b8b80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a740097a0 tx=0x7f5a7402b790 comp rx=0 tx=0).stop 2023-11-18T20:44:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.219+0000 7f5a93215640 1 -- 172.21.15.60:0/68404026 shutdown_connections 2023-11-18T20:44:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.219+0000 7f5a93215640 1 --2- 172.21.15.60:0/68404026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c15d330 0x7f5a8c0b8b80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.219+0000 7f5a93215640 1 -- 172.21.15.60:0/68404026 >> 172.21.15.60:0/68404026 conn(0x7f5a8c072020 msgr2=0x7f5a8c072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.219+0000 7f5a93215640 1 -- 172.21.15.60:0/68404026 shutdown_connections 2023-11-18T20:44:48.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.219+0000 7f5a93215640 1 -- 172.21.15.60:0/68404026 wait complete. 2023-11-18T20:44:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.220+0000 7f5a93215640 1 Processor -- start 2023-11-18T20:44:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.220+0000 7f5a93215640 1 -- start start 2023-11-18T20:44:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a93215640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a8c0d2910 0x7f5a8c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a93215640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a8c0d3900 0x7f5a8c0c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.229 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a93215640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c15d330 0x7f5a8c0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a93215640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c15f330 con 0x7f5a8c0d2910 2023-11-18T20:44:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a93215640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c15e6b0 con 0x7f5a8c15d330 2023-11-18T20:44:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a93215640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c15e990 con 0x7f5a8c0d3900 2023-11-18T20:44:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a90fb1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a8c0d2910 0x7f5a8c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a90fb1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a8c0d2910 0x7f5a8c14e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48934/0 (socket says 172.21.15.60:48934) 2023-11-18T20:44:48.230 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a90fb1640 1 -- 172.21.15.60:0/1227363146 learned_addr learned my addr 172.21.15.60:0/1227363146 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a917b2640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c15d330 0x7f5a8c0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a8bfff640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a8c0d3900 0x7f5a8c0c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a90fb1640 1 -- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a8c0d3900 msgr2=0x7f5a8c0c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a90fb1640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a8c0d3900 0x7f5a8c0c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.231 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a90fb1640 1 -- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c15d330 msgr2=0x7f5a8c0c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a90fb1640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c15d330 0x7f5a8c0c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a90fb1640 1 -- 172.21.15.60:0/1227363146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a74009400 con 0x7f5a8c0d2910 2023-11-18T20:44:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a90fb1640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a8c0d2910 0x7f5a8c14e580 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5a74000c00 tx=0x7f5a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a897fa640 1 -- 172.21.15.60:0/1227363146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7403d890 con 0x7f5a8c0d2910 2023-11-18T20:44:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a897fa640 1 -- 172.21.15.60:0/1227363146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7403da10 con 0x7f5a8c0d2910 2023-11-18T20:44:48.232 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.221+0000 7f5a93215640 1 -- 172.21.15.60:0/1227363146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c15ec70 con 0x7f5a8c0d2910 2023-11-18T20:44:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.222+0000 7f5a897fa640 1 -- 172.21.15.60:0/1227363146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7403ddc0 con 0x7f5a8c0d2910 2023-11-18T20:44:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.222+0000 7f5a93215640 1 -- 172.21.15.60:0/1227363146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c14eac0 con 0x7f5a8c0d2910 2023-11-18T20:44:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.223+0000 7f5a93215640 1 -- 172.21.15.60:0/1227363146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8c0d3170 con 0x7f5a8c0d2910 2023-11-18T20:44:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.223+0000 7f5a897fa640 1 -- 172.21.15.60:0/1227363146 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a74039040 con 0x7f5a8c0d2910 2023-11-18T20:44:48.233 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.223+0000 7f5a897fa640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a6c06ae40 0x7f5a6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.223+0000 7f5a897fa640 1 -- 172.21.15.60:0/1227363146 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a740ab600 con 0x7f5a8c0d2910 2023-11-18T20:44:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.224+0000 7f5a8bfff640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a6c06ae40 0x7f5a6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.224+0000 7f5a8bfff640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a6c06ae40 0x7f5a6c06d300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c160b30 tx=0x7f5a7c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.228+0000 7f5a897fa640 1 -- 172.21.15.60:0/1227363146 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a740767d0 con 0x7f5a8c0d2910 2023-11-18T20:44:48.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.402+0000 7f5a93215640 1 -- 172.21.15.60:0/1227363146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a8c069400 con 0x7f5a8c0d2910 2023-11-18T20:44:48.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.402+0000 7f5a897fa640 1 -- 172.21.15.60:0/1227363146 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a7404d020 con 0x7f5a8c0d2910 2023-11-18T20:44:48.405 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 -- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a6c06ae40 msgr2=0x7f5a6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a6c06ae40 0x7f5a6c06d300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c160b30 tx=0x7f5a7c009040 comp rx=0 tx=0).stop 2023-11-18T20:44:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 -- 172.21.15.60:0/1227363146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a8c0d2910 msgr2=0x7f5a8c14e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a8c0d2910 0x7f5a8c14e580 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5a74000c00 tx=0x7f5a740046c0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 -- 172.21.15.60:0/1227363146 shutdown_connections 2023-11-18T20:44:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5a6c06ae40 0x7f5a6c06d300 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8c15d330 0x7f5a8c0c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a8c0d3900 0x7f5a8c0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 --2- 172.21.15.60:0/1227363146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a8c0d2910 0x7f5a8c14e580 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 -- 172.21.15.60:0/1227363146 >> 172.21.15.60:0/1227363146 conn(0x7f5a8c072020 msgr2=0x7f5a8c0ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:48.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.404+0000 7f5a93215640 1 -- 172.21.15.60:0/1227363146 shutdown_connections 2023-11-18T20:44:48.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.405+0000 7f5a93215640 1 -- 172.21.15.60:0/1227363146 wait complete. 2023-11-18T20:44:48.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:44:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.524+0000 7f20afa32640 1 Processor -- start 2023-11-18T20:44:48.527 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.525+0000 7f20afa32640 1 -- start start 2023-11-18T20:44:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.525+0000 7f20afa32640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20a8150560 0x7f20a80b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.525+0000 7f20afa32640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f20a8150940 0x7f20a80ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.525+0000 7f20afa32640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a80b49f0 con 0x7f20a80b3390 2023-11-18T20:44:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.525+0000 7f20afa32640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a80b3de0 con 0x7f20a8150940 2023-11-18T20:44:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.525+0000 7f20afa32640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80b40c0 con 0x7f20a8150560 2023-11-18T20:44:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.525+0000 7f20adfcf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f20a8150940 0x7f20a80ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.526+0000 7f20acfcd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f20a80b3390 0x7f20a815d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43632/0 (socket says 172.21.15.60:43632) 2023-11-18T20:44:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.526+0000 7f20adfcf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f20a8150940 0x7f20a80ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48938/0 (socket says 172.21.15.60:48938) 2023-11-18T20:44:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.526+0000 7f20acfcd640 1 -- 172.21.15.60:0/4293330132 learned_addr learned my addr 172.21.15.60:0/4293330132 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.526+0000 7f20ad7ce640 1 --2- 172.21.15.60:0/4293330132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20a8150560 0x7f20a80b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.526+0000 7f20adfcf640 1 -- 172.21.15.60:0/4293330132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20a8150560 msgr2=0x7f20a80b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.526+0000 7f20adfcf640 1 --2- 172.21.15.60:0/4293330132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20a8150560 0x7f20a80b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.526+0000 7f20adfcf640 1 -- 172.21.15.60:0/4293330132 >> v1:172.21.15.60:6789/0 conn(0x7f20a80b3390 legacy=0x7f20a815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.526+0000 7f20adfcf640 1 -- 172.21.15.60:0/4293330132 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a80b43a0 con 0x7f20a8150940 2023-11-18T20:44:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f20adfcf640 1 --2- 172.21.15.60:0/4293330132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150940 0x7f20a80ce0e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009d90 tx=0x7f20a002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfeeaf709efc28c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f209ffff640 1 -- 172.21.15.60:0/4293330132 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a002cc50 con 0x7f20a8150940 2023-11-18T20:44:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f209ffff640 1 -- 172.21.15.60:0/4293330132 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a002cdb0 con 0x7f20a8150940 2023-11-18T20:44:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f209ffff640 1 -- 172.21.15.60:0/4293330132 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0034660 con 0x7f20a8150940 2023-11-18T20:44:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f20afa32640 1 -- 172.21.15.60:0/4293330132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150940 msgr2=0x7f20a80ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f20afa32640 1 --2- 172.21.15.60:0/4293330132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150940 0x7f20a80ce0e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009d90 tx=0x7f20a002bea0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f20afa32640 1 -- 172.21.15.60:0/4293330132 shutdown_connections 2023-11-18T20:44:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f20afa32640 1 --2- 172.21.15.60:0/4293330132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150940 0x7f20a80ce0e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f20afa32640 1 --2- 172.21.15.60:0/4293330132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20a8150560 0x7f20a80b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f20afa32640 1 -- 172.21.15.60:0/4293330132 >> 172.21.15.60:0/4293330132 conn(0x7f20a806a4d0 msgr2=0x7f20a806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f20afa32640 1 -- 172.21.15.60:0/4293330132 shutdown_connections 2023-11-18T20:44:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.527+0000 7f20afa32640 1 -- 172.21.15.60:0/4293330132 wait complete. 2023-11-18T20:44:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.528+0000 7f20afa32640 1 Processor -- start 2023-11-18T20:44:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.529+0000 7f20afa32640 1 -- start start 2023-11-18T20:44:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.529+0000 7f20afa32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20a80b3390 0x7f20a81449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.529+0000 7f20afa32640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150560 0x7f20a81598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.529+0000 7f20afa32640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20a81f0be0 0x7f20a80afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.529+0000 7f20afa32640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a81f1be0 con 0x7f20a8150560 2023-11-18T20:44:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.529+0000 7f20afa32640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a8144f30 con 0x7f20a80b3390 2023-11-18T20:44:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.529+0000 7f20afa32640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a81f1d90 con 0x7f20a81f0be0 2023-11-18T20:44:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.529+0000 7f20acfcd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150560 0x7f20a81598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.529+0000 7f20acfcd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150560 0x7f20a81598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48952/0 (socket says 172.21.15.60:48952) 2023-11-18T20:44:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.529+0000 7f20acfcd640 1 -- 172.21.15.60:0/1411075537 learned_addr learned my addr 172.21.15.60:0/1411075537 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20ad7ce640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20a80b3390 0x7f20a81449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20acfcd640 1 -- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20a81f0be0 msgr2=0x7f20a80afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20adfcf640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20a81f0be0 0x7f20a80afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20acfcd640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20a81f0be0 0x7f20a80afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20acfcd640 1 -- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20a80b3390 msgr2=0x7f20a81449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20acfcd640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20a80b3390 0x7f20a81449f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20acfcd640 1 -- 172.21.15.60:0/1411075537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0009a70 con 0x7f20a8150560 2023-11-18T20:44:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20ad7ce640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20a80b3390 0x7f20a81449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20acfcd640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150560 0x7f20a81598d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f209800c4d0 tx=0x7f209800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f209dffb640 1 -- 172.21.15.60:0/1411075537 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098005ce0 con 0x7f20a8150560 2023-11-18T20:44:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f209dffb640 1 -- 172.21.15.60:0/1411075537 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2098005e40 con 0x7f20a8150560 2023-11-18T20:44:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20afa32640 1 -- 172.21.15.60:0/1411075537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a814b780 con 0x7f20a8150560 2023-11-18T20:44:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f209dffb640 1 -- 172.21.15.60:0/1411075537 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098007270 con 0x7f20a8150560 2023-11-18T20:44:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.530+0000 7f20afa32640 1 -- 172.21.15.60:0/1411075537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a81f1340 con 0x7f20a8150560 2023-11-18T20:44:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.531+0000 7f20afa32640 1 -- 172.21.15.60:0/1411075537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a8069400 con 0x7f20a8150560 2023-11-18T20:44:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.532+0000 7f209dffb640 1 -- 172.21.15.60:0/1411075537 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20980073d0 con 0x7f20a8150560 2023-11-18T20:44:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.532+0000 7f209dffb640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208c06ae40 0x7f208c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.532+0000 7f209dffb640 1 -- 172.21.15.60:0/1411075537 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2098088a50 con 0x7f20a8150560 2023-11-18T20:44:48.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.532+0000 7f20ad7ce640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208c06ae40 0x7f208c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.533+0000 7f20ad7ce640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208c06ae40 0x7f208c06d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f20a4006fd0 tx=0x7f20a4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:48.539 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.536+0000 7f209dffb640 1 -- 172.21.15.60:0/1411075537 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2098053bb0 con 0x7f20a8150560 2023-11-18T20:44:48.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.716+0000 7f20afa32640 1 -- 172.21.15.60:0/1411075537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20a815a8f0 con 0x7f20a8150560 2023-11-18T20:44:48.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.716+0000 7f209dffb640 1 -- 172.21.15.60:0/1411075537 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2098017020 con 0x7f20a8150560 2023-11-18T20:44:48.719 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:44:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 -- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208c06ae40 msgr2=0x7f208c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208c06ae40 0x7f208c06d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f20a4006fd0 tx=0x7f20a4008040 comp rx=0 tx=0).stop 2023-11-18T20:44:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 -- 172.21.15.60:0/1411075537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150560 msgr2=0x7f20a81598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150560 0x7f20a81598d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f209800c4d0 tx=0x7f209800c990 comp rx=0 tx=0).stop 2023-11-18T20:44:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 -- 172.21.15.60:0/1411075537 shutdown_connections 2023-11-18T20:44:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f208c06ae40 0x7f208c06d300 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f20a4006fd0 tx=0x7f20a4008040 comp rx=0 tx=0).stop 2023-11-18T20:44:48.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20a81f0be0 0x7f20a80afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20a8150560 0x7f20a81598d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 --2- 172.21.15.60:0/1411075537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20a80b3390 0x7f20a81449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 -- 172.21.15.60:0/1411075537 >> 172.21.15.60:0/1411075537 conn(0x7f20a806a4d0 msgr2=0x7f20a80cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.718+0000 7f20afa32640 1 -- 172.21.15.60:0/1411075537 shutdown_connections 2023-11-18T20:44:48.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.719+0000 7f20afa32640 1 -- 172.21.15.60:0/1411075537 wait complete. 2023-11-18T20:44:48.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:44:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.836+0000 7f2c7b947640 1 Processor -- start 2023-11-18T20:44:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.837+0000 7f2c7b947640 1 -- start start 2023-11-18T20:44:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.837+0000 7f2c7b947640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2c74144a00 0x7f2c7406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.837+0000 7f2c7b947640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c740b2eb0 con 0x7f2c741415f0 2023-11-18T20:44:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.837+0000 7f2c7b947640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c740b3190 con 0x7f2c74144440 2023-11-18T20:44:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.837+0000 7f2c7b947640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c74144820 con 0x7f2c74144a00 2023-11-18T20:44:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.838+0000 7f2c78ee2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2c74144a00 0x7f2c7406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.838+0000 7f2c78ee2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2c74144a00 0x7f2c7406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48954/0 (socket says 172.21.15.60:48954) 2023-11-18T20:44:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.838+0000 7f2c78ee2640 1 -- 172.21.15.60:0/4027437140 learned_addr learned my addr 172.21.15.60:0/4027437140 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:48.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.838+0000 7f2c796e3640 1 --1- 172.21.15.60:0/4027437140 >> v1:172.21.15.60:6789/0 conn(0x7f2c741415f0 0x7f2c74141e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43646/0 (socket says 172.21.15.60:43646) 2023-11-18T20:44:48.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.838+0000 7f2c78ee2640 1 -- 172.21.15.60:0/4027437140 >> v1:172.21.15.100:6789/0 conn(0x7f2c74144440 legacy=0x7f2c740b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.838+0000 7f2c78ee2640 1 -- 172.21.15.60:0/4027437140 >> v1:172.21.15.60:6789/0 conn(0x7f2c741415f0 legacy=0x7f2c74141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.838+0000 7f2c78ee2640 1 -- 172.21.15.60:0/4027437140 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740b3470 con 0x7f2c74144a00 2023-11-18T20:44:48.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.839+0000 7f2c78ee2640 1 --2- 172.21.15.60:0/4027437140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144a00 0x7f2c7406b4e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009870 tx=0x7f2c6402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8804e23dd168162 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:48.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.839+0000 7f2c6bfff640 1 -- 172.21.15.60:0/4027437140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6402f840 con 0x7f2c74144a00 2023-11-18T20:44:48.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.839+0000 7f2c6bfff640 1 -- 172.21.15.60:0/4027437140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6402f9a0 con 0x7f2c74144a00 2023-11-18T20:44:48.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.839+0000 7f2c6bfff640 1 -- 172.21.15.60:0/4027437140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6402fd30 con 0x7f2c74144a00 2023-11-18T20:44:48.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.839+0000 7f2c7b947640 1 -- 172.21.15.60:0/4027437140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144a00 msgr2=0x7f2c7406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.839+0000 7f2c7b947640 1 --2- 172.21.15.60:0/4027437140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144a00 0x7f2c7406b4e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009870 tx=0x7f2c6402ba20 comp rx=0 tx=0).stop 2023-11-18T20:44:48.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.839+0000 7f2c7b947640 1 -- 172.21.15.60:0/4027437140 shutdown_connections 2023-11-18T20:44:48.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.839+0000 7f2c7b947640 1 --2- 172.21.15.60:0/4027437140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144a00 0x7f2c7406b4e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.839+0000 7f2c7b947640 1 -- 172.21.15.60:0/4027437140 >> 172.21.15.60:0/4027437140 conn(0x7f2c74070e70 msgr2=0x7f2c74071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:48.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.839+0000 7f2c7b947640 1 -- 172.21.15.60:0/4027437140 shutdown_connections 2023-11-18T20:44:48.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.840+0000 7f2c7b947640 1 -- 172.21.15.60:0/4027437140 wait complete. 2023-11-18T20:44:48.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.841+0000 7f2c7b947640 1 Processor -- start 2023-11-18T20:44:48.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.841+0000 7f2c7b947640 1 -- start start 2023-11-18T20:44:48.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.841+0000 7f2c7b947640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c741415f0 0x7f2c7413cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.841+0000 7f2c7b947640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144440 0x7f2c7413f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.841+0000 7f2c7b947640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c74144a00 0x7f2c740acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.841+0000 7f2c7b947640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c74159860 con 0x7f2c74144440 2023-11-18T20:44:48.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.841+0000 7f2c7b947640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c74159ab0 con 0x7f2c741415f0 2023-11-18T20:44:48.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.841+0000 7f2c7b947640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c74159d90 con 0x7f2c74144a00 2023-11-18T20:44:48.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.841+0000 7f2c78ee2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144440 0x7f2c7413f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.841+0000 7f2c78ee2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144440 0x7f2c7413f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48968/0 (socket says 172.21.15.60:48968) 2023-11-18T20:44:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c78ee2640 1 -- 172.21.15.60:0/3795724281 learned_addr learned my addr 172.21.15.60:0/3795724281 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c796e3640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c741415f0 0x7f2c7413cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c79ee4640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c74144a00 0x7f2c740acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c78ee2640 1 -- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c74144a00 msgr2=0x7f2c740acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c78ee2640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c74144a00 0x7f2c740acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c78ee2640 1 -- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c741415f0 msgr2=0x7f2c7413cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c78ee2640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c741415f0 0x7f2c7413cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c78ee2640 1 -- 172.21.15.60:0/3795724281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640094d0 con 0x7f2c74144440 2023-11-18T20:44:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c79ee4640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c74144a00 0x7f2c740acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:44:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c78ee2640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144440 0x7f2c7413f5f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2c6402ba60 tx=0x7f2c64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c69ffb640 1 -- 172.21.15.60:0/3795724281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6403c040 con 0x7f2c74144440 2023-11-18T20:44:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c69ffb640 1 -- 172.21.15.60:0/3795724281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6402f810 con 0x7f2c74144440 2023-11-18T20:44:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c69ffb640 1 -- 172.21.15.60:0/3795724281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64042610 con 0x7f2c74144440 2023-11-18T20:44:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c7b947640 1 -- 172.21.15.60:0/3795724281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7415a070 con 0x7f2c74144440 2023-11-18T20:44:48.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.842+0000 7f2c7b947640 1 -- 172.21.15.60:0/3795724281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7415a630 con 0x7f2c74144440 2023-11-18T20:44:48.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.844+0000 7f2c7b947640 1 -- 172.21.15.60:0/3795724281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c74069400 con 0x7f2c74144440 2023-11-18T20:44:48.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.844+0000 7f2c69ffb640 1 -- 172.21.15.60:0/3795724281 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c6402d3d0 con 0x7f2c74144440 2023-11-18T20:44:48.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.844+0000 7f2c69ffb640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2c5806ae70 0x7f2c5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:48.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.844+0000 7f2c69ffb640 1 -- 172.21.15.60:0/3795724281 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c640ab870 con 0x7f2c74144440 2023-11-18T20:44:48.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.845+0000 7f2c796e3640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2c5806ae70 0x7f2c5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:48.848 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.845+0000 7f2c796e3640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2c5806ae70 0x7f2c5806d330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c000f60 tx=0x7f2c6c007c80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:48.851 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:48.849+0000 7f2c69ffb640 1 -- 172.21.15.60:0/3795724281 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c64076920 con 0x7f2c74144440 2023-11-18T20:44:49.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.038+0000 7f2c7b947640 1 -- 172.21.15.60:0/3795724281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2c7406f7f0 con 0x7f2c74144440 2023-11-18T20:44:49.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.038+0000 7f2c69ffb640 1 -- 172.21.15.60:0/3795724281 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2c64047020 con 0x7f2c74144440 2023-11-18T20:44:49.040 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:44:49.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 -- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2c5806ae70 msgr2=0x7f2c5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2c5806ae70 0x7f2c5806d330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c000f60 tx=0x7f2c6c007c80 comp rx=0 tx=0).stop 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 -- 172.21.15.60:0/3795724281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144440 msgr2=0x7f2c7413f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144440 0x7f2c7413f5f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2c6402ba60 tx=0x7f2c64004730 comp rx=0 tx=0).stop 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 -- 172.21.15.60:0/3795724281 shutdown_connections 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2c5806ae70 0x7f2c5806d330 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c74144a00 0x7f2c740acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c74144440 0x7f2c7413f5f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 --2- 172.21.15.60:0/3795724281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c741415f0 0x7f2c7413cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 -- 172.21.15.60:0/3795724281 >> 172.21.15.60:0/3795724281 conn(0x7f2c74070e70 msgr2=0x7f2c74142d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 -- 172.21.15.60:0/3795724281 shutdown_connections 2023-11-18T20:44:49.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:49.040+0000 7f2c7b947640 1 -- 172.21.15.60:0/3795724281 wait complete. 2023-11-18T20:44:49.053 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-11-18T20:44:49.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:49.053 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:49.082 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[107601]: fuse finished with error 0 and tester_r 0 2023-11-18T20:44:49.084 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:44:55.186 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:44:55.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:55.186 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-18T20:44:55.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-18T20:44:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.362+0000 7fe67783f640 1 Processor -- start 2023-11-18T20:44:55.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.362+0000 7fe67783f640 1 -- start start 2023-11-18T20:44:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe67783f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe67015c2f0 0x7fe67006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe67783f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe670140f10 0x7fe67013f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:55.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe67783f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67015ba40 con 0x7fe67006f040 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe67783f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67015c6d0 con 0x7fe67015c2f0 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe67783f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67013e530 con 0x7fe670140f10 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe6755db640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe67015c2f0 0x7fe67006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe674dda640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe67006f040 0x7fe670140800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56034/0 (socket says 172.21.15.60:56034) 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe6755db640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe67015c2f0 0x7fe67006eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39884/0 (socket says 172.21.15.60:39884) 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe674dda640 1 -- 172.21.15.60:0/3812185570 learned_addr learned my addr 172.21.15.60:0/3812185570 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe675ddc640 1 --2- 172.21.15.60:0/3812185570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe670140f10 0x7fe67013f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe6755db640 1 -- 172.21.15.60:0/3812185570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe670140f10 msgr2=0x7fe67013f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe6755db640 1 --2- 172.21.15.60:0/3812185570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe670140f10 0x7fe67013f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe6755db640 1 -- 172.21.15.60:0/3812185570 >> v1:172.21.15.60:6789/0 conn(0x7fe67006f040 legacy=0x7fe670140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe66ffff640 1 -- 172.21.15.60:0/3812185570 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605369272 0 0) 0x7fe67015ba40 con 0x7fe67006f040 2023-11-18T20:44:55.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.363+0000 7fe6755db640 1 -- 172.21.15.60:0/3812185570 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67006f4a0 con 0x7fe67015c2f0 2023-11-18T20:44:55.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.364+0000 7fe6755db640 1 --2- 172.21.15.60:0/3812185570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 0x7fe67006eb00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe668009dc0 tx=0x7fe66802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4cba4f458759df server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.364+0000 7fe66ffff640 1 -- 172.21.15.60:0/3812185570 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66802cc40 con 0x7fe67015c2f0 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.364+0000 7fe66ffff640 1 -- 172.21.15.60:0/3812185570 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66802cda0 con 0x7fe67015c2f0 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.364+0000 7fe66ffff640 1 -- 172.21.15.60:0/3812185570 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668034660 con 0x7fe67015c2f0 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.365+0000 7fe67783f640 1 -- 172.21.15.60:0/3812185570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 msgr2=0x7fe67006eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.365+0000 7fe67783f640 1 --2- 172.21.15.60:0/3812185570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 0x7fe67006eb00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe668009dc0 tx=0x7fe66802be90 comp rx=0 tx=0).stop 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.365+0000 7fe67783f640 1 -- 172.21.15.60:0/3812185570 shutdown_connections 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.365+0000 7fe67783f640 1 --2- 172.21.15.60:0/3812185570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe670140f10 0x7fe67013f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.365+0000 7fe67783f640 1 --2- 172.21.15.60:0/3812185570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 0x7fe67006eb00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.365+0000 7fe67783f640 1 -- 172.21.15.60:0/3812185570 >> 172.21.15.60:0/3812185570 conn(0x7fe67006a6b0 msgr2=0x7fe67006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.365+0000 7fe67783f640 1 -- 172.21.15.60:0/3812185570 shutdown_connections 2023-11-18T20:44:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.365+0000 7fe67783f640 1 -- 172.21.15.60:0/3812185570 wait complete. 2023-11-18T20:44:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.366+0000 7fe67783f640 1 Processor -- start 2023-11-18T20:44:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.366+0000 7fe67783f640 1 -- start start 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe67783f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67006f040 0x7fe6701f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe67783f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe670140f10 0x7fe6701513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe67783f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 0x7fe6700b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe67783f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe670132e60 con 0x7fe67015c2f0 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe67783f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701f71d0 con 0x7fe670140f10 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe67783f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6701f6930 con 0x7fe67006f040 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe675ddc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 0x7fe6700b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe675ddc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 0x7fe6700b57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39898/0 (socket says 172.21.15.60:39898) 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe675ddc640 1 -- 172.21.15.60:0/3908082080 learned_addr learned my addr 172.21.15.60:0/3908082080 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe675ddc640 1 -- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67006f040 msgr2=0x7fe6701f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe6755db640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67006f040 0x7fe6701f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe674dda640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe670140f10 0x7fe6701513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:55.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe675ddc640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67006f040 0x7fe6701f6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe675ddc640 1 -- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe670140f10 msgr2=0x7fe6701513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe675ddc640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe670140f10 0x7fe6701513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.367+0000 7fe675ddc640 1 -- 172.21.15.60:0/3908082080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe668009a70 con 0x7fe67015c2f0 2023-11-18T20:44:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.368+0000 7fe675ddc640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 0x7fe6700b57e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe66000e7a0 tx=0x7fe66000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.368+0000 7fe66dffb640 1 -- 172.21.15.60:0/3908082080 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66000c730 con 0x7fe67015c2f0 2023-11-18T20:44:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.368+0000 7fe66dffb640 1 -- 172.21.15.60:0/3908082080 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66000c890 con 0x7fe67015c2f0 2023-11-18T20:44:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.368+0000 7fe67783f640 1 -- 172.21.15.60:0/3908082080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6701f6c10 con 0x7fe67015c2f0 2023-11-18T20:44:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.368+0000 7fe66dffb640 1 -- 172.21.15.60:0/3908082080 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66000cc40 con 0x7fe67015c2f0 2023-11-18T20:44:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.368+0000 7fe67783f640 1 -- 172.21.15.60:0/3908082080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67006bae0 con 0x7fe67015c2f0 2023-11-18T20:44:55.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.369+0000 7fe67783f640 1 -- 172.21.15.60:0/3908082080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67006bd00 con 0x7fe67015c2f0 2023-11-18T20:44:55.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.369+0000 7fe66dffb640 1 -- 172.21.15.60:0/3908082080 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe660013470 con 0x7fe67015c2f0 2023-11-18T20:44:55.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.370+0000 7fe66dffb640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe65c06ae40 0x7fe65c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:55.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.370+0000 7fe66dffb640 1 -- 172.21.15.60:0/3908082080 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6600896f0 con 0x7fe67015c2f0 2023-11-18T20:44:55.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.370+0000 7fe6755db640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe65c06ae40 0x7fe65c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:55.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.371+0000 7fe6755db640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe65c06ae40 0x7fe65c06d300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe668036000 tx=0x7fe668030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.374+0000 7fe66dffb640 1 -- 172.21.15.60:0/3908082080 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe660054830 con 0x7fe67015c2f0 2023-11-18T20:44:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.589+0000 7fe67783f640 1 -- 172.21.15.60:0/3908082080 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe670069400 con 0x7fe67015c2f0 2023-11-18T20:44:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.590+0000 7fe66dffb640 1 -- 172.21.15.60:0/3908082080 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe660017070 con 0x7fe67015c2f0 2023-11-18T20:44:55.592 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-18T20:44:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 -- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe65c06ae40 msgr2=0x7fe65c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe65c06ae40 0x7fe65c06d300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe668036000 tx=0x7fe668030000 comp rx=0 tx=0).stop 2023-11-18T20:44:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 -- 172.21.15.60:0/3908082080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 msgr2=0x7fe6700b57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 0x7fe6700b57e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe66000e7a0 tx=0x7fe66000ec60 comp rx=0 tx=0).stop 2023-11-18T20:44:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 -- 172.21.15.60:0/3908082080 shutdown_connections 2023-11-18T20:44:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe65c06ae40 0x7fe65c06d300 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67015c2f0 0x7fe6700b57e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe670140f10 0x7fe6701513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 --2- 172.21.15.60:0/3908082080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67006f040 0x7fe6701f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 -- 172.21.15.60:0/3908082080 >> 172.21.15.60:0/3908082080 conn(0x7fe67006a6b0 msgr2=0x7fe67015da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 -- 172.21.15.60:0/3908082080 shutdown_connections 2023-11-18T20:44:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:55.592+0000 7fe67783f640 1 -- 172.21.15.60:0/3908082080 wait complete. 2023-11-18T20:44:55.605 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-18T20:44:55.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-18T20:44:55.605 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2023-11-18T20:44:55.605 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-18T20:44:55.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:55.605 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-18T20:44:55.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-18T20:44:55.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-18T20:44:55.606 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:55.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:55.606 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2023-11-18T20:44:55.621 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-18T20:44:55.621 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-18T20:44:55.621 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2023-11-18T20:44:55.621 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2023-11-18T20:44:55.622 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42561sec preferred_lft 42561sec 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::d896:cfff:fef3:88dc/64 scope link 2023-11-18T20:44:55.623 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:55.624 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:44:55.624 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0a:c3:0c:c2:e6:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:55.624 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8c3:cff:fec2:e672/64 scope link 2023-11-18T20:44:55.624 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:55.624 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-18T20:44:55.624 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 06:0c:de:9d:c1:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:55.624 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::40c:deff:fe9d:c1f6/64 scope link 2023-11-18T20:44:55.624 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2023-11-18T20:44:55.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:55.624 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-18T20:44:55.691 INFO:teuthology.orchestra.run.smithi060.stdout:1 2023-11-18T20:44:55.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:55.693 DEBUG:teuthology.orchestra.run.smithi060:> route 2023-11-18T20:44:55.749 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2023-11-18T20:44:55.749 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-18T20:44:55.749 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-18T20:44:55.749 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-18T20:44:55.749 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-18T20:44:55.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:55.750 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2023-11-18T20:44:55.750 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2023-11-18T20:44:55.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-18T20:44:55.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-18T20:44:55.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-18T20:44:55.750 DEBUG:teuthology.orchestra.run.smithi060:> ') 2023-11-18T20:44:55.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:55.855 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-11-18T20:44:55.870 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-18T20:44:55.870 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-18T20:44:55.870 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:55.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:55.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2023-11-18T20:44:55.942 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-18T20:44:55.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:44:55.942 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:55.997 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-18T20:44:55.997 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-18T20:44:55.997 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:56.051 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:44:56.123 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:44:56.180 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:44:56.180 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:44:56.180 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:44:56.180 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:44:56.180 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:44:56.180 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:44:56.180 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:44:56.180 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:44:56.180 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:44:56.181 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:56.182 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:56.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:56.183 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:44:56.235 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-18T20:44:56.235 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-18T20:44:56.277 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2023-11-18T20:44:56.307 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2023-11-18T20:44:56.363 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2023-11-18T20:44:56.363 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2023-11-18T20:44:56.363 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2023-11-18T20:44:56.363 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2023-11-18T20:44:56.363 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2023-11-18T20:44:56.364 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2023-11-18T20:44:56.365 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2023-11-18T20:44:56.366 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:56.366 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:44:56.366 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:56.366 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:44:56.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:56.367 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2023-11-18T20:44:56.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:2023-11-18T20:44:56.368+0000 7f24fa036480 -1 init, newargv = 0x564b3ffc46f0 newargc=15 2023-11-18T20:44:56.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[108563]: starting ceph client 2023-11-18T20:44:56.375 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[108563]: starting fuse 2023-11-18T20:44:56.386 INFO:teuthology.orchestra.run.smithi060.stdout:41 2023-11-18T20:44:56.386 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-18T20:44:56.386 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2023-11-18T20:44:56.386 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2023-11-18T20:44:56.386 DEBUG:teuthology.orchestra.run.smithi060:> import re 2023-11-18T20:44:56.386 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:56.387 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2023-11-18T20:44:56.388 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-18T20:44:56.388 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-18T20:44:56.388 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-18T20:44:56.388 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2023-11-18T20:44:56.388 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2023-11-18T20:44:56.388 DEBUG:teuthology.orchestra.run.smithi060:> return f 2023-11-18T20:44:56.388 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-18T20:44:56.388 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:56.388 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.0")) 2023-11-18T20:44:56.388 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-11-18T20:44:56.510 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.0.108563.asok 2023-11-18T20:44:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-18T20:44:56.521 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.108563.asok 2023-11-18T20:44:56.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:56.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.108563.asok status 2023-11-18T20:44:56.662 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:44:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2023-11-18T20:44:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe", 2023-11-18T20:44:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-7418-gb0f45a51 (b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe) reef (dev)", 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "0", 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "108563", 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6219, 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2023-11-18T20:44:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 6219 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 415505894 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2023-11-18T20:44:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 415505894 2023-11-18T20:44:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-18T20:44:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.6219 192.168.0.1:0/415505894", 2023-11-18T20:44:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/415505894", 2023-11-18T20:44:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2023-11-18T20:44:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 21, 2023-11-18T20:44:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 49, 2023-11-18T20:44:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2023-11-18T20:44:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2023-11-18T20:44:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2023-11-18T20:44:56.665 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:44:56.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:56.671 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2023-11-18T20:44:56.671 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:56.686 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2023-11-18T20:44:56.687 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:56.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:56.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:56.758 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2023-11-18T20:44:56.758 DEBUG:teuthology.orchestra.run.smithi060:> import os 2023-11-18T20:44:56.758 DEBUG:teuthology.orchestra.run.smithi060:> import errno 2023-11-18T20:44:56.758 DEBUG:teuthology.orchestra.run.smithi060:> 2023-11-18T20:44:56.758 DEBUG:teuthology.orchestra.run.smithi060:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-11-18T20:44:56.758 DEBUG:teuthology.orchestra.run.smithi060:> try: 2023-11-18T20:44:56.758 DEBUG:teuthology.orchestra.run.smithi060:> if True: 2023-11-18T20:44:56.758 DEBUG:teuthology.orchestra.run.smithi060:> ret = os.read(fd, 1024) 2023-11-18T20:44:56.758 DEBUG:teuthology.orchestra.run.smithi060:> else: 2023-11-18T20:44:56.759 DEBUG:teuthology.orchestra.run.smithi060:> os.write(fd, b'"'"'content'"'"') 2023-11-18T20:44:56.759 DEBUG:teuthology.orchestra.run.smithi060:> except OSError as e: 2023-11-18T20:44:56.759 DEBUG:teuthology.orchestra.run.smithi060:> if e.errno != errno.EPERM: 2023-11-18T20:44:56.759 DEBUG:teuthology.orchestra.run.smithi060:> raise 2023-11-18T20:44:56.759 DEBUG:teuthology.orchestra.run.smithi060:> else: 2023-11-18T20:44:56.759 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("client does not check permission of data pool") 2023-11-18T20:44:56.759 DEBUG:teuthology.orchestra.run.smithi060:> ' 2023-11-18T20:44:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:Traceback (most recent call last): 2023-11-18T20:44:56.866 INFO:teuthology.orchestra.run.smithi060.stderr: File "", line 15, in 2023-11-18T20:44:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:RuntimeError: client does not check permission of data pool 2023-11-18T20:44:56.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:56 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2023-11-18T20:44:56.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:44:56.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2023-11-18T20:44:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.027+0000 7f758b501640 1 Processor -- start 2023-11-18T20:44:57.030 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.027+0000 7f758b501640 1 -- start start 2023-11-18T20:44:57.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.028+0000 7f758b501640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f75840b8ec0 0x7f75840d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.028+0000 7f758b501640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758415d8b0 con 0x7f75840b4cd0 2023-11-18T20:44:57.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.028+0000 7f758b501640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75840d1ae0 con 0x7f758415d4d0 2023-11-18T20:44:57.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.028+0000 7f758b501640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75840d1dc0 con 0x7f75840b8ec0 2023-11-18T20:44:57.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.028+0000 7f7588a9c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f75840b8ec0 0x7f75840d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.028+0000 7f7588a9c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f75840b8ec0 0x7f75840d2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39906/0 (socket says 172.21.15.60:39906) 2023-11-18T20:44:57.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.028+0000 7f7588a9c640 1 -- 172.21.15.60:0/4072792297 learned_addr learned my addr 172.21.15.60:0/4072792297 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:57.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.028+0000 7f7589a9e640 1 --1- 172.21.15.60:0/4072792297 >> v1:172.21.15.60:6789/0 conn(0x7f75840b4cd0 0x7f75840b6380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56050/0 (socket says 172.21.15.60:56050) 2023-11-18T20:44:57.031 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.029+0000 7f7588a9c640 1 -- 172.21.15.60:0/4072792297 >> v1:172.21.15.100:6789/0 conn(0x7f758415d4d0 legacy=0x7f75840d1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:57.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.029+0000 7f7588a9c640 1 -- 172.21.15.60:0/4072792297 >> v1:172.21.15.60:6789/0 conn(0x7f75840b4cd0 legacy=0x7f75840b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.029+0000 7f7583fff640 1 -- 172.21.15.60:0/4072792297 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4206218887 0 0) 0x7f758415d8b0 con 0x7f75840b4cd0 2023-11-18T20:44:57.032 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.029+0000 7f7588a9c640 1 -- 172.21.15.60:0/4072792297 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75840d20a0 con 0x7f75840b8ec0 2023-11-18T20:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.029+0000 7f7588a9c640 1 --2- 172.21.15.60:0/4072792297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75840b8ec0 0x7f75840d2b80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7578009870 tx=0x7f757802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcb79c95c25099d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.029+0000 7f7583fff640 1 -- 172.21.15.60:0/4072792297 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578030860 con 0x7f75840b8ec0 2023-11-18T20:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.029+0000 7f7583fff640 1 -- 172.21.15.60:0/4072792297 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75780309c0 con 0x7f75840b8ec0 2023-11-18T20:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.030+0000 7f7583fff640 1 -- 172.21.15.60:0/4072792297 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578030d50 con 0x7f75840b8ec0 2023-11-18T20:44:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.030+0000 7f758b501640 1 -- 172.21.15.60:0/4072792297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75840b8ec0 msgr2=0x7f75840d2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.030+0000 7f758b501640 1 --2- 172.21.15.60:0/4072792297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75840b8ec0 0x7f75840d2b80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7578009870 tx=0x7f757802b9a0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.030+0000 7f758b501640 1 -- 172.21.15.60:0/4072792297 shutdown_connections 2023-11-18T20:44:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.030+0000 7f758b501640 1 --2- 172.21.15.60:0/4072792297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75840b8ec0 0x7f75840d2b80 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.030+0000 7f758b501640 1 -- 172.21.15.60:0/4072792297 >> 172.21.15.60:0/4072792297 conn(0x7f7584072230 msgr2=0x7f75840cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.030+0000 7f758b501640 1 -- 172.21.15.60:0/4072792297 shutdown_connections 2023-11-18T20:44:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.030+0000 7f758b501640 1 -- 172.21.15.60:0/4072792297 wait complete. 2023-11-18T20:44:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.031+0000 7f758b501640 1 Processor -- start 2023-11-18T20:44:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.031+0000 7f758b501640 1 -- start start 2023-11-18T20:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f758b501640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b4cd0 0x7f75840c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f758b501640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75840b8ec0 0x7f75840cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f758b501640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415d4d0 0x7f75840cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f758b501640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75840c91e0 con 0x7f758415d4d0 2023-11-18T20:44:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f758b501640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758415fb60 con 0x7f75840b4cd0 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f758b501640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758415fe40 con 0x7f75840b8ec0 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f7589a9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415d4d0 0x7f75840cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f7589a9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415d4d0 0x7f75840cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39916/0 (socket says 172.21.15.60:39916) 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f7589a9e640 1 -- 172.21.15.60:0/1639156836 learned_addr learned my addr 172.21.15.60:0/1639156836 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f758929d640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b4cd0 0x7f75840c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f7588a9c640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75840b8ec0 0x7f75840cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f7589a9e640 1 -- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75840b8ec0 msgr2=0x7f75840cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f7589a9e640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75840b8ec0 0x7f75840cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f7589a9e640 1 -- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b4cd0 msgr2=0x7f75840c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f7589a9e640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b4cd0 0x7f75840c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f7589a9e640 1 -- 172.21.15.60:0/1639156836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75780094d0 con 0x7f758415d4d0 2023-11-18T20:44:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.032+0000 7f758929d640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b4cd0 0x7f75840c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:57.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.033+0000 7f7589a9e640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415d4d0 0x7f75840cffa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f757400c7e0 tx=0x7f757400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:57.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.033+0000 7f7581ffb640 1 -- 172.21.15.60:0/1639156836 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757400a640 con 0x7f758415d4d0 2023-11-18T20:44:57.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.033+0000 7f7581ffb640 1 -- 172.21.15.60:0/1639156836 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757400a7c0 con 0x7f758415d4d0 2023-11-18T20:44:57.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.033+0000 7f7581ffb640 1 -- 172.21.15.60:0/1639156836 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757400ab70 con 0x7f758415d4d0 2023-11-18T20:44:57.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.033+0000 7f758b501640 1 -- 172.21.15.60:0/1639156836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7584160120 con 0x7f758415d4d0 2023-11-18T20:44:57.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.033+0000 7f758b501640 1 -- 172.21.15.60:0/1639156836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75841606e0 con 0x7f758415d4d0 2023-11-18T20:44:57.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.034+0000 7f758b501640 1 -- 172.21.15.60:0/1639156836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75841608d0 con 0x7f758415d4d0 2023-11-18T20:44:57.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.034+0000 7f7581ffb640 1 -- 172.21.15.60:0/1639156836 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f757400acd0 con 0x7f758415d4d0 2023-11-18T20:44:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.034+0000 7f7581ffb640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f757006ae40 0x7f757006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.035+0000 7f7581ffb640 1 -- 172.21.15.60:0/1639156836 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7574088c30 con 0x7f758415d4d0 2023-11-18T20:44:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.035+0000 7f758929d640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f757006ae40 0x7f757006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.035+0000 7f758929d640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f757006ae40 0x7f757006d300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f757c006fd0 tx=0x7f757c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:57.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.039+0000 7f7581ffb640 1 -- 172.21.15.60:0/1639156836 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7574053ce0 con 0x7f758415d4d0 2023-11-18T20:44:57.285 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.283+0000 7f758b501640 1 -- 172.21.15.60:0/1639156836 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f7584069400 con 0x7f758415d4d0 2023-11-18T20:44:57.288 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.286+0000 7f7581ffb640 1 -- 172.21.15.60:0/1639156836 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v20) v1 ==== 184+0+141 (secure 0 0 0) 0x7f7574018070 con 0x7f758415d4d0 2023-11-18T20:44:57.288 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2023-11-18T20:44:57.288 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCFIVllhityLxAAA8jTaIDFIqvELKeKPMtTEA== 2023-11-18T20:44:57.288 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2023-11-18T20:44:57.288 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2023-11-18T20:44:57.288 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw pool=cephfs_data" 2023-11-18T20:44:57.288 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2023-11-18T20:44:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 -- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f757006ae40 msgr2=0x7f757006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f757006ae40 0x7f757006d300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f757c006fd0 tx=0x7f757c008040 comp rx=0 tx=0).stop 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 -- 172.21.15.60:0/1639156836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415d4d0 msgr2=0x7f75840cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415d4d0 0x7f75840cffa0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f757400c7e0 tx=0x7f757400cca0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 -- 172.21.15.60:0/1639156836 shutdown_connections 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f757006ae40 0x7f757006d300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415d4d0 0x7f75840cffa0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75840b8ec0 0x7f75840cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 --2- 172.21.15.60:0/1639156836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b4cd0 0x7f75840c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 -- 172.21.15.60:0/1639156836 >> 172.21.15.60:0/1639156836 conn(0x7f7584072230 msgr2=0x7f75840c95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 -- 172.21.15.60:0/1639156836 shutdown_connections 2023-11-18T20:44:57.291 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.288+0000 7f758b501640 1 -- 172.21.15.60:0/1639156836 wait complete. 2023-11-18T20:44:57.301 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-18T20:44:57.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-18T20:44:57.347 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-18T20:44:57.399 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-18T20:44:57.399 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:57.399 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:57.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:57.561 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.559+0000 7fc8006da640 1 Processor -- start 2023-11-18T20:44:57.562 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.559+0000 7fc8006da640 1 -- start start 2023-11-18T20:44:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.560+0000 7fc8006da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 0x7fc7f80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.560+0000 7fc8006da640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7f80d2d90 0x7fc7f80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.560+0000 7fc8006da640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f80b2b10 con 0x7fc7f80d2910 2023-11-18T20:44:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.560+0000 7fc8006da640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f80d1590 con 0x7fc7f80d2d90 2023-11-18T20:44:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.560+0000 7fc8006da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80d1870 con 0x7fc7f815d330 2023-11-18T20:44:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.560+0000 7fc7fec77640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7f80d2d90 0x7fc7f80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.560+0000 7fc7fec77640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7f80d2d90 0x7fc7f80b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39926/0 (socket says 172.21.15.60:39926) 2023-11-18T20:44:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.561+0000 7fc7fec77640 1 -- 172.21.15.60:0/2796493709 learned_addr learned my addr 172.21.15.60:0/2796493709 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.561+0000 7fc7fe476640 1 --2- 172.21.15.60:0/2796493709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 0x7fc7f80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.561+0000 7fc7fec77640 1 -- 172.21.15.60:0/2796493709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 msgr2=0x7fc7f80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.561+0000 7fc7fec77640 1 --2- 172.21.15.60:0/2796493709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 0x7fc7f80b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.561+0000 7fc7fec77640 1 -- 172.21.15.60:0/2796493709 >> v1:172.21.15.100:6789/0 conn(0x7fc7f80d2910 legacy=0x7fc7f80d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.561+0000 7fc7fec77640 1 -- 172.21.15.60:0/2796493709 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80d1b50 con 0x7fc7f80d2d90 2023-11-18T20:44:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.561+0000 7fc7fe476640 1 --2- 172.21.15.60:0/2796493709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 0x7fc7f80b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc7fec77640 1 --2- 172.21.15.60:0/2796493709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2d90 0x7fc7f80b6390 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009e50 tx=0x7fc7e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546e2266c37203fa server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc7fd474640 1 -- 172.21.15.60:0/2796493709 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e802ca70 con 0x7fc7f80d2d90 2023-11-18T20:44:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc7fd474640 1 -- 172.21.15.60:0/2796493709 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e802cbd0 con 0x7fc7f80d2d90 2023-11-18T20:44:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc7fd474640 1 -- 172.21.15.60:0/2796493709 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80343f0 con 0x7fc7f80d2d90 2023-11-18T20:44:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc8006da640 1 -- 172.21.15.60:0/2796493709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2d90 msgr2=0x7fc7f80b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc8006da640 1 --2- 172.21.15.60:0/2796493709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2d90 0x7fc7f80b6390 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009e50 tx=0x7fc7e802bed0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc8006da640 1 -- 172.21.15.60:0/2796493709 shutdown_connections 2023-11-18T20:44:57.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc8006da640 1 --2- 172.21.15.60:0/2796493709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2d90 0x7fc7f80b6390 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc8006da640 1 --2- 172.21.15.60:0/2796493709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 0x7fc7f80b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc8006da640 1 -- 172.21.15.60:0/2796493709 >> 172.21.15.60:0/2796493709 conn(0x7fc7f8072020 msgr2=0x7fc7f8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:57.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc8006da640 1 -- 172.21.15.60:0/2796493709 shutdown_connections 2023-11-18T20:44:57.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.562+0000 7fc8006da640 1 -- 172.21.15.60:0/2796493709 wait complete. 2023-11-18T20:44:57.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc8006da640 1 Processor -- start 2023-11-18T20:44:57.567 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc8006da640 1 -- start start 2023-11-18T20:44:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc8006da640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2910 0x7fc7f814b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc8006da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d2d90 0x7fc7f80cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc8006da640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 0x7fc7f815f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc8006da640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80cd1b0 con 0x7fc7f80d2910 2023-11-18T20:44:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc8006da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80cad40 con 0x7fc7f80d2d90 2023-11-18T20:44:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc8006da640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80cbba0 con 0x7fc7f815d330 2023-11-18T20:44:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc7fe476640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2910 0x7fc7f814b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc7fe476640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2910 0x7fc7f814b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39930/0 (socket says 172.21.15.60:39930) 2023-11-18T20:44:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.564+0000 7fc7fe476640 1 -- 172.21.15.60:0/2265745589 learned_addr learned my addr 172.21.15.60:0/2265745589 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7fdc75640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d2d90 0x7fc7f80cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7fec77640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 0x7fc7f815f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7fe476640 1 -- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 msgr2=0x7fc7f815f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7fe476640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 0x7fc7f815f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7fe476640 1 -- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d2d90 msgr2=0x7fc7f80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7fe476640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d2d90 0x7fc7f80cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7fe476640 1 -- 172.21.15.60:0/2265745589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e8009b30 con 0x7fc7f80d2910 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7fdc75640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d2d90 0x7fc7f80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7fe476640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2910 0x7fc7f814b610 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000d7d0 tx=0x7fc7f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7f6ffd640 1 -- 172.21.15.60:0/2265745589 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0004030 con 0x7fc7f80d2910 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7f6ffd640 1 -- 172.21.15.60:0/2265745589 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f0004190 con 0x7fc7f80d2910 2023-11-18T20:44:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc8006da640 1 -- 172.21.15.60:0/2265745589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f80caf90 con 0x7fc7f80d2910 2023-11-18T20:44:57.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc7f6ffd640 1 -- 172.21.15.60:0/2265745589 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0004520 con 0x7fc7f80d2910 2023-11-18T20:44:57.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.565+0000 7fc8006da640 1 -- 172.21.15.60:0/2265745589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f80cb550 con 0x7fc7f80d2910 2023-11-18T20:44:57.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.566+0000 7fc8006da640 1 -- 172.21.15.60:0/2265745589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f8069400 con 0x7fc7f80d2910 2023-11-18T20:44:57.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.566+0000 7fc7f6ffd640 1 -- 172.21.15.60:0/2265745589 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc7f00096a0 con 0x7fc7f80d2910 2023-11-18T20:44:57.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.567+0000 7fc7f6ffd640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7e406ae40 0x7fc7e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.567+0000 7fc7f6ffd640 1 -- 172.21.15.60:0/2265745589 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7f0089800 con 0x7fc7f80d2910 2023-11-18T20:44:57.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.567+0000 7fc7fdc75640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7e406ae40 0x7fc7e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.572 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.568+0000 7fc7fdc75640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7e406ae40 0x7fc7e406d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec007fd0 tx=0x7fc7ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:57.573 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.571+0000 7fc7f6ffd640 1 -- 172.21.15.60:0/2265745589 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc7f0054810 con 0x7fc7f80d2910 2023-11-18T20:44:57.751 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.749+0000 7fc8006da640 1 -- 172.21.15.60:0/2265745589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7f80b3a10 con 0x7fc7f80d2910 2023-11-18T20:44:57.752 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.750+0000 7fc7f6ffd640 1 -- 172.21.15.60:0/2265745589 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fc7f0057e30 con 0x7fc7f80d2910 2023-11-18T20:44:57.752 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:57.752 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:34.103787+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:44:57.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 -- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7e406ae40 msgr2=0x7fc7e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7e406ae40 0x7fc7e406d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec007fd0 tx=0x7fc7ec009040 comp rx=0 tx=0).stop 2023-11-18T20:44:57.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 -- 172.21.15.60:0/2265745589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2910 msgr2=0x7fc7f814b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2910 0x7fc7f814b610 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000d7d0 tx=0x7fc7f000dc90 comp rx=0 tx=0).stop 2023-11-18T20:44:57.754 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 -- 172.21.15.60:0/2265745589 shutdown_connections 2023-11-18T20:44:57.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc7e406ae40 0x7fc7e406d300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815d330 0x7fc7f815f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d2d90 0x7fc7f80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 --2- 172.21.15.60:0/2265745589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80d2910 0x7fc7f814b610 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 -- 172.21.15.60:0/2265745589 >> 172.21.15.60:0/2265745589 conn(0x7fc7f8072020 msgr2=0x7fc7f80b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:57.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 -- 172.21.15.60:0/2265745589 shutdown_connections 2023-11-18T20:44:57.755 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.752+0000 7fc8006da640 1 -- 172.21.15.60:0/2265745589 wait complete. 2023-11-18T20:44:57.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:44:31.358325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:44:32.036538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:44:57.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:44:57.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.872+0000 7f4905294640 1 Processor -- start 2023-11-18T20:44:57.875 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.872+0000 7f4905294640 1 -- start start 2023-11-18T20:44:57.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.873+0000 7f4905294640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000d3900 0x7f49000b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.873+0000 7f4905294640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000b2b10 con 0x7f490015d330 2023-11-18T20:44:57.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.873+0000 7f4905294640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000d1590 con 0x7f49000d2910 2023-11-18T20:44:57.878 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.873+0000 7f4905294640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49000d1870 con 0x7f49000d3900 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.873+0000 7f48feffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f490015d330 0x7f49000b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56058/0 (socket says 172.21.15.60:56058) 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.873+0000 7f48feffd640 1 -- 172.21.15.60:0/4053194020 learned_addr learned my addr 172.21.15.60:0/4053194020 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48ff7fe640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000d3900 0x7f49000b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297589541 0 0) 0x7f49000b2b10 con 0x7f490015d330 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e8003520 con 0x7f490015d330 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475347793 0 0) 0x7f49000d1590 con 0x7f49000d2910 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49000b2b10 con 0x7f49000d2910 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3591311207 0 0) 0x7f48e8003520 con 0x7f490015d330 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000d3900 msgr2=0x7f49000b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000d3900 0x7f49000b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.879 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 >> v1:172.21.15.100:6789/0 conn(0x7f49000d2910 legacy=0x7f49000d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000d1b50 con 0x7f490015d330 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f48f8001670 con 0x7f490015d330 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 >> v1:172.21.15.60:6789/0 conn(0x7f490015d330 legacy=0x7f49000b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e8003b30 0x7f48e8003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48e80047c0 0x7f48e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000d3900 0x7f48e80051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49000d1870 con 0x7f48e8003b30 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e800ba90 con 0x7f48e80047c0 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e800bc50 con 0x7f49000d3900 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48feffd640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e8003b30 0x7f48e8003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48fe7fc640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48e80047c0 0x7f48e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48ff7fe640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000d3900 0x7f48e80051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48feffd640 1 -- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000d3900 msgr2=0x7f48e80051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48feffd640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000d3900 0x7f48e80051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48feffd640 1 -- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48e80047c0 msgr2=0x7f48e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48feffd640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48e80047c0 0x7f48e8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48feffd640 1 -- 172.21.15.60:0/4053194020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e8003520 con 0x7f48e8003b30 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.874+0000 7f48ff7fe640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49000d3900 0x7f48e80051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.875+0000 7f48feffd640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e8003b30 0x7f48e8003f30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f48f8007fd0 tx=0x7f48e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea85e26a4d937356 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.875+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8005ad0 con 0x7f48e8003b30 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.875+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f8005c50 con 0x7f48e8003b30 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.875+0000 7f48fdffb640 1 -- 172.21.15.60:0/4053194020 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f802c850 con 0x7f48e8003b30 2023-11-18T20:44:57.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.876+0000 7f4905294640 1 -- 172.21.15.60:0/4053194020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e8003b30 msgr2=0x7f48e8003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.876+0000 7f4905294640 1 --2- 172.21.15.60:0/4053194020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48e8003b30 0x7f48e8003f30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f48f8007fd0 tx=0x7f48e80057f0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.876+0000 7f48ff7fe640 1 -- 172.21.15.60:0/4053194020 reap_dead start 2023-11-18T20:44:57.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.876+0000 7f4905294640 1 -- 172.21.15.60:0/4053194020 shutdown_connections 2023-11-18T20:44:57.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.876+0000 7f4905294640 1 -- 172.21.15.60:0/4053194020 >> 172.21.15.60:0/4053194020 conn(0x7f4900072020 msgr2=0x7f4900072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:57.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.876+0000 7f4905294640 1 -- 172.21.15.60:0/4053194020 shutdown_connections 2023-11-18T20:44:57.882 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.876+0000 7f4905294640 1 -- 172.21.15.60:0/4053194020 wait complete. 2023-11-18T20:44:57.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.877+0000 7f4905294640 1 Processor -- start 2023-11-18T20:44:57.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.877+0000 7f4905294640 1 -- start start 2023-11-18T20:44:57.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f4905294640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48e8003b30 0x7f49000c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f4905294640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f490014c280 0x7f4900148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f4905294640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4900148bb0 0x7f4900149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f4905294640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001592d0 con 0x7f4900148bb0 2023-11-18T20:44:57.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f4905294640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49000cb910 con 0x7f490014c280 2023-11-18T20:44:57.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f4905294640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49001586c0 con 0x7f48e8003b30 2023-11-18T20:44:57.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f48ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4900148bb0 0x7f4900149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f48ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4900148bb0 0x7f4900149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39952/0 (socket says 172.21.15.60:39952) 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f48ff7fe640 1 -- 172.21.15.60:0/2912501881 learned_addr learned my addr 172.21.15.60:0/2912501881 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f48ff7fe640 1 -- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48e8003b30 msgr2=0x7f49000c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f48feffd640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48e8003b30 0x7f49000c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f48fe7fc640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f490014c280 0x7f4900148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f48ff7fe640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48e8003b30 0x7f49000c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f48ff7fe640 1 -- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f490014c280 msgr2=0x7f4900148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f48ff7fe640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f490014c280 0x7f4900148670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.878+0000 7f48ff7fe640 1 -- 172.21.15.60:0/2912501881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49001589a0 con 0x7f4900148bb0 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.879+0000 7f48ff7fe640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4900148bb0 0x7f4900149060 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f48e8004740 tx=0x7f48f0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.879+0000 7f48e77fe640 1 -- 172.21.15.60:0/2912501881 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f001b070 con 0x7f4900148bb0 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.879+0000 7f48e77fe640 1 -- 172.21.15.60:0/2912501881 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f000f040 con 0x7f4900148bb0 2023-11-18T20:44:57.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.879+0000 7f4905294640 1 -- 172.21.15.60:0/2912501881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4900158c80 con 0x7f4900148bb0 2023-11-18T20:44:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.879+0000 7f48e77fe640 1 -- 172.21.15.60:0/2912501881 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000e050 con 0x7f4900148bb0 2023-11-18T20:44:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.879+0000 7f4905294640 1 -- 172.21.15.60:0/2912501881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49000c6200 con 0x7f4900148bb0 2023-11-18T20:44:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.880+0000 7f4905294640 1 -- 172.21.15.60:0/2912501881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4900069400 con 0x7f4900148bb0 2023-11-18T20:44:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.880+0000 7f48e77fe640 1 -- 172.21.15.60:0/2912501881 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f48f0015440 con 0x7f4900148bb0 2023-11-18T20:44:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.881+0000 7f48e77fe640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f48ec06ae70 0x7f48ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.881+0000 7f48e77fe640 1 -- 172.21.15.60:0/2912501881 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48f0089ab0 con 0x7f4900148bb0 2023-11-18T20:44:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.881+0000 7f48feffd640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f48ec06ae70 0x7f48ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.881+0000 7f48feffd640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f48ec06ae70 0x7f48ec06d330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f48f8034000 tx=0x7f48f8007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:57.888 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:57.885+0000 7f48e77fe640 1 -- 172.21.15.60:0/2912501881 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48f0054b60 con 0x7f4900148bb0 2023-11-18T20:44:58.066 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.064+0000 7f4905294640 1 -- 172.21.15.60:0/2912501881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f490015cbc0 con 0x7f4900148bb0 2023-11-18T20:44:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.064+0000 7f48e77fe640 1 -- 172.21.15.60:0/2912501881 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48f0058180 con 0x7f4900148bb0 2023-11-18T20:44:58.067 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:44:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 -- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f48ec06ae70 msgr2=0x7f48ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f48ec06ae70 0x7f48ec06d330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f48f8034000 tx=0x7f48f8007000 comp rx=0 tx=0).stop 2023-11-18T20:44:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 -- 172.21.15.60:0/2912501881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4900148bb0 msgr2=0x7f4900149060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4900148bb0 0x7f4900149060 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f48e8004740 tx=0x7f48f0009f90 comp rx=0 tx=0).stop 2023-11-18T20:44:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 -- 172.21.15.60:0/2912501881 shutdown_connections 2023-11-18T20:44:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f48ec06ae70 0x7f48ec06d330 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4900148bb0 0x7f4900149060 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f490014c280 0x7f4900148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 --2- 172.21.15.60:0/2912501881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48e8003b30 0x7f49000c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 -- 172.21.15.60:0/2912501881 >> 172.21.15.60:0/2912501881 conn(0x7f4900072020 msgr2=0x7f490015e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:58.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 -- 172.21.15.60:0/2912501881 shutdown_connections 2023-11-18T20:44:58.070 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.066+0000 7f4905294640 1 -- 172.21.15.60:0/2912501881 wait complete. 2023-11-18T20:44:58.080 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:44:58.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.187+0000 7fe9f04cf640 1 Processor -- start 2023-11-18T20:44:58.189 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.187+0000 7fe9f04cf640 1 -- start start 2023-11-18T20:44:58.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9f04cf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9e8144a00 0x7fe9e806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9f04cf640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe9e8144440 0x7fe9e806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9f04cf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e80b2ef0 con 0x7fe9e81415f0 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9f04cf640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e80b3b70 con 0x7fe9e8144a00 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9f04cf640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e8142510 con 0x7fe9e8144440 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9eda6a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9e8144a00 0x7fe9e806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9eda6a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe9e8144a00 0x7fe9e806b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39968/0 (socket says 172.21.15.60:39968) 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9eda6a640 1 -- 172.21.15.60:0/671098735 learned_addr learned my addr 172.21.15.60:0/671098735 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9eea6c640 1 --2- 172.21.15.60:0/671098735 >> v2:172.21.15.100:3300/0 conn(0x7fe9e8144440 0x7fe9e806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9eda6a640 1 -- 172.21.15.60:0/671098735 >> v2:172.21.15.100:3300/0 conn(0x7fe9e8144440 msgr2=0x7fe9e806ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9eda6a640 1 --2- 172.21.15.60:0/671098735 >> v2:172.21.15.100:3300/0 conn(0x7fe9e8144440 0x7fe9e806ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9eda6a640 1 -- 172.21.15.60:0/671098735 >> v1:172.21.15.100:6789/0 conn(0x7fe9e81415f0 legacy=0x7fe9e8141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:58.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.188+0000 7fe9eda6a640 1 -- 172.21.15.60:0/671098735 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e80b31d0 con 0x7fe9e8144a00 2023-11-18T20:44:58.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.189+0000 7fe9eda6a640 1 --2- 172.21.15.60:0/671098735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e8144a00 0x7fe9e806b4e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe9e815ebc0 tx=0x7fe9d802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c52d54bd31953bd server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:58.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.189+0000 7fe9ed269640 1 -- 172.21.15.60:0/671098735 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8030860 con 0x7fe9e8144a00 2023-11-18T20:44:58.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.189+0000 7fe9ed269640 1 -- 172.21.15.60:0/671098735 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d80309c0 con 0x7fe9e8144a00 2023-11-18T20:44:58.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.189+0000 7fe9ed269640 1 -- 172.21.15.60:0/671098735 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8030d50 con 0x7fe9e8144a00 2023-11-18T20:44:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.190+0000 7fe9f04cf640 1 -- 172.21.15.60:0/671098735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e8144a00 msgr2=0x7fe9e806b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.190+0000 7fe9f04cf640 1 --2- 172.21.15.60:0/671098735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e8144a00 0x7fe9e806b4e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe9e815ebc0 tx=0x7fe9d802b9a0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.190+0000 7fe9f04cf640 1 -- 172.21.15.60:0/671098735 shutdown_connections 2023-11-18T20:44:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.190+0000 7fe9f04cf640 1 --2- 172.21.15.60:0/671098735 >> v2:172.21.15.100:3300/0 conn(0x7fe9e8144440 0x7fe9e806ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.190+0000 7fe9f04cf640 1 --2- 172.21.15.60:0/671098735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e8144a00 0x7fe9e806b4e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.190+0000 7fe9f04cf640 1 -- 172.21.15.60:0/671098735 >> 172.21.15.60:0/671098735 conn(0x7fe9e8070e70 msgr2=0x7fe9e8071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.190+0000 7fe9f04cf640 1 -- 172.21.15.60:0/671098735 shutdown_connections 2023-11-18T20:44:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.190+0000 7fe9f04cf640 1 -- 172.21.15.60:0/671098735 wait complete. 2023-11-18T20:44:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.191+0000 7fe9f04cf640 1 Processor -- start 2023-11-18T20:44:58.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.191+0000 7fe9f04cf640 1 -- start start 2023-11-18T20:44:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9f04cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e81415f0 0x7fe9e8140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9f04cf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e8144440 0x7fe9e8158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9f04cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e8144a00 0x7fe9e8158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9f04cf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e81f5020 con 0x7fe9e81415f0 2023-11-18T20:44:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9f04cf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e81f4430 con 0x7fe9e8144a00 2023-11-18T20:44:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9f04cf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e81f4680 con 0x7fe9e8144440 2023-11-18T20:44:58.197 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9ee26b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e81415f0 0x7fe9e8140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9ee26b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e81415f0 0x7fe9e8140220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39978/0 (socket says 172.21.15.60:39978) 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9ee26b640 1 -- 172.21.15.60:0/2852504683 learned_addr learned my addr 172.21.15.60:0/2852504683 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9eea6c640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e8144a00 0x7fe9e8158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9eda6a640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e8144440 0x7fe9e8158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9ee26b640 1 -- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e8144440 msgr2=0x7fe9e8158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9ee26b640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e8144440 0x7fe9e8158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9ee26b640 1 -- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e8144a00 msgr2=0x7fe9e8158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9ee26b640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e8144a00 0x7fe9e8158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9ee26b640 1 -- 172.21.15.60:0/2852504683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d80094d0 con 0x7fe9e81415f0 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9eea6c640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e8144a00 0x7fe9e8158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:58.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9eda6a640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e8144440 0x7fe9e8158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9ee26b640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e81415f0 0x7fe9e8140220 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c450 tx=0x7fe9e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9deffd640 1 -- 172.21.15.60:0/2852504683 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40052d0 con 0x7fe9e81415f0 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9deffd640 1 -- 172.21.15.60:0/2852504683 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e4005430 con 0x7fe9e81415f0 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9deffd640 1 -- 172.21.15.60:0/2852504683 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4012870 con 0x7fe9e81415f0 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9f04cf640 1 -- 172.21.15.60:0/2852504683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e81f4960 con 0x7fe9e81415f0 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.192+0000 7fe9f04cf640 1 -- 172.21.15.60:0/2852504683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e8140760 con 0x7fe9e81415f0 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.193+0000 7fe9f04cf640 1 -- 172.21.15.60:0/2852504683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e8069400 con 0x7fe9e81415f0 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.194+0000 7fe9deffd640 1 -- 172.21.15.60:0/2852504683 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9e40129d0 con 0x7fe9e81415f0 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.194+0000 7fe9deffd640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9cc06ae40 0x7fe9cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.194+0000 7fe9deffd640 1 -- 172.21.15.60:0/2852504683 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9e40890c0 con 0x7fe9e81415f0 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.194+0000 7fe9eda6a640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9cc06ae40 0x7fe9cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.195+0000 7fe9eda6a640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9cc06ae40 0x7fe9cc06d300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8036000 tx=0x7fe9d803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:58.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.199+0000 7fe9deffd640 1 -- 172.21.15.60:0/2852504683 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9e4054200 con 0x7fe9e81415f0 2023-11-18T20:44:58.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.376+0000 7fe9f04cf640 1 -- 172.21.15.60:0/2852504683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9e8066760 con 0x7fe9e81415f0 2023-11-18T20:44:58.379 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.377+0000 7fe9deffd640 1 -- 172.21.15.60:0/2852504683 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9e4057820 con 0x7fe9e81415f0 2023-11-18T20:44:58.379 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:44:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 -- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9cc06ae40 msgr2=0x7fe9cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.381 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9cc06ae40 0x7fe9cc06d300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8036000 tx=0x7fe9d803b040 comp rx=0 tx=0).stop 2023-11-18T20:44:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 -- 172.21.15.60:0/2852504683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e81415f0 msgr2=0x7fe9e8140220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e81415f0 0x7fe9e8140220 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c450 tx=0x7fe9e400c910 comp rx=0 tx=0).stop 2023-11-18T20:44:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 -- 172.21.15.60:0/2852504683 shutdown_connections 2023-11-18T20:44:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe9cc06ae40 0x7fe9cc06d300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9e8144a00 0x7fe9e8158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9e8144440 0x7fe9e8158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 --2- 172.21.15.60:0/2852504683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9e81415f0 0x7fe9e8140220 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 -- 172.21.15.60:0/2852504683 >> 172.21.15.60:0/2852504683 conn(0x7fe9e8070e70 msgr2=0x7fe9e806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 -- 172.21.15.60:0/2852504683 shutdown_connections 2023-11-18T20:44:58.382 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.379+0000 7fe9f04cf640 1 -- 172.21.15.60:0/2852504683 wait complete. 2023-11-18T20:44:58.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:44:58.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.500+0000 7f094ef73640 1 Processor -- start 2023-11-18T20:44:58.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.500+0000 7f094ef73640 1 -- start start 2023-11-18T20:44:58.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.500+0000 7f094ef73640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f094815d330 0x7f09480b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094ef73640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09480d3900 0x7f09480b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094ef73640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09480b2b10 con 0x7f09480d2910 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094ef73640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09480d1590 con 0x7f094815d330 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094ef73640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09480d1870 con 0x7f09480d3900 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094cd0f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f094815d330 0x7f09480b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094cd0f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f094815d330 0x7f09480b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:47378/0 (socket says 172.21.15.60:47378) 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094d510640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09480d3900 0x7f09480b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f0947fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f09480d2910 0x7f09480d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43276/0 (socket says 172.21.15.60:43276) 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094cd0f640 1 -- 172.21.15.60:0/755443749 learned_addr learned my addr 172.21.15.60:0/755443749 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094cd0f640 1 -- 172.21.15.60:0/755443749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09480d3900 msgr2=0x7f09480b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094cd0f640 1 --2- 172.21.15.60:0/755443749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09480d3900 0x7f09480b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094cd0f640 1 -- 172.21.15.60:0/755443749 >> v1:172.21.15.100:6789/0 conn(0x7f09480d2910 legacy=0x7f09480d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f09477fe640 1 -- 172.21.15.60:0/755443749 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36641372 0 0) 0x7f09480b2b10 con 0x7f09480d2910 2023-11-18T20:44:58.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094cd0f640 1 -- 172.21.15.60:0/755443749 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480d1b50 con 0x7f094815d330 2023-11-18T20:44:58.505 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.501+0000 7f094d510640 1 --2- 172.21.15.60:0/755443749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09480d3900 0x7f09480b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:44:58.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.502+0000 7f094cd0f640 1 --2- 172.21.15.60:0/755443749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f094815d330 0x7f09480b8b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f093c009870 tx=0x7f093c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfb25a85ede432ae server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:58.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.502+0000 7f09477fe640 1 -- 172.21.15.60:0/755443749 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c031640 con 0x7f094815d330 2023-11-18T20:44:58.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.502+0000 7f09477fe640 1 -- 172.21.15.60:0/755443749 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093c0317a0 con 0x7f094815d330 2023-11-18T20:44:58.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.502+0000 7f09477fe640 1 -- 172.21.15.60:0/755443749 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c031b30 con 0x7f094815d330 2023-11-18T20:44:58.506 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.503+0000 7f094ef73640 1 -- 172.21.15.60:0/755443749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f094815d330 msgr2=0x7f09480b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.503+0000 7f094ef73640 1 --2- 172.21.15.60:0/755443749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f094815d330 0x7f09480b8b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f093c009870 tx=0x7f093c02b8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.503+0000 7f094ef73640 1 -- 172.21.15.60:0/755443749 shutdown_connections 2023-11-18T20:44:58.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.503+0000 7f094ef73640 1 --2- 172.21.15.60:0/755443749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09480d3900 0x7f09480b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.503+0000 7f094ef73640 1 --2- 172.21.15.60:0/755443749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f094815d330 0x7f09480b8b80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.503+0000 7f094ef73640 1 -- 172.21.15.60:0/755443749 >> 172.21.15.60:0/755443749 conn(0x7f0948072020 msgr2=0x7f0948072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:58.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.503+0000 7f094ef73640 1 -- 172.21.15.60:0/755443749 shutdown_connections 2023-11-18T20:44:58.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.503+0000 7f094ef73640 1 -- 172.21.15.60:0/755443749 wait complete. 2023-11-18T20:44:58.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.504+0000 7f094ef73640 1 Processor -- start 2023-11-18T20:44:58.507 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.504+0000 7f094ef73640 1 -- start start 2023-11-18T20:44:58.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094ef73640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09480d2910 0x7f09480c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094ef73640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09480d3900 0x7f094815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094ef73640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f094815d330 0x7f094815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094ef73640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09480c8480 con 0x7f094815d330 2023-11-18T20:44:58.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094ef73640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09480c75b0 con 0x7f09480d3900 2023-11-18T20:44:58.509 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094ef73640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09480c7800 con 0x7f09480d2910 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094d510640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f094815d330 0x7f094815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094d510640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f094815d330 0x7f094815ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39988/0 (socket says 172.21.15.60:39988) 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094d510640 1 -- 172.21.15.60:0/3967153497 learned_addr learned my addr 172.21.15.60:0/3967153497 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094d510640 1 -- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09480d2910 msgr2=0x7f09480c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094d510640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09480d2910 0x7f09480c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094d510640 1 -- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09480d3900 msgr2=0x7f094815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094d510640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09480d3900 0x7f094815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.505+0000 7f094d510640 1 -- 172.21.15.60:0/3967153497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0940009a70 con 0x7f094815d330 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.506+0000 7f094d510640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f094815d330 0x7f094815ffc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f094000efc0 tx=0x7f0940009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.506+0000 7f09457fa640 1 -- 172.21.15.60:0/3967153497 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000cde0 con 0x7f094815d330 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.506+0000 7f09457fa640 1 -- 172.21.15.60:0/3967153497 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094000f040 con 0x7f094815d330 2023-11-18T20:44:58.510 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.506+0000 7f094ef73640 1 -- 172.21.15.60:0/3967153497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c0094d0 con 0x7f094815d330 2023-11-18T20:44:58.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.506+0000 7f09457fa640 1 -- 172.21.15.60:0/3967153497 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940003bd0 con 0x7f094815d330 2023-11-18T20:44:58.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.506+0000 7f094ef73640 1 -- 172.21.15.60:0/3967153497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09480c7dc0 con 0x7f094815d330 2023-11-18T20:44:58.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.507+0000 7f094ef73640 1 -- 172.21.15.60:0/3967153497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0948069400 con 0x7f094815d330 2023-11-18T20:44:58.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.507+0000 7f09457fa640 1 -- 172.21.15.60:0/3967153497 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0940014480 con 0x7f094815d330 2023-11-18T20:44:58.511 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.508+0000 7f09457fa640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f092c06ae70 0x7f092c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.508+0000 7f09457fa640 1 -- 172.21.15.60:0/3967153497 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f094008a2f0 con 0x7f094815d330 2023-11-18T20:44:58.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.508+0000 7f094cd0f640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f092c06ae70 0x7f092c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.512 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.508+0000 7f094cd0f640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f092c06ae70 0x7f092c06d330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f093c036000 tx=0x7f093c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:58.514 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.512+0000 7f09457fa640 1 -- 172.21.15.60:0/3967153497 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09400553a0 con 0x7f094815d330 2023-11-18T20:44:58.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.686+0000 7f094ef73640 1 -- 172.21.15.60:0/3967153497 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09480b2050 con 0x7f094815d330 2023-11-18T20:44:58.689 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.687+0000 7f09457fa640 1 -- 172.21.15.60:0/3967153497 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09400589c0 con 0x7f094815d330 2023-11-18T20:44:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:44:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 -- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f092c06ae70 msgr2=0x7f092c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f092c06ae70 0x7f092c06d330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f093c036000 tx=0x7f093c030000 comp rx=0 tx=0).stop 2023-11-18T20:44:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 -- 172.21.15.60:0/3967153497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f094815d330 msgr2=0x7f094815ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f094815d330 0x7f094815ffc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f094000efc0 tx=0x7f0940009f90 comp rx=0 tx=0).stop 2023-11-18T20:44:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 -- 172.21.15.60:0/3967153497 shutdown_connections 2023-11-18T20:44:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f092c06ae70 0x7f092c06d330 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f094815d330 0x7f094815ffc0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09480d3900 0x7f094815fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 --2- 172.21.15.60:0/3967153497 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09480d2910 0x7f09480c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 -- 172.21.15.60:0/3967153497 >> 172.21.15.60:0/3967153497 conn(0x7f0948072020 msgr2=0x7f09480d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 -- 172.21.15.60:0/3967153497 shutdown_connections 2023-11-18T20:44:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.689+0000 7f094ef73640 1 -- 172.21.15.60:0/3967153497 wait complete. 2023-11-18T20:44:58.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-18T20:44:58.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.806+0000 7f44f44d4640 1 Processor -- start 2023-11-18T20:44:58.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.806+0000 7f44f44d4640 1 -- start start 2023-11-18T20:44:58.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.807+0000 7f44f44d4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f44ec160fb0 0x7f44ec161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.807+0000 7f44f44d4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec161d20 con 0x7f44ec06ea50 2023-11-18T20:44:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.807+0000 7f44f44d4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec15e760 con 0x7f44ec0d0ed0 2023-11-18T20:44:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.807+0000 7f44f44d4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec15d9a0 con 0x7f44ec160fb0 2023-11-18T20:44:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.807+0000 7f44f1a6f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f44ec160fb0 0x7f44ec161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.807+0000 7f44f1a6f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f44ec160fb0 0x7f44ec161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39994/0 (socket says 172.21.15.60:39994) 2023-11-18T20:44:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.807+0000 7f44f1a6f640 1 -- 172.21.15.60:0/3447506823 learned_addr learned my addr 172.21.15.60:0/3447506823 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.808+0000 7f44f1a6f640 1 -- 172.21.15.60:0/3447506823 >> v1:172.21.15.100:6789/0 conn(0x7f44ec0d0ed0 legacy=0x7f44ec0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.808+0000 7f44f1a6f640 1 -- 172.21.15.60:0/3447506823 >> v1:172.21.15.60:6789/0 conn(0x7f44ec06ea50 legacy=0x7f44ec06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.808+0000 7f44f1a6f640 1 -- 172.21.15.60:0/3447506823 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec15dc80 con 0x7f44ec160fb0 2023-11-18T20:44:58.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.808+0000 7f44f1a6f640 1 --2- 172.21.15.60:0/3447506823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec160fb0 0x7f44ec161440 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009870 tx=0x7f44e002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b24c51d2980771e server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.808+0000 7f44f126e640 1 -- 172.21.15.60:0/3447506823 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0031860 con 0x7f44ec160fb0 2023-11-18T20:44:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.808+0000 7f44f126e640 1 -- 172.21.15.60:0/3447506823 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e00319c0 con 0x7f44ec160fb0 2023-11-18T20:44:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.808+0000 7f44f126e640 1 -- 172.21.15.60:0/3447506823 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0031d50 con 0x7f44ec160fb0 2023-11-18T20:44:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.809+0000 7f44f44d4640 1 -- 172.21.15.60:0/3447506823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec160fb0 msgr2=0x7f44ec161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.809+0000 7f44f44d4640 1 --2- 172.21.15.60:0/3447506823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec160fb0 0x7f44ec161440 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009870 tx=0x7f44e002b900 comp rx=0 tx=0).stop 2023-11-18T20:44:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.809+0000 7f44f44d4640 1 -- 172.21.15.60:0/3447506823 shutdown_connections 2023-11-18T20:44:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.809+0000 7f44f44d4640 1 --2- 172.21.15.60:0/3447506823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec160fb0 0x7f44ec161440 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.809+0000 7f44f44d4640 1 -- 172.21.15.60:0/3447506823 >> 172.21.15.60:0/3447506823 conn(0x7f44ec06a6b0 msgr2=0x7f44ec06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.809+0000 7f44f44d4640 1 -- 172.21.15.60:0/3447506823 shutdown_connections 2023-11-18T20:44:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.809+0000 7f44f44d4640 1 -- 172.21.15.60:0/3447506823 wait complete. 2023-11-18T20:44:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.810+0000 7f44f44d4640 1 Processor -- start 2023-11-18T20:44:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.810+0000 7f44f44d4640 1 -- start start 2023-11-18T20:44:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f44d4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec06ea50 0x7f44ec157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f44d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44ec0d0ed0 0x7f44ec156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f44d4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44ec160fb0 0x7f44ec0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f44d4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec156cc0 con 0x7f44ec06ea50 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f44d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec1e75e0 con 0x7f44ec0d0ed0 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f44d4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec1e6cb0 con 0x7f44ec160fb0 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f2270640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec06ea50 0x7f44ec157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f2270640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec06ea50 0x7f44ec157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40004/0 (socket says 172.21.15.60:40004) 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f2270640 1 -- 172.21.15.60:0/1064452719 learned_addr learned my addr 172.21.15.60:0/1064452719 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f1a6f640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44ec0d0ed0 0x7f44ec156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f2270640 1 -- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44ec160fb0 msgr2=0x7f44ec0b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f2270640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44ec160fb0 0x7f44ec0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f2270640 1 -- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44ec0d0ed0 msgr2=0x7f44ec156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f2270640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44ec0d0ed0 0x7f44ec156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.811+0000 7f44f2270640 1 -- 172.21.15.60:0/1064452719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e00094d0 con 0x7f44ec06ea50 2023-11-18T20:44:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.812+0000 7f44f2270640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec06ea50 0x7f44ec157ec0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c400 tx=0x7f44e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.812+0000 7f44eaffd640 1 -- 172.21.15.60:0/1064452719 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e40052d0 con 0x7f44ec06ea50 2023-11-18T20:44:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.812+0000 7f44eaffd640 1 -- 172.21.15.60:0/1064452719 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4005430 con 0x7f44ec06ea50 2023-11-18T20:44:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.812+0000 7f44f44d4640 1 -- 172.21.15.60:0/1064452719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec1e6f90 con 0x7f44ec06ea50 2023-11-18T20:44:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.812+0000 7f44eaffd640 1 -- 172.21.15.60:0/1064452719 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4003830 con 0x7f44ec06ea50 2023-11-18T20:44:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.812+0000 7f44f44d4640 1 -- 172.21.15.60:0/1064452719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec13db40 con 0x7f44ec06ea50 2023-11-18T20:44:58.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.813+0000 7f44f44d4640 1 -- 172.21.15.60:0/1064452719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec069400 con 0x7f44ec06ea50 2023-11-18T20:44:58.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.813+0000 7f44eaffd640 1 -- 172.21.15.60:0/1064452719 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44e4003990 con 0x7f44ec06ea50 2023-11-18T20:44:58.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.814+0000 7f44eaffd640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f44d806ae40 0x7f44d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:58.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.814+0000 7f44eaffd640 1 -- 172.21.15.60:0/1064452719 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44e4088b70 con 0x7f44ec06ea50 2023-11-18T20:44:58.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.814+0000 7f44f1a6f640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f44d806ae40 0x7f44d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:58.817 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.814+0000 7f44f1a6f640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f44d806ae40 0x7f44d806d300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f44e0036000 tx=0x7f44e0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:58.820 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:58.818+0000 7f44eaffd640 1 -- 172.21.15.60:0/1064452719 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44e4053c20 con 0x7f44ec06ea50 2023-11-18T20:44:59.011 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.009+0000 7f44f44d4640 1 -- 172.21.15.60:0/1064452719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f44ec0b0880 con 0x7f44ec06ea50 2023-11-18T20:44:59.012 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.010+0000 7f44eaffd640 1 -- 172.21.15.60:0/1064452719 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7f44e4017080 con 0x7f44ec06ea50 2023-11-18T20:44:59.012 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2023-11-18T20:44:59.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 -- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f44d806ae40 msgr2=0x7f44d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f44d806ae40 0x7f44d806d300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f44e0036000 tx=0x7f44e0008030 comp rx=0 tx=0).stop 2023-11-18T20:44:59.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 -- 172.21.15.60:0/1064452719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec06ea50 msgr2=0x7f44ec157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.014 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec06ea50 0x7f44ec157ec0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c400 tx=0x7f44e400c8c0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 -- 172.21.15.60:0/1064452719 shutdown_connections 2023-11-18T20:44:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f44d806ae40 0x7f44d806d300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44ec160fb0 0x7f44ec0b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44ec0d0ed0 0x7f44ec156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 --2- 172.21.15.60:0/1064452719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44ec06ea50 0x7f44ec157ec0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 -- 172.21.15.60:0/1064452719 >> 172.21.15.60:0/1064452719 conn(0x7f44ec06a6b0 msgr2=0x7f44ec0d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 -- 172.21.15.60:0/1064452719 shutdown_connections 2023-11-18T20:44:59.015 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.012+0000 7f44f44d4640 1 -- 172.21.15.60:0/1064452719 wait complete. 2023-11-18T20:44:59.024 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2023-11-18T20:44:59.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:59.025 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-18T20:44:59.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi060.stderr:ceph-fuse[108563]: fuse finished with error 0 and tester_r 0 2023-11-18T20:44:59.055 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:44:59.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:59.056 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-18T20:44:59.162 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:44:59.163 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:59.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056339,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=480 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-18T20:44:59.181 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:44:59.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:59.181 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-18T20:44:59.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:44:59.239 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-18T20:44:59.239 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:44:59.239 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:59.255 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=344 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249488k,nr_inodes=812372,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-18T20:44:59.255 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:44:59.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:59.255 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-18T20:44:59.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:44:59.311 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-18T20:44:59.311 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:44:59.312 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:44:59.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056339,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=480 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-18T20:44:59.329 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:44:59.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:44:59.329 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-18T20:44:59.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:44:59.385 INFO:teuthology.orchestra.run.smithi100.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-18T20:44:59.386 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:44:59.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.490+0000 7fd24ab33640 1 Processor -- start 2023-11-18T20:44:59.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.490+0000 7fd24ab33640 1 -- start start 2023-11-18T20:44:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd24ab33640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd244144980 0x7fd24415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd24ab33640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd2440b4370 0x7fd2440b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd24ab33640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd244071770 con 0x7fd2440ceb70 2023-11-18T20:44:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd24ab33640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2440ccb60 con 0x7fd2440b4370 2023-11-18T20:44:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd24ab33640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2440b4ce0 con 0x7fd244144980 2023-11-18T20:44:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd23bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd2440ceb70 0x7fd2440cc450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56086/0 (socket says 172.21.15.60:56086) 2023-11-18T20:44:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd23bfff640 1 -- 172.21.15.60:0/33500752 learned_addr learned my addr 172.21.15.60:0/33500752 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:44:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd2490d0640 1 --2- 172.21.15.60:0/33500752 >> v2:172.21.15.60:3300/0 conn(0x7fd2440b4370 0x7fd2440b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd2488cf640 1 --2- 172.21.15.60:0/33500752 >> v2:172.21.15.100:3300/0 conn(0x7fd244144980 0x7fd24415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd23b7fe640 1 -- 172.21.15.60:0/33500752 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121330328 0 0) 0x7fd244071770 con 0x7fd2440ceb70 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd23b7fe640 1 -- 172.21.15.60:0/33500752 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c003520 con 0x7fd2440ceb70 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd2490d0640 1 -- 172.21.15.60:0/33500752 >> v2:172.21.15.100:3300/0 conn(0x7fd244144980 msgr2=0x7fd24415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd2490d0640 1 --2- 172.21.15.60:0/33500752 >> v2:172.21.15.100:3300/0 conn(0x7fd244144980 0x7fd24415c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd2490d0640 1 -- 172.21.15.60:0/33500752 >> v1:172.21.15.60:6789/0 conn(0x7fd2440ceb70 legacy=0x7fd2440cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.491+0000 7fd2490d0640 1 -- 172.21.15.60:0/33500752 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440ccdc0 con 0x7fd2440b4370 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.492+0000 7fd2490d0640 1 --2- 172.21.15.60:0/33500752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440b4370 0x7fd2440b47a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd2440b4ec0 tx=0x7fd23c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1889e732292c06ec server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.492+0000 7fd23b7fe640 1 -- 172.21.15.60:0/33500752 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c02cc80 con 0x7fd2440b4370 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.492+0000 7fd23b7fe640 1 -- 172.21.15.60:0/33500752 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c02cde0 con 0x7fd2440b4370 2023-11-18T20:44:59.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.492+0000 7fd23b7fe640 1 -- 172.21.15.60:0/33500752 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c034640 con 0x7fd2440b4370 2023-11-18T20:44:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.493+0000 7fd24ab33640 1 -- 172.21.15.60:0/33500752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440b4370 msgr2=0x7fd2440b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.493+0000 7fd24ab33640 1 --2- 172.21.15.60:0/33500752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440b4370 0x7fd2440b47a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd2440b4ec0 tx=0x7fd23c02bed0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.493+0000 7fd24ab33640 1 -- 172.21.15.60:0/33500752 shutdown_connections 2023-11-18T20:44:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.493+0000 7fd24ab33640 1 --2- 172.21.15.60:0/33500752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440b4370 0x7fd2440b47a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.493+0000 7fd24ab33640 1 --2- 172.21.15.60:0/33500752 >> v2:172.21.15.100:3300/0 conn(0x7fd244144980 0x7fd24415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.493+0000 7fd24ab33640 1 -- 172.21.15.60:0/33500752 >> 172.21.15.60:0/33500752 conn(0x7fd24406a4d0 msgr2=0x7fd24406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.493+0000 7fd24ab33640 1 -- 172.21.15.60:0/33500752 shutdown_connections 2023-11-18T20:44:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.493+0000 7fd24ab33640 1 -- 172.21.15.60:0/33500752 wait complete. 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.494+0000 7fd24ab33640 1 Processor -- start 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.494+0000 7fd24ab33640 1 -- start start 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd24ab33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2440b4370 0x7fd244159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd24ab33640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440ceb70 0x7fd24415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd24ab33640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd244144980 0x7fd2440c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd24ab33640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440a85e0 con 0x7fd2440ceb70 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd24ab33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244133830 con 0x7fd2440b4370 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd24ab33640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440ca190 con 0x7fd244144980 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440ceb70 0x7fd24415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd23bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440ceb70 0x7fd24415a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40008/0 (socket says 172.21.15.60:40008) 2023-11-18T20:44:59.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd23bfff640 1 -- 172.21.15.60:0/4137443894 learned_addr learned my addr 172.21.15.60:0/4137443894 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd2488cf640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2440b4370 0x7fd244159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd2490d0640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd244144980 0x7fd2440c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd23bfff640 1 -- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd244144980 msgr2=0x7fd2440c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd23bfff640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd244144980 0x7fd2440c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd23bfff640 1 -- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2440b4370 msgr2=0x7fd244159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd23bfff640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2440b4370 0x7fd244159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd23bfff640 1 -- 172.21.15.60:0/4137443894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c009a70 con 0x7fd2440ceb70 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd23bfff640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440ceb70 0x7fd24415a470 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c430 tx=0x7fd23400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.495+0000 7fd2397fa640 1 -- 172.21.15.60:0/4137443894 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234005ce0 con 0x7fd2440ceb70 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.496+0000 7fd2397fa640 1 -- 172.21.15.60:0/4137443894 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd234005e40 con 0x7fd2440ceb70 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.496+0000 7fd24ab33640 1 -- 172.21.15.60:0/4137443894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2440c9bd0 con 0x7fd2440ceb70 2023-11-18T20:44:59.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.496+0000 7fd2397fa640 1 -- 172.21.15.60:0/4137443894 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340072f0 con 0x7fd2440ceb70 2023-11-18T20:44:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.496+0000 7fd24ab33640 1 -- 172.21.15.60:0/4137443894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2441334c0 con 0x7fd2440ceb70 2023-11-18T20:44:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.496+0000 7fd24ab33640 1 -- 172.21.15.60:0/4137443894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24415b150 con 0x7fd2440ceb70 2023-11-18T20:44:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.497+0000 7fd2397fa640 1 -- 172.21.15.60:0/4137443894 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd234007450 con 0x7fd2440ceb70 2023-11-18T20:44:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.497+0000 7fd2397fa640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd23006ae40 0x7fd23006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.497+0000 7fd2397fa640 1 -- 172.21.15.60:0/4137443894 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd234088d60 con 0x7fd2440ceb70 2023-11-18T20:44:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.498+0000 7fd2488cf640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd23006ae40 0x7fd23006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.498+0000 7fd2488cf640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd23006ae40 0x7fd23006d300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd240006fd0 tx=0x7fd240008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.503+0000 7fd2397fa640 1 -- 172.21.15.60:0/4137443894 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd234053d00 con 0x7fd2440ceb70 2023-11-18T20:44:59.715 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.713+0000 7fd24ab33640 1 -- 172.21.15.60:0/4137443894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd244069400 con 0x7fd2440ceb70 2023-11-18T20:44:59.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.714+0000 7fd2397fa640 1 -- 172.21.15.60:0/4137443894 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd234017090 con 0x7fd2440ceb70 2023-11-18T20:44:59.716 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:44:59.717 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1578336464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1578336464},{"type":"v1","addr":"172.21.15.100:6836","nonce":1578336464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/88984334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":88984334},{"type":"v1","addr":"172.21.15.100:6837","nonce":88984334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3777286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3777286992},{"type":"v1","addr":"172.21.15.60:6833","nonce":3777286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:44:33.070091+0000","modified":"2023-11-18T20:44:35.050014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3098423505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3098423505},{"type":"v1","addr":"172.21.15.60:6835","nonce":3098423505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-18T20:44:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2023-11-18T20:44:59.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.715+0000 7fd24ab33640 1 -- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd23006ae40 msgr2=0x7fd23006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.715+0000 7fd24ab33640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd23006ae40 0x7fd23006d300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd240006fd0 tx=0x7fd240008040 comp rx=0 tx=0).stop 2023-11-18T20:44:59.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.715+0000 7fd24ab33640 1 -- 172.21.15.60:0/4137443894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440ceb70 msgr2=0x7fd24415a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.715+0000 7fd24ab33640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440ceb70 0x7fd24415a470 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c430 tx=0x7fd23400c8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.715+0000 7fd24ab33640 1 -- 172.21.15.60:0/4137443894 shutdown_connections 2023-11-18T20:44:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.715+0000 7fd24ab33640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd23006ae40 0x7fd23006d300 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.715+0000 7fd24ab33640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd244144980 0x7fd2440c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.715+0000 7fd24ab33640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2440ceb70 0x7fd24415a470 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.715+0000 7fd24ab33640 1 --2- 172.21.15.60:0/4137443894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2440b4370 0x7fd244159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.715+0000 7fd24ab33640 1 -- 172.21.15.60:0/4137443894 >> 172.21.15.60:0/4137443894 conn(0x7fd24406a4d0 msgr2=0x7fd24406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.716+0000 7fd24ab33640 1 -- 172.21.15.60:0/4137443894 shutdown_connections 2023-11-18T20:44:59.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.716+0000 7fd24ab33640 1 -- 172.21.15.60:0/4137443894 wait complete. 2023-11-18T20:44:59.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:44:59.835 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.833+0000 7ff881f47640 1 Processor -- start 2023-11-18T20:44:59.836 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.833+0000 7ff881f47640 1 -- start start 2023-11-18T20:44:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff881f47640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff87c06abe0 0x7ff87c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff881f47640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87c1409d0 0x7ff87c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff881f47640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c1406f0 con 0x7ff87c140310 2023-11-18T20:44:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff881f47640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c161e20 con 0x7ff87c06abe0 2023-11-18T20:44:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff881f47640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c143960 con 0x7ff87c1409d0 2023-11-18T20:44:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff87affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff87c06abe0 0x7ff87c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff87affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff87c06abe0 0x7ff87c1618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40012/0 (socket says 172.21.15.60:40012) 2023-11-18T20:44:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff87b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff87c140310 0x7ff87c06a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56090/0 (socket says 172.21.15.60:56090) 2023-11-18T20:44:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff87affd640 1 -- 172.21.15.60:0/2132833230 learned_addr learned my addr 172.21.15.60:0/2132833230 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff87affd640 1 -- 172.21.15.60:0/2132833230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87c1409d0 msgr2=0x7ff87c06efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:44:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff87bfff640 1 --2- 172.21.15.60:0/2132833230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87c1409d0 0x7ff87c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff87affd640 1 --2- 172.21.15.60:0/2132833230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87c1409d0 0x7ff87c06efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.835+0000 7ff87affd640 1 -- 172.21.15.60:0/2132833230 >> v1:172.21.15.60:6789/0 conn(0x7ff87c140310 legacy=0x7ff87c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.834+0000 7ff87a7fc640 1 -- 172.21.15.60:0/2132833230 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064645894 0 0) 0x7ff87c1406f0 con 0x7ff87c140310 2023-11-18T20:44:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.835+0000 7ff87affd640 1 -- 172.21.15.60:0/2132833230 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c142de0 con 0x7ff87c06abe0 2023-11-18T20:44:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.835+0000 7ff87affd640 1 --2- 172.21.15.60:0/2132833230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c06abe0 0x7ff87c1618e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff870009870 tx=0x7ff87002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc2dd5ee5a0dc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.835+0000 7ff87a7fc640 1 -- 172.21.15.60:0/2132833230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870031640 con 0x7ff87c06abe0 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.835+0000 7ff87a7fc640 1 -- 172.21.15.60:0/2132833230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8700317a0 con 0x7ff87c06abe0 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.835+0000 7ff87a7fc640 1 -- 172.21.15.60:0/2132833230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870031b30 con 0x7ff87c06abe0 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.836+0000 7ff881f47640 1 -- 172.21.15.60:0/2132833230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c06abe0 msgr2=0x7ff87c1618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.836+0000 7ff881f47640 1 --2- 172.21.15.60:0/2132833230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c06abe0 0x7ff87c1618e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff870009870 tx=0x7ff87002b8f0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.836+0000 7ff881f47640 1 -- 172.21.15.60:0/2132833230 shutdown_connections 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.836+0000 7ff881f47640 1 --2- 172.21.15.60:0/2132833230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87c1409d0 0x7ff87c06efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.836+0000 7ff881f47640 1 --2- 172.21.15.60:0/2132833230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c06abe0 0x7ff87c1618e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.836+0000 7ff881f47640 1 -- 172.21.15.60:0/2132833230 >> 172.21.15.60:0/2132833230 conn(0x7ff87c0b1e50 msgr2=0x7ff87c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.836+0000 7ff881f47640 1 -- 172.21.15.60:0/2132833230 shutdown_connections 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.836+0000 7ff881f47640 1 -- 172.21.15.60:0/2132833230 wait complete. 2023-11-18T20:44:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.837+0000 7ff881f47640 1 Processor -- start 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.837+0000 7ff881f47640 1 -- start start 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff881f47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff87c06abe0 0x7ff87c146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff881f47640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87c140310 0x7ff87c13f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff881f47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c1409d0 0x7ff87c156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff881f47640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c13fea0 con 0x7ff87c1409d0 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff881f47640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c1572d0 con 0x7ff87c06abe0 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff881f47640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c157860 con 0x7ff87c140310 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff87bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c1409d0 0x7ff87c156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff87bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c1409d0 0x7ff87c156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40018/0 (socket says 172.21.15.60:40018) 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff87bfff640 1 -- 172.21.15.60:0/3460391365 learned_addr learned my addr 172.21.15.60:0/3460391365 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:44:59.843 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff87bfff640 1 -- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87c140310 msgr2=0x7ff87c13f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff87affd640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87c140310 0x7ff87c13f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff87b7fe640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff87c06abe0 0x7ff87c146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff87bfff640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87c140310 0x7ff87c13f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff87bfff640 1 -- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff87c06abe0 msgr2=0x7ff87c146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff87bfff640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff87c06abe0 0x7ff87c146540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.838+0000 7ff87bfff640 1 -- 172.21.15.60:0/3460391365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8700094d0 con 0x7ff87c1409d0 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.839+0000 7ff87bfff640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c1409d0 0x7ff87c156d90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff87400cd70 tx=0x7ff87400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.839+0000 7ff863fff640 1 -- 172.21.15.60:0/3460391365 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874003b90 con 0x7ff87c1409d0 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.839+0000 7ff863fff640 1 -- 172.21.15.60:0/3460391365 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff874003cf0 con 0x7ff87c1409d0 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.839+0000 7ff881f47640 1 -- 172.21.15.60:0/3460391365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c157580 con 0x7ff87c1409d0 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.839+0000 7ff863fff640 1 -- 172.21.15.60:0/3460391365 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874004240 con 0x7ff87c1409d0 2023-11-18T20:44:59.844 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.839+0000 7ff881f47640 1 -- 172.21.15.60:0/3460391365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c143eb0 con 0x7ff87c1409d0 2023-11-18T20:44:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.840+0000 7ff881f47640 1 -- 172.21.15.60:0/3460391365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87c069400 con 0x7ff87c1409d0 2023-11-18T20:44:59.845 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.840+0000 7ff863fff640 1 -- 172.21.15.60:0/3460391365 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff8740043a0 con 0x7ff87c1409d0 2023-11-18T20:44:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.841+0000 7ff863fff640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86806ae70 0x7ff86806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:44:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.841+0000 7ff863fff640 1 -- 172.21.15.60:0/3460391365 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff874012070 con 0x7ff87c1409d0 2023-11-18T20:44:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.841+0000 7ff87b7fe640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86806ae70 0x7ff86806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:44:59.846 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.841+0000 7ff87b7fe640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86806ae70 0x7ff86806d330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff86c000f60 tx=0x7ff86c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:44:59.847 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:44:59.845+0000 7ff863fff640 1 -- 172.21.15.60:0/3460391365 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff874055080 con 0x7ff87c1409d0 2023-11-18T20:45:00.023 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.021+0000 7ff881f47640 1 -- 172.21.15.60:0/3460391365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff87c06ea60 con 0x7ff87c1409d0 2023-11-18T20:45:00.024 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.022+0000 7ff863fff640 1 -- 172.21.15.60:0/3460391365 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16557 (secure 0 0 0) 0x7ff8740586a0 con 0x7ff87c1409d0 2023-11-18T20:45:00.025 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:00.025 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:34.103787+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:45:00.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 -- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86806ae70 msgr2=0x7ff86806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86806ae70 0x7ff86806d330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff86c000f60 tx=0x7ff86c009040 comp rx=0 tx=0).stop 2023-11-18T20:45:00.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 -- 172.21.15.60:0/3460391365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c1409d0 msgr2=0x7ff87c156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c1409d0 0x7ff87c156d90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff87400cd70 tx=0x7ff87400a660 comp rx=0 tx=0).stop 2023-11-18T20:45:00.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 -- 172.21.15.60:0/3460391365 shutdown_connections 2023-11-18T20:45:00.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff86806ae70 0x7ff86806d330 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff87c1409d0 0x7ff87c156d90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.027 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff87c140310 0x7ff87c13f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 --2- 172.21.15.60:0/3460391365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff87c06abe0 0x7ff87c146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 -- 172.21.15.60:0/3460391365 >> 172.21.15.60:0/3460391365 conn(0x7ff87c0b1e50 msgr2=0x7ff87c06f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:00.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 -- 172.21.15.60:0/3460391365 shutdown_connections 2023-11-18T20:45:00.028 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.025+0000 7ff881f47640 1 -- 172.21.15.60:0/3460391365 wait complete. 2023-11-18T20:45:00.038 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-18T20:42:25.491946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-18T20:44:31.358325+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-18T20:44:32.036538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-18T20:45:00.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-18T20:45:00.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.146+0000 7f0869133640 1 Processor -- start 2023-11-18T20:45:00.147 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.146+0000 7f0869133640 1 -- start start 2023-11-18T20:45:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0869133640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 0x7f086406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0869133640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0864160fb0 0x7f0864161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0869133640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08640b3ff0 0x7f08640b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0869133640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0864161d20 con 0x7f08640b3ff0 2023-11-18T20:45:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0869133640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f086406f370 con 0x7f0864160fb0 2023-11-18T20:45:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0869133640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086415e760 con 0x7f086406ea50 2023-11-18T20:45:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086359e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08640b3ff0 0x7f08640b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086359e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08640b3ff0 0x7f08640b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40024/0 (socket says 172.21.15.60:40024) 2023-11-18T20:45:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086359e640 1 -- 172.21.15.60:0/2162305025 learned_addr learned my addr 172.21.15.60:0/2162305025 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:00.149 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0862d9d640 1 --2- 172.21.15.60:0/2162305025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 0x7f086406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086259c640 1 --2- 172.21.15.60:0/2162305025 >> v2:172.21.15.100:3300/0 conn(0x7f0864160fb0 0x7f0864161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086359e640 1 -- 172.21.15.60:0/2162305025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 msgr2=0x7f086406ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:00.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086359e640 1 --2- 172.21.15.60:0/2162305025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 0x7f086406ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086359e640 1 -- 172.21.15.60:0/2162305025 >> v2:172.21.15.100:3300/0 conn(0x7f0864160fb0 msgr2=0x7f0864161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:00.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086359e640 1 --2- 172.21.15.60:0/2162305025 >> v2:172.21.15.100:3300/0 conn(0x7f0864160fb0 0x7f0864161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086359e640 1 -- 172.21.15.60:0/2162305025 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086415d9a0 con 0x7f08640b3ff0 2023-11-18T20:45:00.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0862d9d640 1 --2- 172.21.15.60:0/2162305025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 0x7f086406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:45:00.150 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086259c640 1 --2- 172.21.15.60:0/2162305025 >> v2:172.21.15.100:3300/0 conn(0x7f0864160fb0 0x7f0864161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:45:00.151 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f086359e640 1 --2- 172.21.15.60:0/2162305025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08640b3ff0 0x7f08640b4460 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f08640d2120 tx=0x7f085402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd52238361edf19c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0861d9b640 1 -- 172.21.15.60:0/2162305025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085402c9e0 con 0x7f08640b3ff0 2023-11-18T20:45:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0861d9b640 1 -- 172.21.15.60:0/2162305025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085402cb40 con 0x7f08640b3ff0 2023-11-18T20:45:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.147+0000 7f0861d9b640 1 -- 172.21.15.60:0/2162305025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08540343f0 con 0x7f08640b3ff0 2023-11-18T20:45:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.148+0000 7f0869133640 1 -- 172.21.15.60:0/2162305025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08640b3ff0 msgr2=0x7f08640b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.148+0000 7f0869133640 1 --2- 172.21.15.60:0/2162305025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08640b3ff0 0x7f08640b4460 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f08640d2120 tx=0x7f085402be40 comp rx=0 tx=0).stop 2023-11-18T20:45:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.148+0000 7f0869133640 1 -- 172.21.15.60:0/2162305025 shutdown_connections 2023-11-18T20:45:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.148+0000 7f0869133640 1 --2- 172.21.15.60:0/2162305025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08640b3ff0 0x7f08640b4460 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.148+0000 7f0869133640 1 --2- 172.21.15.60:0/2162305025 >> v2:172.21.15.100:3300/0 conn(0x7f0864160fb0 0x7f0864161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.148+0000 7f0869133640 1 --2- 172.21.15.60:0/2162305025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 0x7f086406ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.148+0000 7f0869133640 1 -- 172.21.15.60:0/2162305025 >> 172.21.15.60:0/2162305025 conn(0x7f086406a6b0 msgr2=0x7f086406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:00.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.148+0000 7f0869133640 1 -- 172.21.15.60:0/2162305025 shutdown_connections 2023-11-18T20:45:00.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.148+0000 7f0869133640 1 -- 172.21.15.60:0/2162305025 wait complete. 2023-11-18T20:45:00.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.149+0000 7f0869133640 1 Processor -- start 2023-11-18T20:45:00.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.149+0000 7f0869133640 1 -- start start 2023-11-18T20:45:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f0869133640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 0x7f086413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f0869133640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08640b3ff0 0x7f086406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f0869133640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0864160fb0 0x7f086406cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f0869133640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641f6fc0 con 0x7f0864160fb0 2023-11-18T20:45:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f0869133640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086413d230 con 0x7f08640b3ff0 2023-11-18T20:45:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f0869133640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08641f6340 con 0x7f086406ea50 2023-11-18T20:45:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f086359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0864160fb0 0x7f086406cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f086359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0864160fb0 0x7f086406cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40034/0 (socket says 172.21.15.60:40034) 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f086359e640 1 -- 172.21.15.60:0/1787239854 learned_addr learned my addr 172.21.15.60:0/1787239854 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f0862d9d640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 0x7f086413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f086259c640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08640b3ff0 0x7f086406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f086359e640 1 -- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 msgr2=0x7f086413ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f086359e640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 0x7f086413ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.150+0000 7f086359e640 1 -- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08640b3ff0 msgr2=0x7f086406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.151+0000 7f086359e640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08640b3ff0 0x7f086406c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.151+0000 7f086359e640 1 -- 172.21.15.60:0/1787239854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0854009b30 con 0x7f0864160fb0 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.151+0000 7f086259c640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08640b3ff0 0x7f086406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.151+0000 7f086359e640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0864160fb0 0x7f086406cb70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0854009f40 tx=0x7f0854004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.151+0000 7f085b7fe640 1 -- 172.21.15.60:0/1787239854 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854009510 con 0x7f0864160fb0 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.151+0000 7f085b7fe640 1 -- 172.21.15.60:0/1787239854 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085403e450 con 0x7f0864160fb0 2023-11-18T20:45:00.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.151+0000 7f085b7fe640 1 -- 172.21.15.60:0/1787239854 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085403e800 con 0x7f0864160fb0 2023-11-18T20:45:00.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.151+0000 7f0869133640 1 -- 172.21.15.60:0/1787239854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08641f6620 con 0x7f0864160fb0 2023-11-18T20:45:00.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.151+0000 7f0869133640 1 -- 172.21.15.60:0/1787239854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08641f6be0 con 0x7f0864160fb0 2023-11-18T20:45:00.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.152+0000 7f0869133640 1 -- 172.21.15.60:0/1787239854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08641f7100 con 0x7f0864160fb0 2023-11-18T20:45:00.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.152+0000 7f085b7fe640 1 -- 172.21.15.60:0/1787239854 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f085403e960 con 0x7f0864160fb0 2023-11-18T20:45:00.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.153+0000 7f085b7fe640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f084406ae40 0x7f084406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.153+0000 7f085b7fe640 1 -- 172.21.15.60:0/1787239854 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08540ac580 con 0x7f0864160fb0 2023-11-18T20:45:00.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.153+0000 7f0862d9d640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f084406ae40 0x7f084406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.154+0000 7f0862d9d640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f084406ae40 0x7f084406d300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f085c00a950 tx=0x7f085c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.157+0000 7f085b7fe640 1 -- 172.21.15.60:0/1787239854 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08641f7100 con 0x7f0864160fb0 2023-11-18T20:45:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.334+0000 7f0869133640 1 -- 172.21.15.60:0/1787239854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08641f7100 con 0x7f0864160fb0 2023-11-18T20:45:00.336 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.334+0000 7f085b7fe640 1 -- 172.21.15.60:0/1787239854 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08641f7100 con 0x7f0864160fb0 2023-11-18T20:45:00.337 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2023-11-18T20:45:00.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 -- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f084406ae40 msgr2=0x7f084406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f084406ae40 0x7f084406d300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f085c00a950 tx=0x7f085c008040 comp rx=0 tx=0).stop 2023-11-18T20:45:00.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 -- 172.21.15.60:0/1787239854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0864160fb0 msgr2=0x7f086406cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0864160fb0 0x7f086406cb70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0854009f40 tx=0x7f0854004a40 comp rx=0 tx=0).stop 2023-11-18T20:45:00.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 -- 172.21.15.60:0/1787239854 shutdown_connections 2023-11-18T20:45:00.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f084406ae40 0x7f084406d300 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0864160fb0 0x7f086406cb70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08640b3ff0 0x7f086406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.339 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 --2- 172.21.15.60:0/1787239854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086406ea50 0x7f086413ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 -- 172.21.15.60:0/1787239854 >> 172.21.15.60:0/1787239854 conn(0x7f086406a6b0 msgr2=0x7f08640b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:00.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 -- 172.21.15.60:0/1787239854 shutdown_connections 2023-11-18T20:45:00.340 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.336+0000 7f0869133640 1 -- 172.21.15.60:0/1787239854 wait complete. 2023-11-18T20:45:00.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-18T20:45:00.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f3450b89640 1 Processor -- start 2023-11-18T20:45:00.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f3450b89640 1 -- start start 2023-11-18T20:45:00.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f3450b89640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f344c140310 0x7f344c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f3450b89640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f344c06abe0 0x7f344c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f3450b89640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c1406f0 con 0x7f344c1409d0 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f3450b89640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344c161e20 con 0x7f344c06abe0 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f3450b89640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344c143960 con 0x7f344c140310 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f344ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f344c06abe0 0x7f344c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f344ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f344c06abe0 0x7f344c06efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40044/0 (socket says 172.21.15.60:40044) 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f344ad9d640 1 -- 172.21.15.60:0/965318061 learned_addr learned my addr 172.21.15.60:0/965318061 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f344a59c640 1 --2- 172.21.15.60:0/965318061 >> v2:172.21.15.100:3300/0 conn(0x7f344c140310 0x7f344c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f344ad9d640 1 -- 172.21.15.60:0/965318061 >> v2:172.21.15.100:3300/0 conn(0x7f344c140310 msgr2=0x7f344c1618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f344ad9d640 1 --2- 172.21.15.60:0/965318061 >> v2:172.21.15.100:3300/0 conn(0x7f344c140310 0x7f344c1618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f344ad9d640 1 -- 172.21.15.60:0/965318061 >> v1:172.21.15.100:6789/0 conn(0x7f344c1409d0 legacy=0x7f344c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.458+0000 7f344ad9d640 1 -- 172.21.15.60:0/965318061 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c142de0 con 0x7f344c06abe0 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.459+0000 7f344ad9d640 1 --2- 172.21.15.60:0/965318061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c06abe0 0x7f344c06efb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f344c0b5840 tx=0x7f344402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0de2404ccecf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.459+0000 7f3449d9b640 1 -- 172.21.15.60:0/965318061 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344402ca80 con 0x7f344c06abe0 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.459+0000 7f3449d9b640 1 -- 172.21.15.60:0/965318061 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344402cbe0 con 0x7f344c06abe0 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.459+0000 7f3449d9b640 1 -- 172.21.15.60:0/965318061 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34440343f0 con 0x7f344c06abe0 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.460+0000 7f3450b89640 1 -- 172.21.15.60:0/965318061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c06abe0 msgr2=0x7f344c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.460+0000 7f3450b89640 1 --2- 172.21.15.60:0/965318061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c06abe0 0x7f344c06efb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f344c0b5840 tx=0x7f344402bee0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.460+0000 7f3450b89640 1 -- 172.21.15.60:0/965318061 shutdown_connections 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.460+0000 7f3450b89640 1 --2- 172.21.15.60:0/965318061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c06abe0 0x7f344c06efb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.460+0000 7f3450b89640 1 --2- 172.21.15.60:0/965318061 >> v2:172.21.15.100:3300/0 conn(0x7f344c140310 0x7f344c1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.460+0000 7f3450b89640 1 -- 172.21.15.60:0/965318061 >> 172.21.15.60:0/965318061 conn(0x7f344c0b1e50 msgr2=0x7f344c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.460+0000 7f3450b89640 1 -- 172.21.15.60:0/965318061 shutdown_connections 2023-11-18T20:45:00.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.460+0000 7f3450b89640 1 -- 172.21.15.60:0/965318061 wait complete. 2023-11-18T20:45:00.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.461+0000 7f3450b89640 1 Processor -- start 2023-11-18T20:45:00.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.461+0000 7f3450b89640 1 -- start start 2023-11-18T20:45:00.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3450b89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c06abe0 0x7f344c132fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3450b89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c140310 0x7f344c1f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3450b89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f344c1409d0 0x7f344c1f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3450b89640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c133a60 con 0x7f344c140310 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3450b89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c133d40 con 0x7f344c06abe0 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3450b89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c134020 con 0x7f344c1409d0 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3443fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c140310 0x7f344c1f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3443fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c140310 0x7f344c1f6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40058/0 (socket says 172.21.15.60:40058) 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3443fff640 1 -- 172.21.15.60:0/1604136821 learned_addr learned my addr 172.21.15.60:0/1604136821 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f344a59c640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c06abe0 0x7f344c132fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f344ad9d640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f344c1409d0 0x7f344c1f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3443fff640 1 -- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f344c1409d0 msgr2=0x7f344c1f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3443fff640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f344c1409d0 0x7f344c1f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3443fff640 1 -- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c06abe0 msgr2=0x7f344c132fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3443fff640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c06abe0 0x7f344c132fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f3443fff640 1 -- 172.21.15.60:0/1604136821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3444009b30 con 0x7f344c140310 2023-11-18T20:45:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.462+0000 7f344a59c640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c06abe0 0x7f344c132fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.463+0000 7f3443fff640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c140310 0x7f344c1f6130 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f343400c450 tx=0x7f343400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.463+0000 7f3442ffd640 1 -- 172.21.15.60:0/1604136821 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340052d0 con 0x7f344c140310 2023-11-18T20:45:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.463+0000 7f3442ffd640 1 -- 172.21.15.60:0/1604136821 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434005430 con 0x7f344c140310 2023-11-18T20:45:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.463+0000 7f3442ffd640 1 -- 172.21.15.60:0/1604136821 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434012870 con 0x7f344c140310 2023-11-18T20:45:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.463+0000 7f3450b89640 1 -- 172.21.15.60:0/1604136821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c134300 con 0x7f344c140310 2023-11-18T20:45:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.463+0000 7f3450b89640 1 -- 172.21.15.60:0/1604136821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c0b4120 con 0x7f344c140310 2023-11-18T20:45:00.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.464+0000 7f3450b89640 1 -- 172.21.15.60:0/1604136821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c134820 con 0x7f344c140310 2023-11-18T20:45:00.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.464+0000 7f3442ffd640 1 -- 172.21.15.60:0/1604136821 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f34340129d0 con 0x7f344c140310 2023-11-18T20:45:00.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.465+0000 7f3442ffd640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f342c06ae40 0x7f342c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.465+0000 7f3442ffd640 1 -- 172.21.15.60:0/1604136821 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34340891f0 con 0x7f344c140310 2023-11-18T20:45:00.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.465+0000 7f344a59c640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f342c06ae40 0x7f342c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.465+0000 7f344a59c640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f342c06ae40 0x7f342c06d300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f343c006fd0 tx=0x7f343c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:00.471 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.470+0000 7f3442ffd640 1 -- 172.21.15.60:0/1604136821 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f344c134820 con 0x7f344c140310 2023-11-18T20:45:00.646 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.644+0000 7f3450b89640 1 -- 172.21.15.60:0/1604136821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f344c134820 con 0x7f344c140310 2023-11-18T20:45:00.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.645+0000 7f3442ffd640 1 -- 172.21.15.60:0/1604136821 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f344c134820 con 0x7f344c140310 2023-11-18T20:45:00.647 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2023-11-18T20:45:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.646+0000 7f3450b89640 1 -- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f342c06ae40 msgr2=0x7f342c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.646+0000 7f3450b89640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f342c06ae40 0x7f342c06d300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f343c006fd0 tx=0x7f343c008040 comp rx=0 tx=0).stop 2023-11-18T20:45:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.646+0000 7f3450b89640 1 -- 172.21.15.60:0/1604136821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c140310 msgr2=0x7f344c1f6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.646+0000 7f3450b89640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c140310 0x7f344c1f6130 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f343400c450 tx=0x7f343400c910 comp rx=0 tx=0).stop 2023-11-18T20:45:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.646+0000 7f3450b89640 1 -- 172.21.15.60:0/1604136821 shutdown_connections 2023-11-18T20:45:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.646+0000 7f3450b89640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f342c06ae40 0x7f342c06d300 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f343c006fd0 tx=0x7f343c008040 comp rx=0 tx=0).stop 2023-11-18T20:45:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.647+0000 7f3450b89640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f344c1409d0 0x7f344c1f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.649 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.647+0000 7f3450b89640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f344c140310 0x7f344c1f6130 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.647+0000 7f3450b89640 1 --2- 172.21.15.60:0/1604136821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c06abe0 0x7f344c132fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.647+0000 7f3450b89640 1 -- 172.21.15.60:0/1604136821 >> 172.21.15.60:0/1604136821 conn(0x7f344c0b1e50 msgr2=0x7f344c15eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.647+0000 7f3450b89640 1 -- 172.21.15.60:0/1604136821 shutdown_connections 2023-11-18T20:45:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.647+0000 7f3450b89640 1 -- 172.21.15.60:0/1604136821 wait complete. 2023-11-18T20:45:00.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-18T20:45:00.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.768+0000 7f45ddc61640 1 Processor -- start 2023-11-18T20:45:00.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.768+0000 7f45ddc61640 1 -- start start 2023-11-18T20:45:00.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.768+0000 7f45ddc61640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f45d80aebf0 0x7f45d815e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.768+0000 7f45ddc61640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d80b4750 con 0x7f45d80cf160 2023-11-18T20:45:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.768+0000 7f45ddc61640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d80cbc60 con 0x7f45d806b4e0 2023-11-18T20:45:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.768+0000 7f45ddc61640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d80cbf40 con 0x7f45d80aebf0 2023-11-18T20:45:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f45d80cf160 0x7f45d80b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45536/0 (socket says 172.21.15.60:45536) 2023-11-18T20:45:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d7fff640 1 -- 172.21.15.60:0/2957048436 learned_addr learned my addr 172.21.15.60:0/2957048436 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d77fe640 1 --2- 172.21.15.60:0/2957048436 >> v2:172.21.15.100:3300/0 conn(0x7f45d80aebf0 0x7f45d815e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520570727 0 0) 0x7f45d80b4750 con 0x7f45d80cf160 2023-11-18T20:45:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45c0003520 con 0x7f45d80cf160 2023-11-18T20:45:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1680284597 0 0) 0x7f45c0003520 con 0x7f45d80cf160 2023-11-18T20:45:00.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 >> v2:172.21.15.100:3300/0 conn(0x7f45d80aebf0 msgr2=0x7f45d815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 --2- 172.21.15.60:0/2957048436 >> v2:172.21.15.100:3300/0 conn(0x7f45d80aebf0 0x7f45d815e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 >> v1:172.21.15.100:6789/0 conn(0x7f45d806b4e0 legacy=0x7f45d80b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80cc220 con 0x7f45d80cf160 2023-11-18T20:45:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f45d00019f0 con 0x7f45d80cf160 2023-11-18T20:45:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 >> v1:172.21.15.60:6789/0 conn(0x7f45d80cf160 legacy=0x7f45d80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c0003970 0x7f45c0003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45c0004370 0x7f45c0004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c0005040 0x7f45c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80cbf40 con 0x7f45c0005040 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c000bfa0 con 0x7f45c0003970 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c000c160 con 0x7f45c0004370 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d7fff640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c0005040 0x7f45c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d77fe640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c0003970 0x7f45c0003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d6ffd640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45c0004370 0x7f45c0004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d7fff640 1 -- 172.21.15.60:0/2957048436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45c0004370 msgr2=0x7f45c0004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d7fff640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45c0004370 0x7f45c0004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d7fff640 1 -- 172.21.15.60:0/2957048436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c0003970 msgr2=0x7f45c0003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.769+0000 7f45d7fff640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c0003970 0x7f45c0003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.770+0000 7f45d7fff640 1 -- 172.21.15.60:0/2957048436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0003520 con 0x7f45c0005040 2023-11-18T20:45:00.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.770+0000 7f45d77fe640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c0003970 0x7f45c0003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.770+0000 7f45d7fff640 1 -- 172.21.15.60:0/2957048436 reap_dead start 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.770+0000 7f45d7fff640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c0005040 0x7f45c000b980 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f45d000a840 tx=0x7f45c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacf1dd95c28f21 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.770+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d002cea0 con 0x7f45c0005040 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.770+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d002f070 con 0x7f45c0005040 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.770+0000 7f45d67fc640 1 -- 172.21.15.60:0/2957048436 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d00348b0 con 0x7f45c0005040 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.771+0000 7f45ddc61640 1 -- 172.21.15.60:0/2957048436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c0005040 msgr2=0x7f45c000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.771+0000 7f45ddc61640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c0005040 0x7f45c000b980 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f45d000a840 tx=0x7f45c00057f0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.771+0000 7f45ddc61640 1 -- 172.21.15.60:0/2957048436 shutdown_connections 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.771+0000 7f45ddc61640 1 --2- 172.21.15.60:0/2957048436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c0005040 0x7f45c000b980 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.771+0000 7f45ddc61640 1 -- 172.21.15.60:0/2957048436 >> 172.21.15.60:0/2957048436 conn(0x7f45d8070ec0 msgr2=0x7f45d80712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:00.777 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.771+0000 7f45ddc61640 1 -- 172.21.15.60:0/2957048436 shutdown_connections 2023-11-18T20:45:00.778 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.771+0000 7f45ddc61640 1 -- 172.21.15.60:0/2957048436 wait complete. 2023-11-18T20:45:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.772+0000 7f45ddc61640 1 Processor -- start 2023-11-18T20:45:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.772+0000 7f45ddc61640 1 -- start start 2023-11-18T20:45:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45ddc61640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c0005040 0x7f45d80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45ddc61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ad660 0x7f45d8139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45ddc61640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45d813a0a0 0x7f45d814d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45ddc61640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80ad1d0 con 0x7f45d80ad660 2023-11-18T20:45:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45ddc61640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80ac8e0 con 0x7f45c0005040 2023-11-18T20:45:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45ddc61640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d813a4f0 con 0x7f45d813a0a0 2023-11-18T20:45:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ad660 0x7f45d8139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.780 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ad660 0x7f45d8139b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55946/0 (socket says 172.21.15.60:55946) 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45d6ffd640 1 -- 172.21.15.60:0/3691284842 learned_addr learned my addr 172.21.15.60:0/3691284842 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45d6ffd640 1 -- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45d813a0a0 msgr2=0x7f45d814d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45d6ffd640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45d813a0a0 0x7f45d814d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45d6ffd640 1 -- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c0005040 msgr2=0x7f45d80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45d6ffd640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c0005040 0x7f45d80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45d6ffd640 1 -- 172.21.15.60:0/3691284842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d814dd70 con 0x7f45d80ad660 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.773+0000 7f45d6ffd640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ad660 0x7f45d8139b60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00e5f0 tx=0x7f45cc00eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.774+0000 7f45bffff640 1 -- 172.21.15.60:0/3691284842 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0052e0 con 0x7f45d80ad660 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.774+0000 7f45bffff640 1 -- 172.21.15.60:0/3691284842 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc005440 con 0x7f45d80ad660 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.774+0000 7f45ddc61640 1 -- 172.21.15.60:0/3691284842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d80acb80 con 0x7f45d80ad660 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.774+0000 7f45bffff640 1 -- 172.21.15.60:0/3691284842 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc003830 con 0x7f45d80ad660 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.774+0000 7f45ddc61640 1 -- 172.21.15.60:0/3691284842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d80b0a70 con 0x7f45d80ad660 2023-11-18T20:45:00.781 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.775+0000 7f45ddc61640 1 -- 172.21.15.60:0/3691284842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d8067420 con 0x7f45d80ad660 2023-11-18T20:45:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.775+0000 7f45bffff640 1 -- 172.21.15.60:0/3691284842 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45cc003990 con 0x7f45d80ad660 2023-11-18T20:45:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.776+0000 7f45bffff640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f45c406ae40 0x7f45c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.776+0000 7f45bffff640 1 -- 172.21.15.60:0/3691284842 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45cc0894f0 con 0x7f45d80ad660 2023-11-18T20:45:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.776+0000 7f45d77fe640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f45c406ae40 0x7f45c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.776+0000 7f45d77fe640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f45c406ae40 0x7f45c406d300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009830 tx=0x7f45c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.780+0000 7f45bffff640 1 -- 172.21.15.60:0/3691284842 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45cc054660 con 0x7f45d80ad660 2023-11-18T20:45:00.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.957+0000 7f45ddc61640 1 -- 172.21.15.60:0/3691284842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45d80b0c30 con 0x7f45d80ad660 2023-11-18T20:45:00.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.957+0000 7f45bffff640 1 -- 172.21.15.60:0/3691284842 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45cc057c80 con 0x7f45d80ad660 2023-11-18T20:45:00.960 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2023-11-18T20:45:00.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 -- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f45c406ae40 msgr2=0x7f45c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f45c406ae40 0x7f45c406d300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009830 tx=0x7f45c8008040 comp rx=0 tx=0).stop 2023-11-18T20:45:00.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 -- 172.21.15.60:0/3691284842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ad660 msgr2=0x7f45d8139b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:00.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ad660 0x7f45d8139b60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00e5f0 tx=0x7f45cc00eab0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 -- 172.21.15.60:0/3691284842 shutdown_connections 2023-11-18T20:45:00.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f45c406ae40 0x7f45c406d300 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45d813a0a0 0x7f45d814d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45d80ad660 0x7f45d8139b60 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 --2- 172.21.15.60:0/3691284842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c0005040 0x7f45d80b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:00.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 -- 172.21.15.60:0/3691284842 >> 172.21.15.60:0/3691284842 conn(0x7f45d8070ec0 msgr2=0x7f45d81595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:00.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 -- 172.21.15.60:0/3691284842 shutdown_connections 2023-11-18T20:45:00.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:00.959+0000 7f45ddc61640 1 -- 172.21.15.60:0/3691284842 wait complete. 2023-11-18T20:45:00.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:45:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.078+0000 7f6cf4861640 1 Processor -- start 2023-11-18T20:45:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.078+0000 7f6cf4861640 1 -- start start 2023-11-18T20:45:01.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6cf4861640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf0140310 0x7f6cf01618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6cf4861640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cf006abe0 0x7f6cf006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6cf4861640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf01406f0 con 0x7f6cf01409d0 2023-11-18T20:45:01.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6cf4861640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf0161e20 con 0x7f6cf006abe0 2023-11-18T20:45:01.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6cf4861640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0143960 con 0x7f6cf0140310 2023-11-18T20:45:01.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6ceed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cf006abe0 0x7f6cf006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6ceed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cf006abe0 0x7f6cf006efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55952/0 (socket says 172.21.15.60:55952) 2023-11-18T20:45:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6ceed9d640 1 -- 172.21.15.60:0/2038447837 learned_addr learned my addr 172.21.15.60:0/2038447837 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6cee59c640 1 --2- 172.21.15.60:0/2038447837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf0140310 0x7f6cf01618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6ceed9d640 1 -- 172.21.15.60:0/2038447837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf0140310 msgr2=0x7f6cf01618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6ceed9d640 1 --2- 172.21.15.60:0/2038447837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf0140310 0x7f6cf01618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6ceed9d640 1 -- 172.21.15.60:0/2038447837 >> v1:172.21.15.100:6789/0 conn(0x7f6cf01409d0 legacy=0x7f6cf006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.079+0000 7f6ceed9d640 1 -- 172.21.15.60:0/2038447837 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf0142de0 con 0x7f6cf006abe0 2023-11-18T20:45:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.080+0000 7f6ceed9d640 1 --2- 172.21.15.60:0/2038447837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf006abe0 0x7f6cf006efb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0161f60 tx=0x7f6ce802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd81c068ccfabca4 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.080+0000 7f6cedd9b640 1 -- 172.21.15.60:0/2038447837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce802ca50 con 0x7f6cf006abe0 2023-11-18T20:45:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.080+0000 7f6cedd9b640 1 -- 172.21.15.60:0/2038447837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce802cbb0 con 0x7f6cf006abe0 2023-11-18T20:45:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.080+0000 7f6cedd9b640 1 -- 172.21.15.60:0/2038447837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80343f0 con 0x7f6cf006abe0 2023-11-18T20:45:01.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.080+0000 7f6cf4861640 1 -- 172.21.15.60:0/2038447837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf006abe0 msgr2=0x7f6cf006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.080+0000 7f6cf4861640 1 --2- 172.21.15.60:0/2038447837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf006abe0 0x7f6cf006efb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0161f60 tx=0x7f6ce802beb0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.080+0000 7f6cf4861640 1 -- 172.21.15.60:0/2038447837 shutdown_connections 2023-11-18T20:45:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.080+0000 7f6cf4861640 1 --2- 172.21.15.60:0/2038447837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf006abe0 0x7f6cf006efb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.080+0000 7f6cf4861640 1 --2- 172.21.15.60:0/2038447837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf0140310 0x7f6cf01618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.080+0000 7f6cf4861640 1 -- 172.21.15.60:0/2038447837 >> 172.21.15.60:0/2038447837 conn(0x7f6cf00b1e50 msgr2=0x7f6cf00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.081+0000 7f6cf4861640 1 -- 172.21.15.60:0/2038447837 shutdown_connections 2023-11-18T20:45:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.081+0000 7f6cf4861640 1 -- 172.21.15.60:0/2038447837 wait complete. 2023-11-18T20:45:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.082+0000 7f6cf4861640 1 Processor -- start 2023-11-18T20:45:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.082+0000 7f6cf4861640 1 -- start start 2023-11-18T20:45:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6cf4861640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf006abe0 0x7f6cf01f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6cf4861640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf0140310 0x7f6cf013f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6cf4861640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf01409d0 0x7f6cf0158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6cf4861640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0155910 con 0x7f6cf0140310 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6cf4861640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0155d40 con 0x7f6cf01409d0 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6cf4861640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf01f33f0 con 0x7f6cf006abe0 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf0140310 0x7f6cf013f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf0140310 0x7f6cf013f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55966/0 (socket says 172.21.15.60:55966) 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce7fff640 1 -- 172.21.15.60:0/162519552 learned_addr learned my addr 172.21.15.60:0/162519552 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ceed9d640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf01409d0 0x7f6cf0158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6cee59c640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf006abe0 0x7f6cf01f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce7fff640 1 -- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf006abe0 msgr2=0x7f6cf01f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce7fff640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf006abe0 0x7f6cf01f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce7fff640 1 -- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf01409d0 msgr2=0x7f6cf0158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce7fff640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf01409d0 0x7f6cf0158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce7fff640 1 -- 172.21.15.60:0/162519552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce8009b30 con 0x7f6cf0140310 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ceed9d640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf01409d0 0x7f6cf0158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6cee59c640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf006abe0 0x7f6cf01f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce7fff640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf0140310 0x7f6cf013f880 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800c400 tx=0x7f6cd800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce6ffd640 1 -- 172.21.15.60:0/162519552 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80052d0 con 0x7f6cf0140310 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.083+0000 7f6ce6ffd640 1 -- 172.21.15.60:0/162519552 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd8005430 con 0x7f6cf0140310 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.084+0000 7f6cf4861640 1 -- 172.21.15.60:0/162519552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf0155fe0 con 0x7f6cf0140310 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.084+0000 7f6ce6ffd640 1 -- 172.21.15.60:0/162519552 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8003830 con 0x7f6cf0140310 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.084+0000 7f6cf4861640 1 -- 172.21.15.60:0/162519552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf01565a0 con 0x7f6cf0140310 2023-11-18T20:45:01.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.084+0000 7f6cf4861640 1 -- 172.21.15.60:0/162519552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf01567c0 con 0x7f6cf0140310 2023-11-18T20:45:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.085+0000 7f6ce6ffd640 1 -- 172.21.15.60:0/162519552 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6cd8003990 con 0x7f6cf0140310 2023-11-18T20:45:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.085+0000 7f6ce6ffd640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6cd006ae40 0x7f6cd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.085+0000 7f6ce6ffd640 1 -- 172.21.15.60:0/162519552 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cd8088bb0 con 0x7f6cf0140310 2023-11-18T20:45:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.086+0000 7f6cee59c640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6cd006ae40 0x7f6cd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.086+0000 7f6cee59c640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6cd006ae40 0x7f6cd006d300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009b90 tx=0x7f6ce0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.090+0000 7f6ce6ffd640 1 -- 172.21.15.60:0/162519552 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6cd8053cf0 con 0x7f6cf0140310 2023-11-18T20:45:01.301 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.300+0000 7f6cf4861640 1 -- 172.21.15.60:0/162519552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cf0069400 con 0x7f6cf0140310 2023-11-18T20:45:01.302 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.301+0000 7f6ce6ffd640 1 -- 172.21.15.60:0/162519552 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f6cd8017090 con 0x7f6cf0140310 2023-11-18T20:45:01.302 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:01.302 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1578336464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1578336464},{"type":"v1","addr":"172.21.15.100:6836","nonce":1578336464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/88984334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":88984334},{"type":"v1","addr":"172.21.15.100:6837","nonce":88984334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3777286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3777286992},{"type":"v1","addr":"172.21.15.60:6833","nonce":3777286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:44:33.070091+0000","modified":"2023-11-18T20:44:35.050014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3098423505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3098423505},{"type":"v1","addr":"172.21.15.60:6835","nonce":3098423505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-18T20:45:01.303 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2023-11-18T20:45:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.302+0000 7f6cf4861640 1 -- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6cd006ae40 msgr2=0x7f6cd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.302+0000 7f6cf4861640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6cd006ae40 0x7f6cd006d300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009b90 tx=0x7f6ce0008040 comp rx=0 tx=0).stop 2023-11-18T20:45:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.302+0000 7f6cf4861640 1 -- 172.21.15.60:0/162519552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf0140310 msgr2=0x7f6cf013f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.302+0000 7f6cf4861640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf0140310 0x7f6cf013f880 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800c400 tx=0x7f6cd800c8c0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.302+0000 7f6cf4861640 1 -- 172.21.15.60:0/162519552 shutdown_connections 2023-11-18T20:45:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.302+0000 7f6cf4861640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6cd006ae40 0x7f6cd006d300 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.302+0000 7f6cf4861640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf01409d0 0x7f6cf0158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.302+0000 7f6cf4861640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf0140310 0x7f6cf013f880 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.302+0000 7f6cf4861640 1 --2- 172.21.15.60:0/162519552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf006abe0 0x7f6cf01f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.302+0000 7f6cf4861640 1 -- 172.21.15.60:0/162519552 >> 172.21.15.60:0/162519552 conn(0x7f6cf00b1e50 msgr2=0x7f6cf00afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.303+0000 7f6cf4861640 1 -- 172.21.15.60:0/162519552 shutdown_connections 2023-11-18T20:45:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.303+0000 7f6cf4861640 1 -- 172.21.15.60:0/162519552 wait complete. 2023-11-18T20:45:01.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:45:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.420+0000 7fa2d6e21640 1 Processor -- start 2023-11-18T20:45:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.420+0000 7fa2d6e21640 1 -- start start 2023-11-18T20:45:01.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.420+0000 7fa2d6e21640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2d015d330 0x7fa2d00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.420+0000 7fa2d6e21640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2d00d2910 0x7fa2d00d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.420+0000 7fa2d6e21640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2d00d3730 0x7fa2d00b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.420+0000 7fa2d6e21640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d00b2b10 con 0x7fa2d015d330 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.420+0000 7fa2d6e21640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d00d3b50 con 0x7fa2d00d3730 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.420+0000 7fa2d6e21640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d00d1590 con 0x7fa2d00d2910 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.421+0000 7fa2d4bbd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2d015d330 0x7fa2d00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.421+0000 7fa2d4bbd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2d015d330 0x7fa2d00b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55978/0 (socket says 172.21.15.60:55978) 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.421+0000 7fa2d4bbd640 1 -- 172.21.15.60:0/2240743909 learned_addr learned my addr 172.21.15.60:0/2240743909 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.421+0000 7fa2d53be640 1 --2- 172.21.15.60:0/2240743909 >> v2:172.21.15.100:3300/0 conn(0x7fa2d00d3730 0x7fa2d00b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.421+0000 7fa2cffff640 1 --2- 172.21.15.60:0/2240743909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2d00d2910 0x7fa2d00d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.421+0000 7fa2d4bbd640 1 -- 172.21.15.60:0/2240743909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2d00d2910 msgr2=0x7fa2d00d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.421+0000 7fa2d4bbd640 1 --2- 172.21.15.60:0/2240743909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2d00d2910 0x7fa2d00d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.421+0000 7fa2d4bbd640 1 -- 172.21.15.60:0/2240743909 >> v2:172.21.15.100:3300/0 conn(0x7fa2d00d3730 msgr2=0x7fa2d00b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.421+0000 7fa2d4bbd640 1 --2- 172.21.15.60:0/2240743909 >> v2:172.21.15.100:3300/0 conn(0x7fa2d00d3730 0x7fa2d00b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.421+0000 7fa2d4bbd640 1 -- 172.21.15.60:0/2240743909 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00d1870 con 0x7fa2d015d330 2023-11-18T20:45:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2d4bbd640 1 --2- 172.21.15.60:0/2240743909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d015d330 0x7fa2d00b8b80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00d1510 tx=0x7fa2c402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5553662857639f54 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2cf7fe640 1 -- 172.21.15.60:0/2240743909 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c40317e0 con 0x7fa2d015d330 2023-11-18T20:45:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2cf7fe640 1 -- 172.21.15.60:0/2240743909 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c4031940 con 0x7fa2d015d330 2023-11-18T20:45:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2cf7fe640 1 -- 172.21.15.60:0/2240743909 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4031cd0 con 0x7fa2d015d330 2023-11-18T20:45:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2d6e21640 1 -- 172.21.15.60:0/2240743909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d015d330 msgr2=0x7fa2d00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2d6e21640 1 --2- 172.21.15.60:0/2240743909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d015d330 0x7fa2d00b8b80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00d1510 tx=0x7fa2c402b880 comp rx=0 tx=0).stop 2023-11-18T20:45:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2d6e21640 1 -- 172.21.15.60:0/2240743909 shutdown_connections 2023-11-18T20:45:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2d6e21640 1 --2- 172.21.15.60:0/2240743909 >> v2:172.21.15.100:3300/0 conn(0x7fa2d00d3730 0x7fa2d00b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2d6e21640 1 --2- 172.21.15.60:0/2240743909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2d00d2910 0x7fa2d00d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2d6e21640 1 --2- 172.21.15.60:0/2240743909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d015d330 0x7fa2d00b8b80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2d6e21640 1 -- 172.21.15.60:0/2240743909 >> 172.21.15.60:0/2240743909 conn(0x7fa2d0072020 msgr2=0x7fa2d0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2d6e21640 1 -- 172.21.15.60:0/2240743909 shutdown_connections 2023-11-18T20:45:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.422+0000 7fa2d6e21640 1 -- 172.21.15.60:0/2240743909 wait complete. 2023-11-18T20:45:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.424+0000 7fa2d6e21640 1 Processor -- start 2023-11-18T20:45:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.424+0000 7fa2d6e21640 1 -- start start 2023-11-18T20:45:01.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.424+0000 7fa2d6e21640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d00d2910 0x7fa2d0145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.424+0000 7fa2d6e21640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2d00d3730 0x7fa2d01532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.424+0000 7fa2d6e21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2d015d330 0x7fa2d0145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.424+0000 7fa2d6e21640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d01564e0 con 0x7fa2d00d2910 2023-11-18T20:45:01.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.424+0000 7fa2d6e21640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d01537f0 con 0x7fa2d015d330 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.424+0000 7fa2d6e21640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d0155b80 con 0x7fa2d00d3730 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d4bbd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d00d2910 0x7fa2d0145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d4bbd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d00d2910 0x7fa2d0145a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55984/0 (socket says 172.21.15.60:55984) 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d4bbd640 1 -- 172.21.15.60:0/58553205 learned_addr learned my addr 172.21.15.60:0/58553205 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2cffff640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2d00d3730 0x7fa2d01532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d53be640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2d015d330 0x7fa2d0145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d4bbd640 1 -- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2d00d3730 msgr2=0x7fa2d01532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d4bbd640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2d00d3730 0x7fa2d01532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d4bbd640 1 -- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2d015d330 msgr2=0x7fa2d0145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d4bbd640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2d015d330 0x7fa2d0145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d4bbd640 1 -- 172.21.15.60:0/58553205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40094d0 con 0x7fa2d00d2910 2023-11-18T20:45:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d4bbd640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d00d2910 0x7fa2d0145a40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4000c00 tx=0x7fa2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2cd7fa640 1 -- 172.21.15.60:0/58553205 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4032880 con 0x7fa2d00d2910 2023-11-18T20:45:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2cd7fa640 1 -- 172.21.15.60:0/58553205 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c4032a20 con 0x7fa2d00d2910 2023-11-18T20:45:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d6e21640 1 -- 172.21.15.60:0/58553205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d0155e20 con 0x7fa2d00d2910 2023-11-18T20:45:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2cd7fa640 1 -- 172.21.15.60:0/58553205 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4032dd0 con 0x7fa2d00d2910 2023-11-18T20:45:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.425+0000 7fa2d6e21640 1 -- 172.21.15.60:0/58553205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d0153a30 con 0x7fa2d00d2910 2023-11-18T20:45:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.426+0000 7fa2d6e21640 1 -- 172.21.15.60:0/58553205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d0069400 con 0x7fa2d00d2910 2023-11-18T20:45:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.427+0000 7fa2cd7fa640 1 -- 172.21.15.60:0/58553205 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2c404e450 con 0x7fa2d00d2910 2023-11-18T20:45:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.427+0000 7fa2cd7fa640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa2b406ae40 0x7fa2b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.427+0000 7fa2cd7fa640 1 -- 172.21.15.60:0/58553205 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2c40ab6f0 con 0x7fa2d00d2910 2023-11-18T20:45:01.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.427+0000 7fa2cffff640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa2b406ae40 0x7fa2b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.431 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.428+0000 7fa2cffff640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa2b406ae40 0x7fa2b406d300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa2d015bea0 tx=0x7fa2bc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:01.432 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.431+0000 7fa2cd7fa640 1 -- 172.21.15.60:0/58553205 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2c4076870 con 0x7fa2d00d2910 2023-11-18T20:45:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.611+0000 7fa2d6e21640 1 -- 172.21.15.60:0/58553205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2d006a4d0 con 0x7fa2d00d2910 2023-11-18T20:45:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.612+0000 7fa2cd7fa640 1 -- 172.21.15.60:0/58553205 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fa2c4079e90 con 0x7fa2d00d2910 2023-11-18T20:45:01.614 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:01.614 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:34.103787+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:45:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.614+0000 7fa2d6e21640 1 -- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa2b406ae40 msgr2=0x7fa2b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.614+0000 7fa2d6e21640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa2b406ae40 0x7fa2b406d300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa2d015bea0 tx=0x7fa2bc008040 comp rx=0 tx=0).stop 2023-11-18T20:45:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.614+0000 7fa2d6e21640 1 -- 172.21.15.60:0/58553205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d00d2910 msgr2=0x7fa2d0145a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.614+0000 7fa2d6e21640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d00d2910 0x7fa2d0145a40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4000c00 tx=0x7fa2c40046c0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.615+0000 7fa2d6e21640 1 -- 172.21.15.60:0/58553205 shutdown_connections 2023-11-18T20:45:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.615+0000 7fa2d6e21640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa2b406ae40 0x7fa2b406d300 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.615+0000 7fa2d6e21640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2d015d330 0x7fa2d0145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.615+0000 7fa2d6e21640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2d00d3730 0x7fa2d01532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.615+0000 7fa2d6e21640 1 --2- 172.21.15.60:0/58553205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2d00d2910 0x7fa2d0145a40 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.615+0000 7fa2d6e21640 1 -- 172.21.15.60:0/58553205 >> 172.21.15.60:0/58553205 conn(0x7fa2d0072020 msgr2=0x7fa2d00b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:01.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.615+0000 7fa2d6e21640 1 -- 172.21.15.60:0/58553205 shutdown_connections 2023-11-18T20:45:01.617 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.615+0000 7fa2d6e21640 1 -- 172.21.15.60:0/58553205 wait complete. 2023-11-18T20:45:01.627 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-18T20:45:01.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:45:01.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.732+0000 7fb0ade1f640 1 Processor -- start 2023-11-18T20:45:01.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.732+0000 7fb0ade1f640 1 -- start start 2023-11-18T20:45:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0ade1f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb0a815d1d0 0x7fb0a806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0ade1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80aeb80 0x7fb0a806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0ade1f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a80adae0 con 0x7fb0a80ad640 2023-11-18T20:45:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0ade1f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a80addc0 con 0x7fb0a815d1d0 2023-11-18T20:45:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0ade1f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a80ae0a0 con 0x7fb0a80aeb80 2023-11-18T20:45:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0a6ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb0a80ad640 0x7fb0a8160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45550/0 (socket says 172.21.15.60:45550) 2023-11-18T20:45:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0a6ffd640 1 -- 172.21.15.60:0/1386358465 learned_addr learned my addr 172.21.15.60:0/1386358465 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0a77fe640 1 --2- 172.21.15.60:0/1386358465 >> v2:172.21.15.100:3300/0 conn(0x7fb0a815d1d0 0x7fb0a806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0a7fff640 1 --2- 172.21.15.60:0/1386358465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80aeb80 0x7fb0a806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0a67fc640 1 -- 172.21.15.60:0/1386358465 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598099973 0 0) 0x7fb0a80adae0 con 0x7fb0a80ad640 2023-11-18T20:45:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.733+0000 7fb0a67fc640 1 -- 172.21.15.60:0/1386358465 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb090003520 con 0x7fb0a80ad640 2023-11-18T20:45:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.734+0000 7fb0a77fe640 1 -- 172.21.15.60:0/1386358465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80aeb80 msgr2=0x7fb0a806ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.734+0000 7fb0a77fe640 1 --2- 172.21.15.60:0/1386358465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80aeb80 0x7fb0a806ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.734+0000 7fb0a67fc640 1 -- 172.21.15.60:0/1386358465 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 584167337 0 0) 0x7fb090003520 con 0x7fb0a80ad640 2023-11-18T20:45:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.734+0000 7fb0a77fe640 1 -- 172.21.15.60:0/1386358465 >> v1:172.21.15.60:6789/0 conn(0x7fb0a80ad640 legacy=0x7fb0a8160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.734+0000 7fb0a77fe640 1 -- 172.21.15.60:0/1386358465 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a80ae380 con 0x7fb0a815d1d0 2023-11-18T20:45:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.734+0000 7fb0a7fff640 1 --2- 172.21.15.60:0/1386358465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80aeb80 0x7fb0a806ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:01.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.734+0000 7fb0a77fe640 1 --2- 172.21.15.60:0/1386358465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a815d1d0 0x7fb0a806b4e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb0980097a0 tx=0x7fb09802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf2ad1702bb9ba0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0a67fc640 1 -- 172.21.15.60:0/1386358465 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c001670 con 0x7fb0a815d1d0 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0a67fc640 1 -- 172.21.15.60:0/1386358465 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09802e050 con 0x7fb0a815d1d0 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0a67fc640 1 -- 172.21.15.60:0/1386358465 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098030730 con 0x7fb0a815d1d0 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0ade1f640 1 -- 172.21.15.60:0/1386358465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a815d1d0 msgr2=0x7fb0a806b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0ade1f640 1 --2- 172.21.15.60:0/1386358465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a815d1d0 0x7fb0a806b4e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb0980097a0 tx=0x7fb09802b790 comp rx=0 tx=0).stop 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0ade1f640 1 -- 172.21.15.60:0/1386358465 shutdown_connections 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0ade1f640 1 --2- 172.21.15.60:0/1386358465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80aeb80 0x7fb0a806ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0ade1f640 1 --2- 172.21.15.60:0/1386358465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a815d1d0 0x7fb0a806b4e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0ade1f640 1 -- 172.21.15.60:0/1386358465 >> 172.21.15.60:0/1386358465 conn(0x7fb0a8070e70 msgr2=0x7fb0a8071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0ade1f640 1 -- 172.21.15.60:0/1386358465 shutdown_connections 2023-11-18T20:45:01.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.735+0000 7fb0ade1f640 1 -- 172.21.15.60:0/1386358465 wait complete. 2023-11-18T20:45:01.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.736+0000 7fb0ade1f640 1 Processor -- start 2023-11-18T20:45:01.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.737+0000 7fb0ade1f640 1 -- start start 2023-11-18T20:45:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.737+0000 7fb0ade1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80ad640 0x7fb0a8153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.737+0000 7fb0ade1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a80aeb80 0x7fb0a8145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.737+0000 7fb0ade1f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a815d1d0 0x7fb0a80cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.737+0000 7fb0ade1f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a8154200 con 0x7fb0a80aeb80 2023-11-18T20:45:01.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.737+0000 7fb0ade1f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a8159610 con 0x7fb0a815d1d0 2023-11-18T20:45:01.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.737+0000 7fb0ade1f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a8146380 con 0x7fb0a80ad640 2023-11-18T20:45:01.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.737+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a80aeb80 0x7fb0a8145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.737+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a80aeb80 0x7fb0a8145e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55986/0 (socket says 172.21.15.60:55986) 2023-11-18T20:45:01.741 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.737+0000 7fb0a6ffd640 1 -- 172.21.15.60:0/2245542449 learned_addr learned my addr 172.21.15.60:0/2245542449 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb0a77fe640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80ad640 0x7fb0a8153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb0a7fff640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a815d1d0 0x7fb0a80cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb0a6ffd640 1 -- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80ad640 msgr2=0x7fb0a8153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb0a6ffd640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80ad640 0x7fb0a8153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb0a6ffd640 1 -- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a815d1d0 msgr2=0x7fb0a80cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb0a6ffd640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a815d1d0 0x7fb0a80cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb0a6ffd640 1 -- 172.21.15.60:0/2245542449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0009a70 con 0x7fb0a80aeb80 2023-11-18T20:45:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb0a6ffd640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a80aeb80 0x7fb0a8145e40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c740 tx=0x7fb09c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb08ffff640 1 -- 172.21.15.60:0/2245542449 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c005ad0 con 0x7fb0a80aeb80 2023-11-18T20:45:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb08ffff640 1 -- 172.21.15.60:0/2245542449 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c005c30 con 0x7fb0a80aeb80 2023-11-18T20:45:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb0ade1f640 1 -- 172.21.15.60:0/2245542449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb098009400 con 0x7fb0a80aeb80 2023-11-18T20:45:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb08ffff640 1 -- 172.21.15.60:0/2245542449 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c007070 con 0x7fb0a80aeb80 2023-11-18T20:45:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.738+0000 7fb0ade1f640 1 -- 172.21.15.60:0/2245542449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a8159b60 con 0x7fb0a80aeb80 2023-11-18T20:45:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.739+0000 7fb0ade1f640 1 -- 172.21.15.60:0/2245542449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a8069400 con 0x7fb0a80aeb80 2023-11-18T20:45:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.740+0000 7fb08ffff640 1 -- 172.21.15.60:0/2245542449 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb09c0071d0 con 0x7fb0a80aeb80 2023-11-18T20:45:01.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.740+0000 7fb08ffff640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb09406ae70 0x7fb09406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:01.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.740+0000 7fb08ffff640 1 -- 172.21.15.60:0/2245542449 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb09c089220 con 0x7fb0a80aeb80 2023-11-18T20:45:01.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.740+0000 7fb0a77fe640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb09406ae70 0x7fb09406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:01.744 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.741+0000 7fb0a77fe640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb09406ae70 0x7fb09406d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb098034000 tx=0x7fb09802f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.744+0000 7fb08ffff640 1 -- 172.21.15.60:0/2245542449 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb09c054360 con 0x7fb0a80aeb80 2023-11-18T20:45:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.955+0000 7fb0ade1f640 1 -- 172.21.15.60:0/2245542449 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0a80afe70 con 0x7fb0a80aeb80 2023-11-18T20:45:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.955+0000 7fb08ffff640 1 -- 172.21.15.60:0/2245542449 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fb09c057980 con 0x7fb0a80aeb80 2023-11-18T20:45:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:01.957 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1578336464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1578336464},{"type":"v1","addr":"172.21.15.100:6836","nonce":1578336464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/88984334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":88984334},{"type":"v1","addr":"172.21.15.100:6837","nonce":88984334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3777286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3777286992},{"type":"v1","addr":"172.21.15.60:6833","nonce":3777286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:44:33.070091+0000","modified":"2023-11-18T20:44:35.050014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3098423505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3098423505},{"type":"v1","addr":"172.21.15.60:6835","nonce":3098423505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-18T20:45:01.957 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2023-11-18T20:45:01.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 -- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb09406ae70 msgr2=0x7fb09406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb09406ae70 0x7fb09406d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb098034000 tx=0x7fb09802f000 comp rx=0 tx=0).stop 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 -- 172.21.15.60:0/2245542449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a80aeb80 msgr2=0x7fb0a8145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a80aeb80 0x7fb0a8145e40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c740 tx=0x7fb09c00cb40 comp rx=0 tx=0).stop 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 -- 172.21.15.60:0/2245542449 shutdown_connections 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb09406ae70 0x7fb09406d330 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a815d1d0 0x7fb0a80cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a80aeb80 0x7fb0a8145e40 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 --2- 172.21.15.60:0/2245542449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a80ad640 0x7fb0a8153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 -- 172.21.15.60:0/2245542449 >> 172.21.15.60:0/2245542449 conn(0x7fb0a8070e70 msgr2=0x7fb0a806e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 -- 172.21.15.60:0/2245542449 shutdown_connections 2023-11-18T20:45:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:01.957+0000 7fb0ade1f640 1 -- 172.21.15.60:0/2245542449 wait complete. 2023-11-18T20:45:01.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:45:02.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.074+0000 7f10e4524640 1 Processor -- start 2023-11-18T20:45:02.076 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.074+0000 7f10e4524640 1 -- start start 2023-11-18T20:45:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.075+0000 7f10e4524640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10dc143490 0x7f10dc0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.075+0000 7f10e4524640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f10dc15e730 0x7f10dc0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.075+0000 7f10e4524640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f10dc15a4e0 0x7f10dc15a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.075+0000 7f10e4524640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10dc15b520 con 0x7f10dc15e730 2023-11-18T20:45:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.075+0000 7f10e4524640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10dc143870 con 0x7f10dc15a4e0 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.075+0000 7f10e4524640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc15e2a0 con 0x7f10dc143490 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e1abf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f10dc15e730 0x7f10dc0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e1abf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f10dc15e730 0x7f10dc0b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56000/0 (socket says 172.21.15.60:56000) 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e1abf640 1 -- 172.21.15.60:0/293429201 learned_addr learned my addr 172.21.15.60:0/293429201 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e22c0640 1 --2- 172.21.15.60:0/293429201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10dc143490 0x7f10dc0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e2ac1640 1 --2- 172.21.15.60:0/293429201 >> v2:172.21.15.100:3300/0 conn(0x7f10dc15a4e0 0x7f10dc15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e1abf640 1 -- 172.21.15.60:0/293429201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10dc143490 msgr2=0x7f10dc0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e1abf640 1 --2- 172.21.15.60:0/293429201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10dc143490 0x7f10dc0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e1abf640 1 -- 172.21.15.60:0/293429201 >> v2:172.21.15.100:3300/0 conn(0x7f10dc15a4e0 msgr2=0x7f10dc15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e1abf640 1 --2- 172.21.15.60:0/293429201 >> v2:172.21.15.100:3300/0 conn(0x7f10dc15a4e0 0x7f10dc15a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e1abf640 1 -- 172.21.15.60:0/293429201 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc15b240 con 0x7f10dc15e730 2023-11-18T20:45:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.076+0000 7f10e2ac1640 1 --2- 172.21.15.60:0/293429201 >> v2:172.21.15.100:3300/0 conn(0x7f10dc15a4e0 0x7f10dc15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e1abf640 1 --2- 172.21.15.60:0/293429201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc15e730 0x7f10dc0b1230 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f10dc15eb90 tx=0x7f10d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe2d02ba01a22c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e12be640 1 -- 172.21.15.60:0/293429201 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0031870 con 0x7f10dc15e730 2023-11-18T20:45:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e12be640 1 -- 172.21.15.60:0/293429201 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d00319d0 con 0x7f10dc15e730 2023-11-18T20:45:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e12be640 1 -- 172.21.15.60:0/293429201 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0031d60 con 0x7f10dc15e730 2023-11-18T20:45:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e4524640 1 -- 172.21.15.60:0/293429201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc15e730 msgr2=0x7f10dc0b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e4524640 1 --2- 172.21.15.60:0/293429201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc15e730 0x7f10dc0b1230 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f10dc15eb90 tx=0x7f10d002b910 comp rx=0 tx=0).stop 2023-11-18T20:45:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e4524640 1 -- 172.21.15.60:0/293429201 shutdown_connections 2023-11-18T20:45:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e4524640 1 --2- 172.21.15.60:0/293429201 >> v2:172.21.15.100:3300/0 conn(0x7f10dc15a4e0 0x7f10dc15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e4524640 1 --2- 172.21.15.60:0/293429201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc15e730 0x7f10dc0b1230 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e4524640 1 --2- 172.21.15.60:0/293429201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10dc143490 0x7f10dc0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e4524640 1 -- 172.21.15.60:0/293429201 >> 172.21.15.60:0/293429201 conn(0x7f10dc06a4d0 msgr2=0x7f10dc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e4524640 1 -- 172.21.15.60:0/293429201 shutdown_connections 2023-11-18T20:45:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.077+0000 7f10e4524640 1 -- 172.21.15.60:0/293429201 wait complete. 2023-11-18T20:45:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e4524640 1 Processor -- start 2023-11-18T20:45:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e4524640 1 -- start start 2023-11-18T20:45:02.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e4524640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc143490 0x7f10dc159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e4524640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc15a4e0 0x7f10dc13ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e4524640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10dc15e730 0x7f10dc13f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e4524640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc140540 con 0x7f10dc143490 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e4524640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc159160 con 0x7f10dc15a4e0 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e4524640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc140260 con 0x7f10dc15e730 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e22c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc143490 0x7f10dc159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e22c0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc143490 0x7f10dc159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56012/0 (socket says 172.21.15.60:56012) 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.079+0000 7f10e22c0640 1 -- 172.21.15.60:0/445454351 learned_addr learned my addr 172.21.15.60:0/445454351 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10e1abf640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc15a4e0 0x7f10dc13ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10e2ac1640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10dc15e730 0x7f10dc13f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10e22c0640 1 -- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10dc15e730 msgr2=0x7f10dc13f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10e22c0640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10dc15e730 0x7f10dc13f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10e22c0640 1 -- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc15a4e0 msgr2=0x7f10dc13ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10e22c0640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc15a4e0 0x7f10dc13ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10e22c0640 1 -- 172.21.15.60:0/445454351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d00094d0 con 0x7f10dc143490 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10e22c0640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc143490 0x7f10dc159440 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f10d400e9b0 tx=0x7f10d400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10daffd640 1 -- 172.21.15.60:0/445454351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d4009670 con 0x7f10dc143490 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10daffd640 1 -- 172.21.15.60:0/445454351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d400c9a0 con 0x7f10dc143490 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10daffd640 1 -- 172.21.15.60:0/445454351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d400cd50 con 0x7f10dc143490 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10e4524640 1 -- 172.21.15.60:0/445454351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10dc158b10 con 0x7f10dc143490 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.080+0000 7f10e4524640 1 -- 172.21.15.60:0/445454351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10dc157b60 con 0x7f10dc143490 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.081+0000 7f10e4524640 1 -- 172.21.15.60:0/445454351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10dc157db0 con 0x7f10dc143490 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.082+0000 7f10daffd640 1 -- 172.21.15.60:0/445454351 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f10d4013470 con 0x7f10dc143490 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.082+0000 7f10daffd640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10c806ae40 0x7f10c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.082+0000 7f10daffd640 1 -- 172.21.15.60:0/445454351 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10d4089c00 con 0x7f10dc143490 2023-11-18T20:45:02.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.082+0000 7f10e1abf640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10c806ae40 0x7f10c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.086 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.083+0000 7f10e1abf640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10c806ae40 0x7f10c806d300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f10d0036000 tx=0x7f10d003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:02.087 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.086+0000 7f10daffd640 1 -- 172.21.15.60:0/445454351 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f10d4090080 con 0x7f10dc143490 2023-11-18T20:45:02.298 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.297+0000 7f10e4524640 1 -- 172.21.15.60:0/445454351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10dc158970 con 0x7f10dc143490 2023-11-18T20:45:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.299+0000 7f10daffd640 1 -- 172.21.15.60:0/445454351 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f10dc158970 con 0x7f10dc143490 2023-11-18T20:45:02.300 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:02.300 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1578336464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1578336464},{"type":"v1","addr":"172.21.15.100:6836","nonce":1578336464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/88984334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":88984334},{"type":"v1","addr":"172.21.15.100:6837","nonce":88984334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3777286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3777286992},{"type":"v1","addr":"172.21.15.60:6833","nonce":3777286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-18T20:44:33.070091+0000","modified":"2023-11-18T20:44:35.050014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3098423505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3098423505},{"type":"v1","addr":"172.21.15.60:6835","nonce":3098423505}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-18T20:45:02.300 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 21 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 -- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10c806ae40 msgr2=0x7f10c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10c806ae40 0x7f10c806d300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f10d0036000 tx=0x7f10d003c040 comp rx=0 tx=0).stop 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 -- 172.21.15.60:0/445454351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc143490 msgr2=0x7f10dc159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc143490 0x7f10dc159440 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f10d400e9b0 tx=0x7f10d400ee70 comp rx=0 tx=0).stop 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 -- 172.21.15.60:0/445454351 shutdown_connections 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f10c806ae40 0x7f10c806d300 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10dc15e730 0x7f10dc13f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10dc15a4e0 0x7f10dc13ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 --2- 172.21.15.60:0/445454351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10dc143490 0x7f10dc159440 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 -- 172.21.15.60:0/445454351 >> 172.21.15.60:0/445454351 conn(0x7f10dc06a4d0 msgr2=0x7f10dc070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 -- 172.21.15.60:0/445454351 shutdown_connections 2023-11-18T20:45:02.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.301+0000 7f10e4524640 1 -- 172.21.15.60:0/445454351 wait complete. 2023-11-18T20:45:02.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:45:02.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.419+0000 7f1e7c86f640 1 Processor -- start 2023-11-18T20:45:02.420 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.419+0000 7f1e7c86f640 1 -- start start 2023-11-18T20:45:02.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.419+0000 7f1e7c86f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1e7806abe0 0x7f1e781618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.421 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.419+0000 7f1e7c86f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e781406f0 con 0x7f1e78140310 2023-11-18T20:45:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.420+0000 7f1e7c86f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e78143960 con 0x7f1e781409d0 2023-11-18T20:45:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.420+0000 7f1e7c86f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e78161e20 con 0x7f1e7806abe0 2023-11-18T20:45:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.420+0000 7f1e6ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1e7806abe0 0x7f1e781618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.420+0000 7f1e7659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1e78140310 0x7f1e7806a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45564/0 (socket says 172.21.15.60:45564) 2023-11-18T20:45:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.420+0000 7f1e7659c640 1 -- 172.21.15.60:0/1344717189 learned_addr learned my addr 172.21.15.60:0/1344717189 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.420+0000 7f1e6ffff640 1 -- 172.21.15.60:0/1344717189 >> v1:172.21.15.100:6789/0 conn(0x7f1e781409d0 legacy=0x7f1e7806efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.420+0000 7f1e6ffff640 1 -- 172.21.15.60:0/1344717189 >> v1:172.21.15.60:6789/0 conn(0x7f1e78140310 legacy=0x7f1e7806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.420+0000 7f1e6ffff640 1 -- 172.21.15.60:0/1344717189 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e78142de0 con 0x7f1e7806abe0 2023-11-18T20:45:02.422 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.420+0000 7f1e75d9b640 1 -- 172.21.15.60:0/1344717189 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808350678 0 0) 0x7f1e781406f0 con 0x7f1e78140310 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e6ffff640 1 --2- 172.21.15.60:0/1344717189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 0x7f1e781618e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1e600097a0 tx=0x7f1e6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7804452b60a3046d server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e75d9b640 1 -- 172.21.15.60:0/1344717189 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60031830 con 0x7f1e7806abe0 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e75d9b640 1 -- 172.21.15.60:0/1344717189 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e60031990 con 0x7f1e7806abe0 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e75d9b640 1 -- 172.21.15.60:0/1344717189 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60031d20 con 0x7f1e7806abe0 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e7c86f640 1 -- 172.21.15.60:0/1344717189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 msgr2=0x7f1e781618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e7c86f640 1 --2- 172.21.15.60:0/1344717189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 0x7f1e781618e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1e600097a0 tx=0x7f1e6002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e7c86f640 1 -- 172.21.15.60:0/1344717189 shutdown_connections 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e7c86f640 1 --2- 172.21.15.60:0/1344717189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 0x7f1e781618e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e7c86f640 1 -- 172.21.15.60:0/1344717189 >> 172.21.15.60:0/1344717189 conn(0x7f1e780b1e50 msgr2=0x7f1e780b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e7c86f640 1 -- 172.21.15.60:0/1344717189 shutdown_connections 2023-11-18T20:45:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.421+0000 7f1e7c86f640 1 -- 172.21.15.60:0/1344717189 wait complete. 2023-11-18T20:45:02.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.423+0000 7f1e7c86f640 1 Processor -- start 2023-11-18T20:45:02.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.423+0000 7f1e7c86f640 1 -- start start 2023-11-18T20:45:02.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.423+0000 7f1e7c86f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 0x7f1e780afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.423+0000 7f1e7c86f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e78140310 0x7f1e78155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.423+0000 7f1e7c86f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e781409d0 0x7f1e780b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.423+0000 7f1e7c86f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780b03e0 con 0x7f1e7806abe0 2023-11-18T20:45:02.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.423+0000 7f1e7c86f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780b06c0 con 0x7f1e78140310 2023-11-18T20:45:02.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.423+0000 7f1e7c86f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780b09a0 con 0x7f1e781409d0 2023-11-18T20:45:02.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 0x7f1e780afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 0x7f1e780afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56022/0 (socket says 172.21.15.60:56022) 2023-11-18T20:45:02.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7659c640 1 -- 172.21.15.60:0/739451100 learned_addr learned my addr 172.21.15.60:0/739451100 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e6ffff640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e78140310 0x7f1e78155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e76d9d640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e781409d0 0x7f1e780b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7659c640 1 -- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e781409d0 msgr2=0x7f1e780b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7659c640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e781409d0 0x7f1e780b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7659c640 1 -- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e78140310 msgr2=0x7f1e78155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7659c640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e78140310 0x7f1e78155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7659c640 1 -- 172.21.15.60:0/739451100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60009400 con 0x7f1e7806abe0 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7659c640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 0x7f1e780afdc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800c4a0 tx=0x7f1e6800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e6effd640 1 -- 172.21.15.60:0/739451100 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680074e0 con 0x7f1e7806abe0 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e6effd640 1 -- 172.21.15.60:0/739451100 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68007640 con 0x7f1e7806abe0 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7c86f640 1 -- 172.21.15.60:0/739451100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e780b0c80 con 0x7f1e7806abe0 2023-11-18T20:45:02.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e6effd640 1 -- 172.21.15.60:0/739451100 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68004a00 con 0x7f1e7806abe0 2023-11-18T20:45:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.424+0000 7f1e7c86f640 1 -- 172.21.15.60:0/739451100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e78156130 con 0x7f1e7806abe0 2023-11-18T20:45:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.425+0000 7f1e7c86f640 1 -- 172.21.15.60:0/739451100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e78161030 con 0x7f1e7806abe0 2023-11-18T20:45:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.426+0000 7f1e6effd640 1 -- 172.21.15.60:0/739451100 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1e68004b60 con 0x7f1e7806abe0 2023-11-18T20:45:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.426+0000 7f1e6effd640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1e5806ae40 0x7f1e5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.426+0000 7f1e6effd640 1 -- 172.21.15.60:0/739451100 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e68088aa0 con 0x7f1e7806abe0 2023-11-18T20:45:02.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.427+0000 7f1e6ffff640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1e5806ae40 0x7f1e5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.427+0000 7f1e6ffff640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1e5806ae40 0x7f1e5806d300 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1e60036000 tx=0x7f1e6003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:02.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.431+0000 7f1e6effd640 1 -- 172.21.15.60:0/739451100 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1e68053be0 con 0x7f1e7806abe0 2023-11-18T20:45:02.609 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.607+0000 7f1e7c86f640 1 -- 172.21.15.60:0/739451100 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e78069400 con 0x7f1e7806abe0 2023-11-18T20:45:02.610 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.609+0000 7f1e6effd640 1 -- 172.21.15.60:0/739451100 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f1e68018020 con 0x7f1e7806abe0 2023-11-18T20:45:02.610 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:02.610 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:44:34.103787+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-18T20:44:31.358325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-18T20:44:32.036538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:45:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 -- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1e5806ae40 msgr2=0x7f1e5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1e5806ae40 0x7f1e5806d300 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1e60036000 tx=0x7f1e6003c040 comp rx=0 tx=0).stop 2023-11-18T20:45:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 -- 172.21.15.60:0/739451100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 msgr2=0x7f1e780afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 0x7f1e780afdc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800c4a0 tx=0x7f1e6800c960 comp rx=0 tx=0).stop 2023-11-18T20:45:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 -- 172.21.15.60:0/739451100 shutdown_connections 2023-11-18T20:45:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1e5806ae40 0x7f1e5806d300 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e781409d0 0x7f1e780b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e78140310 0x7f1e78155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 --2- 172.21.15.60:0/739451100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e7806abe0 0x7f1e780afdc0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 -- 172.21.15.60:0/739451100 >> 172.21.15.60:0/739451100 conn(0x7f1e780b1e50 msgr2=0x7f1e78140e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 -- 172.21.15.60:0/739451100 shutdown_connections 2023-11-18T20:45:02.613 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.611+0000 7f1e7c86f640 1 -- 172.21.15.60:0/739451100 wait complete. 2023-11-18T20:45:02.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-18T20:45:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.729+0000 7fb3b510f640 1 Processor -- start 2023-11-18T20:45:02.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.729+0000 7fb3b510f640 1 -- start start 2023-11-18T20:45:02.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.730+0000 7fb3b510f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b00cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.730+0000 7fb3b510f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb3b00d2040 0x7fb3b00cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.730+0000 7fb3b510f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00ce4c0 con 0x7fb3b015d2d0 2023-11-18T20:45:02.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.730+0000 7fb3b510f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b00d1bb0 con 0x7fb3b00d2040 2023-11-18T20:45:02.731 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.730+0000 7fb3b510f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b015d6b0 con 0x7fb3b00b5370 2023-11-18T20:45:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.730+0000 7fb3ae59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b00cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.730+0000 7fb3af59e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb3b00d2040 0x7fb3b00cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.730+0000 7fb3aed9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb3b015d2d0 0x7fb3b00b4c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:40924/0 (socket says 172.21.15.60:40924) 2023-11-18T20:45:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.730+0000 7fb3aed9d640 1 -- 172.21.15.60:0/2386574155 learned_addr learned my addr 172.21.15.60:0/2386574155 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.730+0000 7fb3ae59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b00cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:57300/0 (socket says 172.21.15.60:57300) 2023-11-18T20:45:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.731+0000 7fb3add9b640 1 -- 172.21.15.60:0/2386574155 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950959407 0 0) 0x7fb3a8001110 con 0x7fb3b015d2d0 2023-11-18T20:45:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.731+0000 7fb3add9b640 1 -- 172.21.15.60:0/2386574155 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb39c003520 con 0x7fb3b015d2d0 2023-11-18T20:45:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.731+0000 7fb3ae59c640 1 -- 172.21.15.60:0/2386574155 >> v2:172.21.15.100:3300/0 conn(0x7fb3b00d2040 msgr2=0x7fb3b00cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.731+0000 7fb3ae59c640 1 --2- 172.21.15.60:0/2386574155 >> v2:172.21.15.100:3300/0 conn(0x7fb3b00d2040 0x7fb3b00cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.731+0000 7fb3ae59c640 1 -- 172.21.15.60:0/2386574155 >> v1:172.21.15.100:6789/0 conn(0x7fb3b015d2d0 legacy=0x7fb3b00b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.731+0000 7fb3ae59c640 1 -- 172.21.15.60:0/2386574155 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00ce700 con 0x7fb3b00b5370 2023-11-18T20:45:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.731+0000 7fb3af59e640 1 --2- 172.21.15.60:0/2386574155 >> v2:172.21.15.100:3300/0 conn(0x7fb3b00d2040 0x7fb3b00cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3ae59c640 1 --2- 172.21.15.60:0/2386574155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b00cda40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb3980097a0 tx=0x7fb39802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=909bc4b582b21cd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3add9b640 1 -- 172.21.15.60:0/2386574155 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398031830 con 0x7fb3b00b5370 2023-11-18T20:45:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3add9b640 1 -- 172.21.15.60:0/2386574155 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb398031990 con 0x7fb3b00b5370 2023-11-18T20:45:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3add9b640 1 -- 172.21.15.60:0/2386574155 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398031d20 con 0x7fb3b00b5370 2023-11-18T20:45:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3b510f640 1 -- 172.21.15.60:0/2386574155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 msgr2=0x7fb3b00cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3b510f640 1 --2- 172.21.15.60:0/2386574155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b00cda40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb3980097a0 tx=0x7fb39802b8d0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3b510f640 1 -- 172.21.15.60:0/2386574155 shutdown_connections 2023-11-18T20:45:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3b510f640 1 --2- 172.21.15.60:0/2386574155 >> v2:172.21.15.100:3300/0 conn(0x7fb3b00d2040 0x7fb3b00cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3b510f640 1 --2- 172.21.15.60:0/2386574155 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b00cda40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3b510f640 1 -- 172.21.15.60:0/2386574155 >> 172.21.15.60:0/2386574155 conn(0x7fb3b0072050 msgr2=0x7fb3b0072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3b510f640 1 -- 172.21.15.60:0/2386574155 shutdown_connections 2023-11-18T20:45:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.732+0000 7fb3b510f640 1 -- 172.21.15.60:0/2386574155 wait complete. 2023-11-18T20:45:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.734+0000 7fb3b510f640 1 Processor -- start 2023-11-18T20:45:02.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.734+0000 7fb3b510f640 1 -- start start 2023-11-18T20:45:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3b510f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3b510f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b00d2040 0x7fb3b01e6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3b510f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3b015d2d0 0x7fb3b00cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3b510f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01f2e00 con 0x7fb3b00d2040 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3b510f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b015cc50 con 0x7fb3b015d2d0 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3b510f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00c9390 con 0x7fb3b00b5370 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3ae59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b00d2040 0x7fb3b01e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3ae59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b00d2040 0x7fb3b01e6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56028/0 (socket says 172.21.15.60:56028) 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3ae59c640 1 -- 172.21.15.60:0/2237413646 learned_addr learned my addr 172.21.15.60:0/2237413646 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3aed9d640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3af59e640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3b015d2d0 0x7fb3b00cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3ae59c640 1 -- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 msgr2=0x7fb3b015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3ae59c640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3ae59c640 1 -- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3b015d2d0 msgr2=0x7fb3b00cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3ae59c640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3b015d2d0 0x7fb3b00cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3ae59c640 1 -- 172.21.15.60:0/2237413646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a0009a70 con 0x7fb3b00d2040 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3aed9d640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.735+0000 7fb3ae59c640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b00d2040 0x7fb3b01e6870 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb39802bee0 tx=0x7fb3980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.736+0000 7fb3a77fe640 1 -- 172.21.15.60:0/2237413646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39803a630 con 0x7fb3b00d2040 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.736+0000 7fb3a77fe640 1 -- 172.21.15.60:0/2237413646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39803a7b0 con 0x7fb3b00d2040 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.736+0000 7fb3a77fe640 1 -- 172.21.15.60:0/2237413646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39803ab60 con 0x7fb3b00d2040 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.736+0000 7fb3b510f640 1 -- 172.21.15.60:0/2237413646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb398009400 con 0x7fb3b00d2040 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.736+0000 7fb3b510f640 1 -- 172.21.15.60:0/2237413646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b00c9950 con 0x7fb3b00d2040 2023-11-18T20:45:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.737+0000 7fb3b510f640 1 -- 172.21.15.60:0/2237413646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b0069400 con 0x7fb3b00d2040 2023-11-18T20:45:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.737+0000 7fb3a77fe640 1 -- 172.21.15.60:0/2237413646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb39803acc0 con 0x7fb3b00d2040 2023-11-18T20:45:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.737+0000 7fb3a77fe640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb39006ae40 0x7fb39006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.738+0000 7fb3a77fe640 1 -- 172.21.15.60:0/2237413646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3980aad40 con 0x7fb3b00d2040 2023-11-18T20:45:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.738+0000 7fb3aed9d640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb39006ae40 0x7fb39006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.738+0000 7fb3aed9d640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb39006ae40 0x7fb39006d300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8001270 tx=0x7fb3a8007bb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:02.743 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.742+0000 7fb3a77fe640 1 -- 172.21.15.60:0/2237413646 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb398075ea0 con 0x7fb3b00d2040 2023-11-18T20:45:02.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.954+0000 7fb3b510f640 1 -- 172.21.15.60:0/2237413646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb3b00d3f50 con 0x7fb3b00d2040 2023-11-18T20:45:02.955 INFO:tasks.ceph.mon.a.smithi060.stderr:2023-11-18T20:45:02.955+0000 7fe66ea9a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-18T20:45:02.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.956+0000 7fb3a77fe640 1 -- 172.21.15.60:0/2237413646 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb398035090 con 0x7fb3b00d2040 2023-11-18T20:45:02.957 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-18T20:45:02.960 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.958+0000 7fb3b510f640 1 -- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb39006ae40 msgr2=0x7fb39006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.958+0000 7fb3b510f640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb39006ae40 0x7fb39006d300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8001270 tx=0x7fb3a8007bb0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.959+0000 7fb3b510f640 1 -- 172.21.15.60:0/2237413646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b00d2040 msgr2=0x7fb3b01e6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.959+0000 7fb3b510f640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b00d2040 0x7fb3b01e6870 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb39802bee0 tx=0x7fb3980046c0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.959+0000 7fb3b510f640 1 -- 172.21.15.60:0/2237413646 shutdown_connections 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.959+0000 7fb3b510f640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fb39006ae40 0x7fb39006d300 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.959+0000 7fb3b510f640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3b015d2d0 0x7fb3b00cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.959+0000 7fb3b510f640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3b00d2040 0x7fb3b01e6870 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.959+0000 7fb3b510f640 1 --2- 172.21.15.60:0/2237413646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3b00b5370 0x7fb3b015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.959+0000 7fb3b510f640 1 -- 172.21.15.60:0/2237413646 >> 172.21.15.60:0/2237413646 conn(0x7fb3b0072050 msgr2=0x7fb3b0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.959+0000 7fb3b510f640 1 -- 172.21.15.60:0/2237413646 shutdown_connections 2023-11-18T20:45:02.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:02.959+0000 7fb3b510f640 1 -- 172.21.15.60:0/2237413646 wait complete. 2023-11-18T20:45:02.962 INFO:tasks.ceph.mds.a.smithi060.stderr: -2605> 2023-11-18T20:44:35.684+0000 7f63a7854640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-18T20:45:02.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-18T20:45:02.989 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2023-11-18T20:45:03.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.079+0000 7fcb610bb640 1 Processor -- start 2023-11-18T20:45:03.081 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.080+0000 7fcb610bb640 1 -- start start 2023-11-18T20:45:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.080+0000 7fcb610bb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcb5c160fb0 0x7fcb5c161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.080+0000 7fcb610bb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcb5c0d0ed0 0x7fcb5c0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.080+0000 7fcb610bb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c161d20 con 0x7fcb5c06ea50 2023-11-18T20:45:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.080+0000 7fcb610bb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c15e760 con 0x7fcb5c160fb0 2023-11-18T20:45:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.080+0000 7fcb610bb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c15d9a0 con 0x7fcb5c0d0ed0 2023-11-18T20:45:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.081+0000 7fcb5a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcb5c160fb0 0x7fcb5c161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:03.082 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.081+0000 7fcb5a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcb5c160fb0 0x7fcb5c161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56040/0 (socket says 172.21.15.60:56040) 2023-11-18T20:45:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.081+0000 7fcb5a59c640 1 -- 172.21.15.60:0/1255229368 learned_addr learned my addr 172.21.15.60:0/1255229368 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.081+0000 7fcb5a59c640 1 -- 172.21.15.60:0/1255229368 >> v2:172.21.15.100:3300/0 conn(0x7fcb5c0d0ed0 msgr2=0x7fcb5c0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.081+0000 7fcb5b59e640 1 --2- 172.21.15.60:0/1255229368 >> v2:172.21.15.100:3300/0 conn(0x7fcb5c0d0ed0 0x7fcb5c0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.081+0000 7fcb5a59c640 1 --2- 172.21.15.60:0/1255229368 >> v2:172.21.15.100:3300/0 conn(0x7fcb5c0d0ed0 0x7fcb5c0d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.081+0000 7fcb5a59c640 1 -- 172.21.15.60:0/1255229368 >> v1:172.21.15.100:6789/0 conn(0x7fcb5c06ea50 legacy=0x7fcb5c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:03.083 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.081+0000 7fcb5a59c640 1 -- 172.21.15.60:0/1255229368 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c15dc80 con 0x7fcb5c160fb0 2023-11-18T20:45:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb5a59c640 1 --2- 172.21.15.60:0/1255229368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 0x7fcb5c161440 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0b55a0 tx=0x7fcb5002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9faef532b1d9c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb59d9b640 1 -- 172.21.15.60:0/1255229368 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5002c9f0 con 0x7fcb5c160fb0 2023-11-18T20:45:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb59d9b640 1 -- 172.21.15.60:0/1255229368 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5002cb50 con 0x7fcb5c160fb0 2023-11-18T20:45:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb59d9b640 1 -- 172.21.15.60:0/1255229368 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500343f0 con 0x7fcb5c160fb0 2023-11-18T20:45:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb610bb640 1 -- 172.21.15.60:0/1255229368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 msgr2=0x7fcb5c161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:03.084 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb610bb640 1 --2- 172.21.15.60:0/1255229368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 0x7fcb5c161440 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0b55a0 tx=0x7fcb5002be50 comp rx=0 tx=0).stop 2023-11-18T20:45:03.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb610bb640 1 -- 172.21.15.60:0/1255229368 shutdown_connections 2023-11-18T20:45:03.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb610bb640 1 --2- 172.21.15.60:0/1255229368 >> v2:172.21.15.100:3300/0 conn(0x7fcb5c0d0ed0 0x7fcb5c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:03.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb610bb640 1 --2- 172.21.15.60:0/1255229368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 0x7fcb5c161440 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:03.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb610bb640 1 -- 172.21.15.60:0/1255229368 >> 172.21.15.60:0/1255229368 conn(0x7fcb5c06a6b0 msgr2=0x7fcb5c06aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:03.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb610bb640 1 -- 172.21.15.60:0/1255229368 shutdown_connections 2023-11-18T20:45:03.085 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.082+0000 7fcb610bb640 1 -- 172.21.15.60:0/1255229368 wait complete. 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.084+0000 7fcb610bb640 1 Processor -- start 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.084+0000 7fcb610bb640 1 -- start start 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb610bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb5c06ea50 0x7fcb5c1601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb610bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c0d0ed0 0x7fcb5c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb610bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 0x7fcb5c141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb610bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c1453a0 con 0x7fcb5c160fb0 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb610bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c15eda0 con 0x7fcb5c0d0ed0 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb610bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c15f080 con 0x7fcb5c06ea50 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 0x7fcb5c141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 0x7fcb5c141f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56052/0 (socket says 172.21.15.60:56052) 2023-11-18T20:45:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5b59e640 1 -- 172.21.15.60:0/1079488490 learned_addr learned my addr 172.21.15.60:0/1079488490 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5a59c640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c0d0ed0 0x7fcb5c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5ad9d640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb5c06ea50 0x7fcb5c1601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5b59e640 1 -- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb5c06ea50 msgr2=0x7fcb5c1601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5b59e640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb5c06ea50 0x7fcb5c1601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5b59e640 1 -- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c0d0ed0 msgr2=0x7fcb5c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5b59e640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c0d0ed0 0x7fcb5c15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5b59e640 1 -- 172.21.15.60:0/1079488490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50009b30 con 0x7fcb5c160fb0 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5a59c640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c0d0ed0 0x7fcb5c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb5b59e640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 0x7fcb5c141f40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c540 tx=0x7fcb5400ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb437fe640 1 -- 172.21.15.60:0/1079488490 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54004220 con 0x7fcb5c160fb0 2023-11-18T20:45:03.089 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb437fe640 1 -- 172.21.15.60:0/1079488490 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54004380 con 0x7fcb5c160fb0 2023-11-18T20:45:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.085+0000 7fcb610bb640 1 -- 172.21.15.60:0/1079488490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c15f360 con 0x7fcb5c160fb0 2023-11-18T20:45:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.086+0000 7fcb437fe640 1 -- 172.21.15.60:0/1079488490 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54003850 con 0x7fcb5c160fb0 2023-11-18T20:45:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.086+0000 7fcb610bb640 1 -- 172.21.15.60:0/1079488490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c160700 con 0x7fcb5c160fb0 2023-11-18T20:45:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.086+0000 7fcb610bb640 1 -- 172.21.15.60:0/1079488490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c145550 con 0x7fcb5c160fb0 2023-11-18T20:45:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.087+0000 7fcb437fe640 1 -- 172.21.15.60:0/1079488490 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb540039b0 con 0x7fcb5c160fb0 2023-11-18T20:45:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.087+0000 7fcb437fe640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcb4806ae40 0x7fcb4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.088+0000 7fcb437fe640 1 -- 172.21.15.60:0/1079488490 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcb54089380 con 0x7fcb5c160fb0 2023-11-18T20:45:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.088+0000 7fcb5ad9d640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcb4806ae40 0x7fcb4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.088+0000 7fcb5ad9d640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcb4806ae40 0x7fcb4806d300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c006fd0 tx=0x7fcb4c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:03.093 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.092+0000 7fcb437fe640 1 -- 172.21.15.60:0/1079488490 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb5c145550 con 0x7fcb5c160fb0 2023-11-18T20:45:03.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.307+0000 7fcb610bb640 1 -- 172.21.15.60:0/1079488490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcb5c145550 con 0x7fcb5c160fb0 2023-11-18T20:45:03.959 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.957+0000 7fcb437fe640 1 -- 172.21.15.60:0/1079488490 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcb5c145550 con 0x7fcb5c160fb0 2023-11-18T20:45:03.961 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 -- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcb4806ae40 msgr2=0x7fcb4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcb4806ae40 0x7fcb4806d300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c006fd0 tx=0x7fcb4c008040 comp rx=0 tx=0).stop 2023-11-18T20:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 -- 172.21.15.60:0/1079488490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 msgr2=0x7fcb5c141f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 0x7fcb5c141f40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c540 tx=0x7fcb5400ca00 comp rx=0 tx=0).stop 2023-11-18T20:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 -- 172.21.15.60:0/1079488490 shutdown_connections 2023-11-18T20:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fcb4806ae40 0x7fcb4806d300 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb5c160fb0 0x7fcb5c141f40 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c0d0ed0 0x7fcb5c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 --2- 172.21.15.60:0/1079488490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb5c06ea50 0x7fcb5c1601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 -- 172.21.15.60:0/1079488490 >> 172.21.15.60:0/1079488490 conn(0x7fcb5c06a6b0 msgr2=0x7fcb5c0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:03.962 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 -- 172.21.15.60:0/1079488490 shutdown_connections 2023-11-18T20:45:03.963 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:03.960+0000 7fcb610bb640 1 -- 172.21.15.60:0/1079488490 wait complete. 2023-11-18T20:45:03.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-18T20:45:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.132+0000 7f28082cf640 1 Processor -- start 2023-11-18T20:45:04.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.132+0000 7f28082cf640 1 -- start start 2023-11-18T20:45:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f28082cf640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2800144980 0x7f280015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:04.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f28082cf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 0x7f28000cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f28082cf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2800071770 con 0x7f28000b4370 2023-11-18T20:45:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f28082cf640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28000ccdc0 con 0x7f2800144980 2023-11-18T20:45:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f28082cf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28000cd0a0 con 0x7f28000ceb70 2023-11-18T20:45:04.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f280606b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2800144980 0x7f280015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f280586a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 0x7f28000cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f280606b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2800144980 0x7f280015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37016/0 (socket says 172.21.15.60:37016) 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f280606b640 1 -- 172.21.15.60:0/1633686661 learned_addr learned my addr 172.21.15.60:0/1633686661 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f280686c640 1 --1- 172.21.15.60:0/1633686661 >> v1:172.21.15.100:6789/0 conn(0x7f28000b4370 0x7f2800071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:40948/0 (socket says 172.21.15.60:40948) 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f280606b640 1 -- 172.21.15.60:0/1633686661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 msgr2=0x7f28000cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.133+0000 7f280606b640 1 --2- 172.21.15.60:0/1633686661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 0x7f28000cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.134+0000 7f280606b640 1 -- 172.21.15.60:0/1633686661 >> v1:172.21.15.100:6789/0 conn(0x7f28000b4370 legacy=0x7f2800071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.134+0000 7f280606b640 1 -- 172.21.15.60:0/1633686661 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28000cd380 con 0x7f2800144980 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.134+0000 7f2805069640 1 -- 172.21.15.60:0/1633686661 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244322206 0 0) 0x7f2800071770 con 0x7f28000b4370 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.134+0000 7f280586a640 1 --2- 172.21.15.60:0/1633686661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 0x7f28000cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:04.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.134+0000 7f280606b640 1 --2- 172.21.15.60:0/1633686661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2800144980 0x7f280015c7c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009870 tx=0x7f27fc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38dcd357705ccd server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f2805069640 1 -- 172.21.15.60:0/1633686661 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc030840 con 0x7f2800144980 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f2805069640 1 -- 172.21.15.60:0/1633686661 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc0309a0 con 0x7f2800144980 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f2805069640 1 -- 172.21.15.60:0/1633686661 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc030d30 con 0x7f2800144980 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f28082cf640 1 -- 172.21.15.60:0/1633686661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2800144980 msgr2=0x7f280015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f28082cf640 1 --2- 172.21.15.60:0/1633686661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2800144980 0x7f280015c7c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009870 tx=0x7f27fc02b980 comp rx=0 tx=0).stop 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f28082cf640 1 -- 172.21.15.60:0/1633686661 shutdown_connections 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f28082cf640 1 --2- 172.21.15.60:0/1633686661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 0x7f28000cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f28082cf640 1 --2- 172.21.15.60:0/1633686661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2800144980 0x7f280015c7c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f28082cf640 1 -- 172.21.15.60:0/1633686661 >> 172.21.15.60:0/1633686661 conn(0x7f280006a4d0 msgr2=0x7f280006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f28082cf640 1 -- 172.21.15.60:0/1633686661 shutdown_connections 2023-11-18T20:45:04.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.135+0000 7f28082cf640 1 -- 172.21.15.60:0/1633686661 wait complete. 2023-11-18T20:45:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.136+0000 7f28082cf640 1 Processor -- start 2023-11-18T20:45:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.136+0000 7f28082cf640 1 -- start start 2023-11-18T20:45:04.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.137+0000 7f28082cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28000b4370 0x7f28001396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.137+0000 7f28082cf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 0x7f28000ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.137+0000 7f28082cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2800144980 0x7f28000ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.137+0000 7f28082cf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28001f1040 con 0x7f28000b4370 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.137+0000 7f28082cf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000a8b90 con 0x7f2800144980 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.137+0000 7f28082cf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28000a8e70 con 0x7f28000ceb70 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.137+0000 7f280606b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28000b4370 0x7f28001396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.137+0000 7f280606b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28000b4370 0x7f28001396a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56062/0 (socket says 172.21.15.60:56062) 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.137+0000 7f280606b640 1 -- 172.21.15.60:0/2255255394 learned_addr learned my addr 172.21.15.60:0/2255255394 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.137+0000 7f280686c640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2800144980 0x7f28000ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f280586a640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 0x7f28000ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f280606b640 1 -- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 msgr2=0x7f28000ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f280606b640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 0x7f28000ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:04.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f280606b640 1 -- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2800144980 msgr2=0x7f28000ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:04.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f280606b640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2800144980 0x7f28000ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:04.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f280606b640 1 -- 172.21.15.60:0/2255255394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f0009400 con 0x7f28000b4370 2023-11-18T20:45:04.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f280606b640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28000b4370 0x7f28001396a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009870 tx=0x7f27fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:04.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f27f6ffd640 1 -- 172.21.15.60:0/2255255394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc03e070 con 0x7f28000b4370 2023-11-18T20:45:04.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f27f6ffd640 1 -- 172.21.15.60:0/2255255394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc004030 con 0x7f28000b4370 2023-11-18T20:45:04.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f28082cf640 1 -- 172.21.15.60:0/2255255394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27fc0094d0 con 0x7f28000b4370 2023-11-18T20:45:04.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f27f6ffd640 1 -- 172.21.15.60:0/2255255394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0043e0 con 0x7f28000b4370 2023-11-18T20:45:04.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.138+0000 7f28082cf640 1 -- 172.21.15.60:0/2255255394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28000a9430 con 0x7f28000b4370 2023-11-18T20:45:04.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.139+0000 7f28082cf640 1 -- 172.21.15.60:0/2255255394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28000a9620 con 0x7f28000b4370 2023-11-18T20:45:04.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.140+0000 7f27f6ffd640 1 -- 172.21.15.60:0/2255255394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27fc042070 con 0x7f28000b4370 2023-11-18T20:45:04.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.140+0000 7f27f6ffd640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f27e406ae40 0x7f27e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:04.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.140+0000 7f27f6ffd640 1 -- 172.21.15.60:0/2255255394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f27fc0ab3f0 con 0x7f28000b4370 2023-11-18T20:45:04.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.140+0000 7f280586a640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f27e406ae40 0x7f27e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:04.143 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.141+0000 7f280586a640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f27e406ae40 0x7f27e406d300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f280015e4c0 tx=0x7f27f0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:04.145 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.145+0000 7f27f6ffd640 1 -- 172.21.15.60:0/2255255394 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27fc0763c0 con 0x7f28000b4370 2023-11-18T20:45:04.324 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:04.323+0000 7f28082cf640 1 -- 172.21.15.60:0/2255255394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2800069400 con 0x7f28000b4370 2023-11-18T20:45:05.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.311+0000 7f27f6ffd640 1 -- 172.21.15.60:0/2255255394 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7f27fc0799e0 con 0x7f28000b4370 2023-11-18T20:45:05.313 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2023-11-18T20:45:05.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.317+0000 7f28082cf640 1 -- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f27e406ae40 msgr2=0x7f27e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:05.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.317+0000 7f28082cf640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f27e406ae40 0x7f27e406d300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f280015e4c0 tx=0x7f27f0009190 comp rx=0 tx=0).stop 2023-11-18T20:45:05.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.317+0000 7f28082cf640 1 -- 172.21.15.60:0/2255255394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28000b4370 msgr2=0x7f28001396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:05.319 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.317+0000 7f28082cf640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28000b4370 0x7f28001396a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009870 tx=0x7f27fc0046c0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.317+0000 7f28082cf640 1 -- 172.21.15.60:0/2255255394 shutdown_connections 2023-11-18T20:45:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.317+0000 7f28082cf640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f27e406ae40 0x7f27e406d300 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.317+0000 7f28082cf640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2800144980 0x7f28000ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.317+0000 7f28082cf640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28000ceb70 0x7f28000ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.317+0000 7f28082cf640 1 --2- 172.21.15.60:0/2255255394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28000b4370 0x7f28001396a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.317+0000 7f28082cf640 1 -- 172.21.15.60:0/2255255394 >> 172.21.15.60:0/2255255394 conn(0x7f280006a4d0 msgr2=0x7f2800159830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.318+0000 7f28082cf640 1 -- 172.21.15.60:0/2255255394 shutdown_connections 2023-11-18T20:45:05.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.318+0000 7f28082cf640 1 -- 172.21.15.60:0/2255255394 wait complete. 2023-11-18T20:45:05.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-18T20:45:05.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.449+0000 7f46507fc640 1 Processor -- start 2023-11-18T20:45:05.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.449+0000 7f46507fc640 1 -- start start 2023-11-18T20:45:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.450+0000 7f46507fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f46480d3900 0x7f46480b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.450+0000 7f46507fc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46480b2b10 con 0x7f464815d330 2023-11-18T20:45:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.450+0000 7f46507fc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46480d1590 con 0x7f46480d2910 2023-11-18T20:45:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.450+0000 7f46507fc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46480d1870 con 0x7f46480d3900 2023-11-18T20:45:05.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464e598640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f464815d330 0x7f46480b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45572/0 (socket says 172.21.15.60:45572) 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464e598640 1 -- 172.21.15.60:0/2655414826 learned_addr learned my addr 172.21.15.60:0/2655414826 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464ed99640 1 --2- 172.21.15.60:0/2655414826 >> v2:172.21.15.100:3300/0 conn(0x7f46480d3900 0x7f46480b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160062737 0 0) 0x7f46480b2b10 con 0x7f464815d330 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4634003520 con 0x7f464815d330 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1204613460 0 0) 0x7f4634003520 con 0x7f464815d330 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 >> v2:172.21.15.100:3300/0 conn(0x7f46480d3900 msgr2=0x7f46480b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 --2- 172.21.15.60:0/2655414826 >> v2:172.21.15.100:3300/0 conn(0x7f46480d3900 0x7f46480b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 >> v1:172.21.15.100:6789/0 conn(0x7f46480d2910 legacy=0x7f46480d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46480d1b50 con 0x7f464815d330 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f4644001670 con 0x7f464815d330 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 >> v1:172.21.15.60:6789/0 conn(0x7f464815d330 legacy=0x7f46480b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4634003970 0x7f4634003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46340047c0 0x7f4634004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4634005180 0x7f463400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480d1870 con 0x7f4634003970 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463400c120 con 0x7f4634005180 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463400c2e0 con 0x7f46340047c0 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464e598640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4634003970 0x7f4634003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464dd97640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46340047c0 0x7f4634004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.451+0000 7f464ed99640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4634005180 0x7f463400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.452+0000 7f464e598640 1 -- 172.21.15.60:0/2655414826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46340047c0 msgr2=0x7f4634004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.452+0000 7f464e598640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46340047c0 0x7f4634004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.452+0000 7f464e598640 1 -- 172.21.15.60:0/2655414826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4634005180 msgr2=0x7f463400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.452+0000 7f464e598640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4634005180 0x7f463400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.452+0000 7f464ed99640 1 -- 172.21.15.60:0/2655414826 reap_dead start 2023-11-18T20:45:05.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.452+0000 7f464e598640 1 -- 172.21.15.60:0/2655414826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634003520 con 0x7f4634003970 2023-11-18T20:45:05.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.452+0000 7f464e598640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4634003970 0x7f4634003d90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f464400c240 tx=0x7f46340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12905247b7d2f582 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:05.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.453+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644005ad0 con 0x7f4634003970 2023-11-18T20:45:05.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.453+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644005c30 con 0x7f4634003970 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.453+0000 7f464d596640 1 -- 172.21.15.60:0/2655414826 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464402ca90 con 0x7f4634003970 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.453+0000 7f46507fc640 1 -- 172.21.15.60:0/2655414826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4634003970 msgr2=0x7f4634003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.453+0000 7f46507fc640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4634003970 0x7f4634003d90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f464400c240 tx=0x7f46340057f0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.453+0000 7f46507fc640 1 -- 172.21.15.60:0/2655414826 shutdown_connections 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.453+0000 7f46507fc640 1 --2- 172.21.15.60:0/2655414826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4634003970 0x7f4634003d90 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.453+0000 7f46507fc640 1 -- 172.21.15.60:0/2655414826 >> 172.21.15.60:0/2655414826 conn(0x7f4648072020 msgr2=0x7f4648072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.453+0000 7f46507fc640 1 -- 172.21.15.60:0/2655414826 shutdown_connections 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.453+0000 7f46507fc640 1 -- 172.21.15.60:0/2655414826 wait complete. 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.454+0000 7f46507fc640 1 Processor -- start 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f46507fc640 1 -- start start 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f46507fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4634003970 0x7f46481572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f46507fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46480cd000 0x7f46481f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f46507fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4648158e50 0x7f46481592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f46507fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4648159c30 con 0x7f4648158e50 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f46507fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46480c8c00 con 0x7f4634003970 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f46507fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46481579b0 con 0x7f46480cd000 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f464ed99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4648158e50 0x7f46481592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f464ed99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4648158e50 0x7f46481592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56092/0 (socket says 172.21.15.60:56092) 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f464ed99640 1 -- 172.21.15.60:0/1920670998 learned_addr learned my addr 172.21.15.60:0/1920670998 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f464e598640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4634003970 0x7f46481572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.455+0000 7f464dd97640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46480cd000 0x7f46481f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f464ed99640 1 -- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46480cd000 msgr2=0x7f46481f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f464ed99640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46480cd000 0x7f46481f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f464ed99640 1 -- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4634003970 msgr2=0x7f46481572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:05.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f464ed99640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4634003970 0x7f46481572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:05.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f464ed99640 1 -- 172.21.15.60:0/1920670998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46480c8ea0 con 0x7f4648158e50 2023-11-18T20:45:05.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f464ed99640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4648158e50 0x7f46481592d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f46480d1510 tx=0x7f464000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:05.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f463effd640 1 -- 172.21.15.60:0/1920670998 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464000e040 con 0x7f4648158e50 2023-11-18T20:45:05.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f463effd640 1 -- 172.21.15.60:0/1920670998 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464000f040 con 0x7f4648158e50 2023-11-18T20:45:05.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f46507fc640 1 -- 172.21.15.60:0/1920670998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46480c9180 con 0x7f4648158e50 2023-11-18T20:45:05.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f463effd640 1 -- 172.21.15.60:0/1920670998 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640015430 con 0x7f4648158e50 2023-11-18T20:45:05.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.456+0000 7f46507fc640 1 -- 172.21.15.60:0/1920670998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4648159810 con 0x7f4648158e50 2023-11-18T20:45:05.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.457+0000 7f46507fc640 1 -- 172.21.15.60:0/1920670998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46480c96a0 con 0x7f4648158e50 2023-11-18T20:45:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.458+0000 7f463effd640 1 -- 172.21.15.60:0/1920670998 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4640015590 con 0x7f4648158e50 2023-11-18T20:45:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.458+0000 7f463effd640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f462c06ae70 0x7f462c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.458+0000 7f463effd640 1 -- 172.21.15.60:0/1920670998 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f4640089a80 con 0x7f4648158e50 2023-11-18T20:45:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.458+0000 7f464e598640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f462c06ae70 0x7f462c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.459+0000 7f464e598640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f462c06ae70 0x7f462c06d330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4644034000 tx=0x7f4644009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:05.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.462+0000 7f463effd640 1 -- 172.21.15.60:0/1920670998 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f46480c96a0 con 0x7f4648158e50 2023-11-18T20:45:05.639 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:05.638+0000 7f46507fc640 1 -- 172.21.15.60:0/1920670998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f46480c96a0 con 0x7f4648158e50 2023-11-18T20:45:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.315+0000 7f463effd640 1 -- 172.21.15.60:0/1920670998 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7f46480c96a0 con 0x7f4648158e50 2023-11-18T20:45:06.317 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2023-11-18T20:45:06.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.319+0000 7f46507fc640 1 -- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f462c06ae70 msgr2=0x7f462c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.319+0000 7f46507fc640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f462c06ae70 0x7f462c06d330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4644034000 tx=0x7f4644009030 comp rx=0 tx=0).stop 2023-11-18T20:45:06.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.319+0000 7f46507fc640 1 -- 172.21.15.60:0/1920670998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4648158e50 msgr2=0x7f46481592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.319+0000 7f46507fc640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4648158e50 0x7f46481592d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f46480d1510 tx=0x7f464000f580 comp rx=0 tx=0).stop 2023-11-18T20:45:06.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.320+0000 7f46507fc640 1 -- 172.21.15.60:0/1920670998 shutdown_connections 2023-11-18T20:45:06.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.320+0000 7f46507fc640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f462c06ae70 0x7f462c06d330 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.320+0000 7f46507fc640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4648158e50 0x7f46481592d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.320+0000 7f46507fc640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46480cd000 0x7f46481f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.320+0000 7f46507fc640 1 --2- 172.21.15.60:0/1920670998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4634003970 0x7f46481572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.320+0000 7f46507fc640 1 -- 172.21.15.60:0/1920670998 >> 172.21.15.60:0/1920670998 conn(0x7f4648072020 msgr2=0x7f464815e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:06.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.320+0000 7f46507fc640 1 -- 172.21.15.60:0/1920670998 shutdown_connections 2023-11-18T20:45:06.322 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.320+0000 7f46507fc640 1 -- 172.21.15.60:0/1920670998 wait complete. 2023-11-18T20:45:06.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-18T20:45:06.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.448+0000 7fd2c7af7640 1 Processor -- start 2023-11-18T20:45:06.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.448+0000 7fd2c7af7640 1 -- start start 2023-11-18T20:45:06.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c7af7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd2c006ea50 0x7fd2c006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c7af7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c00d0ed0 0x7fd2c00d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c7af7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c0161d20 con 0x7fd2c0160fb0 2023-11-18T20:45:06.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c7af7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c006f370 con 0x7fd2c006ea50 2023-11-18T20:45:06.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c7af7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c015e760 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c5092640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd2c0160fb0 0x7fd2c0161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45586/0 (socket says 172.21.15.60:45586) 2023-11-18T20:45:06.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c5893640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd2c006ea50 0x7fd2c006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c5092640 1 -- 172.21.15.60:0/819597509 learned_addr learned my addr 172.21.15.60:0/819597509 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:06.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c4891640 1 -- 172.21.15.60:0/819597509 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628613707 0 0) 0x7fd2c0161d20 con 0x7fd2c0160fb0 2023-11-18T20:45:06.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c6094640 1 --2- 172.21.15.60:0/819597509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c00d0ed0 0x7fd2c00d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c4891640 1 -- 172.21.15.60:0/819597509 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ac003520 con 0x7fd2c0160fb0 2023-11-18T20:45:06.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c5893640 1 -- 172.21.15.60:0/819597509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c00d0ed0 msgr2=0x7fd2c00d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:06.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c5893640 1 --2- 172.21.15.60:0/819597509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c00d0ed0 0x7fd2c00d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c5893640 1 -- 172.21.15.60:0/819597509 >> v1:172.21.15.60:6789/0 conn(0x7fd2c0160fb0 legacy=0x7fd2c0161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c5893640 1 -- 172.21.15.60:0/819597509 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c015d9a0 con 0x7fd2c006ea50 2023-11-18T20:45:06.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.449+0000 7fd2c6094640 1 --2- 172.21.15.60:0/819597509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c00d0ed0 0x7fd2c00d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:45:06.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.450+0000 7fd2c5893640 1 --2- 172.21.15.60:0/819597509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c006ea50 0x7fd2c006ee30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009d90 tx=0x7fd2b802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc37cbff1587d6ad server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:06.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.450+0000 7fd2c4891640 1 -- 172.21.15.60:0/819597509 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b802cc50 con 0x7fd2c006ea50 2023-11-18T20:45:06.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.450+0000 7fd2c4891640 1 -- 172.21.15.60:0/819597509 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b802cdb0 con 0x7fd2c006ea50 2023-11-18T20:45:06.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.450+0000 7fd2c4891640 1 -- 172.21.15.60:0/819597509 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8034660 con 0x7fd2c006ea50 2023-11-18T20:45:06.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.451+0000 7fd2c7af7640 1 -- 172.21.15.60:0/819597509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c006ea50 msgr2=0x7fd2c006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.451+0000 7fd2c7af7640 1 --2- 172.21.15.60:0/819597509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c006ea50 0x7fd2c006ee30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009d90 tx=0x7fd2b802bea0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.451+0000 7fd2c7af7640 1 -- 172.21.15.60:0/819597509 shutdown_connections 2023-11-18T20:45:06.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.451+0000 7fd2c7af7640 1 --2- 172.21.15.60:0/819597509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c00d0ed0 0x7fd2c00d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.451+0000 7fd2c7af7640 1 --2- 172.21.15.60:0/819597509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c006ea50 0x7fd2c006ee30 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.451+0000 7fd2c7af7640 1 -- 172.21.15.60:0/819597509 >> 172.21.15.60:0/819597509 conn(0x7fd2c006a6b0 msgr2=0x7fd2c006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.451+0000 7fd2c7af7640 1 -- 172.21.15.60:0/819597509 shutdown_connections 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.451+0000 7fd2c7af7640 1 -- 172.21.15.60:0/819597509 wait complete. 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.452+0000 7fd2c7af7640 1 Processor -- start 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.452+0000 7fd2c7af7640 1 -- start start 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c7af7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c006ea50 0x7fd2c00b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c7af7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c00d0ed0 0x7fd2c00cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c7af7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2c0160fb0 0x7fd2c013ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c7af7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01f6ea0 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c7af7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00cbd60 con 0x7fd2c0160fb0 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c7af7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01f6260 con 0x7fd2c006ea50 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c5092640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c00d0ed0 0x7fd2c00cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c5092640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c00d0ed0 0x7fd2c00cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56110/0 (socket says 172.21.15.60:56110) 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c5092640 1 -- 172.21.15.60:0/2844470324 learned_addr learned my addr 172.21.15.60:0/2844470324 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c6094640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2c0160fb0 0x7fd2c013ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c5893640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c006ea50 0x7fd2c00b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c5092640 1 -- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c006ea50 msgr2=0x7fd2c00b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c5092640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c006ea50 0x7fd2c00b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c5092640 1 -- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2c0160fb0 msgr2=0x7fd2c013ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c5092640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2c0160fb0 0x7fd2c013ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c5092640 1 -- 172.21.15.60:0/2844470324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b8009a70 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.453+0000 7fd2c5092640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c00d0ed0 0x7fd2c00cfc50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000c500 tx=0x7fd2b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:06.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.454+0000 7fd2b67fc640 1 -- 172.21.15.60:0/2844470324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0005ce0 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.454+0000 7fd2b67fc640 1 -- 172.21.15.60:0/2844470324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b0005e40 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.454+0000 7fd2c7af7640 1 -- 172.21.15.60:0/2844470324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c01f6500 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.454+0000 7fd2b67fc640 1 -- 172.21.15.60:0/2844470324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0007330 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.454+0000 7fd2c7af7640 1 -- 172.21.15.60:0/2844470324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c01f6ac0 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.455+0000 7fd2c7af7640 1 -- 172.21.15.60:0/2844470324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c01f6fe0 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.455+0000 7fd2b67fc640 1 -- 172.21.15.60:0/2844470324 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2b0007490 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.455+0000 7fd2b67fc640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd2a406ae40 0x7fd2a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.456+0000 7fd2b67fc640 1 -- 172.21.15.60:0/2844470324 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd2b0088950 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.456+0000 7fd2c5893640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd2a406ae40 0x7fd2a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.456+0000 7fd2c5893640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd2a406ae40 0x7fd2a406d300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8036000 tx=0x7fd2b8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:06.461 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.460+0000 7fd2b67fc640 1 -- 172.21.15.60:0/2844470324 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2c01f6fe0 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.640+0000 7fd2c7af7640 1 -- 172.21.15.60:0/2844470324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd2c01f6fe0 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.644 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.643+0000 7fd2b67fc640 1 -- 172.21.15.60:0/2844470324 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd2c01f6fe0 con 0x7fd2c00d0ed0 2023-11-18T20:45:06.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.645+0000 7fd2c7af7640 1 -- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd2a406ae40 msgr2=0x7fd2a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.645+0000 7fd2c7af7640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd2a406ae40 0x7fd2a406d300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8036000 tx=0x7fd2b8030000 comp rx=0 tx=0).stop 2023-11-18T20:45:06.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.645+0000 7fd2c7af7640 1 -- 172.21.15.60:0/2844470324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c00d0ed0 msgr2=0x7fd2c00cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.645+0000 7fd2c7af7640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c00d0ed0 0x7fd2c00cfc50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000c500 tx=0x7fd2b000c9c0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.646+0000 7fd2c7af7640 1 -- 172.21.15.60:0/2844470324 shutdown_connections 2023-11-18T20:45:06.647 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.646+0000 7fd2c7af7640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fd2a406ae40 0x7fd2a406d300 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.646+0000 7fd2c7af7640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2c0160fb0 0x7fd2c013ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.646+0000 7fd2c7af7640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2c00d0ed0 0x7fd2c00cfc50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.646+0000 7fd2c7af7640 1 --2- 172.21.15.60:0/2844470324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2c006ea50 0x7fd2c00b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.646+0000 7fd2c7af7640 1 -- 172.21.15.60:0/2844470324 >> 172.21.15.60:0/2844470324 conn(0x7fd2c006a6b0 msgr2=0x7fd2c006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:06.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.646+0000 7fd2c7af7640 1 -- 172.21.15.60:0/2844470324 shutdown_connections 2023-11-18T20:45:06.648 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.646+0000 7fd2c7af7640 1 -- 172.21.15.60:0/2844470324 wait complete. 2023-11-18T20:45:06.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-11-18T20:45:06.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.762+0000 7f420c875640 1 Processor -- start 2023-11-18T20:45:06.764 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.762+0000 7f420c875640 1 -- start start 2023-11-18T20:45:06.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.763+0000 7f420c875640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080d2910 0x7f42080d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.763+0000 7f420c875640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42080b2b10 con 0x7f42080d3730 2023-11-18T20:45:06.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.763+0000 7f420c875640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42080d3b50 con 0x7f420815d330 2023-11-18T20:45:06.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.763+0000 7f420c875640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42080d1590 con 0x7f42080d2910 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.763+0000 7f4206d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f42080d3730 0x7f42080b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45602/0 (socket says 172.21.15.60:45602) 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.763+0000 7f4206d9d640 1 -- 172.21.15.60:0/803851369 learned_addr learned my addr 172.21.15.60:0/803851369 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f4205d9b640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080d2910 0x7f42080d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420258368 0 0) 0x7f42080b2b10 con 0x7f42080d3730 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f4003520 con 0x7f42080d3730 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583331518 0 0) 0x7f42080d3b50 con 0x7f420815d330 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42080b2b10 con 0x7f420815d330 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2594310254 0 0) 0x7f41f4003520 con 0x7f42080d3730 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080d2910 msgr2=0x7f42080d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080d2910 0x7f42080d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 >> v1:172.21.15.100:6789/0 conn(0x7f420815d330 legacy=0x7f42080b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42080d1870 con 0x7f42080d3730 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f41f80019f0 con 0x7f42080d3730 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 >> v1:172.21.15.60:6789/0 conn(0x7f42080d3730 legacy=0x7f42080b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080d2910 0x7f41f4003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 0x7f41f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41f4004d00 0x7f41f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42080d1590 con 0x7f41f4004090 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f400bb10 con 0x7f41f4004d00 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f400bca0 con 0x7f42080d2910 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f4205d9b640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 0x7f41f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f420659c640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080d2910 0x7f41f4003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f4206d9d640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41f4004d00 0x7f41f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f4205d9b640 1 -- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080d2910 msgr2=0x7f41f4003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f4205d9b640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080d2910 0x7f41f4003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f4205d9b640 1 -- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41f4004d00 msgr2=0x7f41f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f4205d9b640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41f4004d00 0x7f41f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f4205d9b640 1 -- 172.21.15.60:0/803851369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f4003520 con 0x7f41f4004090 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.764+0000 7f4206d9d640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41f4004d00 0x7f41f4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.765+0000 7f4205d9b640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 0x7f41f40047c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009ac0 tx=0x7f41f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d99bc921a2b064 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.765+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f002c6b0 con 0x7f41f4004090 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.765+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f002c810 con 0x7f41f4004090 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.765+0000 7f420559a640 1 -- 172.21.15.60:0/803851369 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f002cba0 con 0x7f41f4004090 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.766+0000 7f420c875640 1 -- 172.21.15.60:0/803851369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 msgr2=0x7f41f40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.766+0000 7f420c875640 1 --2- 172.21.15.60:0/803851369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 0x7f41f40047c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009ac0 tx=0x7f41f40057f0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.766+0000 7f4206d9d640 1 -- 172.21.15.60:0/803851369 reap_dead start 2023-11-18T20:45:06.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.766+0000 7f420c875640 1 -- 172.21.15.60:0/803851369 shutdown_connections 2023-11-18T20:45:06.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.766+0000 7f420c875640 1 -- 172.21.15.60:0/803851369 >> 172.21.15.60:0/803851369 conn(0x7f4208072020 msgr2=0x7f4208072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:06.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.766+0000 7f420c875640 1 -- 172.21.15.60:0/803851369 shutdown_connections 2023-11-18T20:45:06.771 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.766+0000 7f420c875640 1 -- 172.21.15.60:0/803851369 wait complete. 2023-11-18T20:45:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.767+0000 7f420c875640 1 Processor -- start 2023-11-18T20:45:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.767+0000 7f420c875640 1 -- start start 2023-11-18T20:45:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.768+0000 7f420c875640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 0x7f4208159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.768+0000 7f420c875640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4208156a70 0x7f4208156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.768+0000 7f420c875640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080c7bf0 0x7f42080c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.768+0000 7f420c875640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4208157410 con 0x7f41f4004090 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.768+0000 7f420c875640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42081592c0 con 0x7f4208156a70 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.768+0000 7f420c875640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4208146240 con 0x7f42080c7bf0 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.768+0000 7f420659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 0x7f4208159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.768+0000 7f420659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 0x7f4208159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56128/0 (socket says 172.21.15.60:56128) 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.768+0000 7f420659c640 1 -- 172.21.15.60:0/2492394618 learned_addr learned my addr 172.21.15.60:0/2492394618 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f420659c640 1 -- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080c7bf0 msgr2=0x7f42080c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f420659c640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080c7bf0 0x7f42080c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f420659c640 1 -- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4208156a70 msgr2=0x7f4208156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f420659c640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4208156a70 0x7f4208156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f420659c640 1 -- 172.21.15.60:0/2492394618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4208158c70 con 0x7f41f4004090 2023-11-18T20:45:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f420659c640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 0x7f4208159750 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f420000f770 tx=0x7f420000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f41feffd640 1 -- 172.21.15.60:0/2492394618 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000014a0 con 0x7f41f4004090 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f41feffd640 1 -- 172.21.15.60:0/2492394618 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420000fe80 con 0x7f41f4004090 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f420c875640 1 -- 172.21.15.60:0/2492394618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4208158f50 con 0x7f41f4004090 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f41feffd640 1 -- 172.21.15.60:0/2492394618 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420000d730 con 0x7f41f4004090 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.769+0000 7f420c875640 1 -- 172.21.15.60:0/2492394618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42080c85b0 con 0x7f41f4004090 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.770+0000 7f420c875640 1 -- 172.21.15.60:0/2492394618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4208069400 con 0x7f41f4004090 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.770+0000 7f41feffd640 1 -- 172.21.15.60:0/2492394618 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f420000d890 con 0x7f41f4004090 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.771+0000 7f41feffd640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41e806ae70 0x7f41e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.771+0000 7f41feffd640 1 -- 172.21.15.60:0/2492394618 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4200089320 con 0x7f41f4004090 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.771+0000 7f4205d9b640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41e806ae70 0x7f41e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.772+0000 7f4205d9b640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41e806ae70 0x7f41e806d330 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f41f0036000 tx=0x7f41f002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:06.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:06.775+0000 7f41feffd640 1 -- 172.21.15.60:0/2492394618 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4200054760 con 0x7f41f4004090 2023-11-18T20:45:07.021 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.020+0000 7f420c875640 1 -- 172.21.15.60:0/2492394618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7f42080b2340 con 0x7f41f4004090 2023-11-18T20:45:07.318 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.317+0000 7f41feffd640 1 -- 172.21.15.60:0/2492394618 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v158) v1 ==== 134+0+0 (secure 0 0 0) 0x7f4200057d80 con 0x7f41f4004090 2023-11-18T20:45:07.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 -- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41e806ae70 msgr2=0x7f41e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:07.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41e806ae70 0x7f41e806d330 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f41f0036000 tx=0x7f41f002e000 comp rx=0 tx=0).stop 2023-11-18T20:45:07.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 -- 172.21.15.60:0/2492394618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 msgr2=0x7f4208159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 0x7f4208159750 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f420000f770 tx=0x7f420000fc30 comp rx=0 tx=0).stop 2023-11-18T20:45:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 -- 172.21.15.60:0/2492394618 shutdown_connections 2023-11-18T20:45:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f41e806ae70 0x7f41e806d330 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42080c7bf0 0x7f42080c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4208156a70 0x7f4208156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 --2- 172.21.15.60:0/2492394618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41f4004090 0x7f4208159750 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 -- 172.21.15.60:0/2492394618 >> 172.21.15.60:0/2492394618 conn(0x7f4208072020 msgr2=0x7f42080b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 -- 172.21.15.60:0/2492394618 shutdown_connections 2023-11-18T20:45:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:07.319+0000 7f420c875640 1 -- 172.21.15.60:0/2492394618 wait complete. 2023-11-18T20:45:07.333 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2023-11-18T20:45:07.333 INFO:tasks.cephfs_test_runner: 2023-11-18T20:45:07.333 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-18T20:45:07.333 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-11-18T20:45:07.333 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-18T20:45:07.333 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-18T20:45:07.333 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-11-18T20:45:07.333 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-11-18T20:45:07.333 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/qa/tasks/cephfs/mount.py", line 775, in run_python 2023-11-18T20:45:07.333 INFO:tasks.cephfs_test_runner: p.wait() 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 161, in wait 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner: 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner:Ran 2 tests in 123.399s 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner: 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner: 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-18T20:45:07.334 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/qa/tasks/cephfs/mount.py", line 775, in run_python 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner: p.wait() 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 161, in wait 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi060 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-11-18T20:45:07.335 INFO:tasks.cephfs_test_runner: 2023-11-18T20:45:07.335 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-11-18T20:45:07.400 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a6dd878b0f454787bdc7ba4f6aaffdfa Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0f45a51a24e8eca7cf153dba57cbb536ce5b3fe/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-11-18T20:45:07.401 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-18T20:45:07.410 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-18T20:45:07.420 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-18T20:45:07.421 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:45:07.421 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:45:07.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=344 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249488k,nr_inodes=812372,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-18T20:45:07.441 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-18T20:45:07.441 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:45:07.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057150,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=344 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249488k,nr_inodes=812372,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498980k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-18T20:45:07.497 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:45:07.498 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:45:07.516 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056339,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=480 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-18T20:45:07.516 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2023-11-18T20:45:07.516 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-18T20:45:07.574 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056339,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=480 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-18T20:45:07.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:45:07.575 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2023-11-18T20:45:07.632 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-18T20:45:07.633 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-18T20:45:07.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:45:07.633 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-18T20:45:07.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:45:07.704 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-18T20:45:07.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:45:07.774 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2023-11-18T20:45:07.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:45:07.874 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2023-11-18T20:45:07.895 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-18T20:45:07.895 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-18T20:45:07.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:45:07.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-18T20:45:07.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:45:07.971 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-18T20:45:08.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-18T20:45:08.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2023-11-18T20:45:08.152 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-18T20:45:08.161 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-18T20:45:08.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:45:08.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.267+0000 7f756d8bb640 1 Processor -- start 2023-11-18T20:45:08.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.267+0000 7f756d8bb640 1 -- start start 2023-11-18T20:45:08.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f756d8bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75680b1400 0x7f756815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.270 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f756d8bb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f756815e730 0x7f756815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f756d8bb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756815e2a0 con 0x7f7568143490 2023-11-18T20:45:08.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f756d8bb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756815b520 con 0x7f756815e730 2023-11-18T20:45:08.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f756d8bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7568143870 con 0x7f75680b1400 2023-11-18T20:45:08.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f7566ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7568143490 0x7f75680b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45616/0 (socket says 172.21.15.60:45616) 2023-11-18T20:45:08.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f7566ffd640 1 -- 172.21.15.60:0/1138773200 learned_addr learned my addr 172.21.15.60:0/1138773200 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:08.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f755e7fc640 1 --2- 172.21.15.60:0/1138773200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75680b1400 0x7f756815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.271 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f75677fe640 1 --2- 172.21.15.60:0/1138773200 >> v2:172.21.15.100:3300/0 conn(0x7f756815e730 0x7f756815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f75667fc640 1 -- 172.21.15.60:0/1138773200 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001643856 0 0) 0x7f756815e2a0 con 0x7f7568143490 2023-11-18T20:45:08.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f75667fc640 1 -- 172.21.15.60:0/1138773200 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7554003520 con 0x7f7568143490 2023-11-18T20:45:08.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f75677fe640 1 -- 172.21.15.60:0/1138773200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75680b1400 msgr2=0x7f756815a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:08.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f75677fe640 1 --2- 172.21.15.60:0/1138773200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75680b1400 0x7f756815a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f75677fe640 1 -- 172.21.15.60:0/1138773200 >> v1:172.21.15.60:6789/0 conn(0x7f7568143490 legacy=0x7f75680b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f75667fc640 1 -- 172.21.15.60:0/1138773200 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 822053918 0 0) 0x7f7554003520 con 0x7f7568143490 2023-11-18T20:45:08.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f75677fe640 1 -- 172.21.15.60:0/1138773200 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756815b240 con 0x7f756815e730 2023-11-18T20:45:08.272 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.268+0000 7f755e7fc640 1 --2- 172.21.15.60:0/1138773200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75680b1400 0x7f756815a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:08.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.269+0000 7f75677fe640 1 --2- 172.21.15.60:0/1138773200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f756815e730 0x7f756815aa20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7560009d90 tx=0x7f756002bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c0168bbf1135fe server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:08.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.269+0000 7f75667fc640 1 -- 172.21.15.60:0/1138773200 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558001670 con 0x7f756815e730 2023-11-18T20:45:08.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.269+0000 7f75667fc640 1 -- 172.21.15.60:0/1138773200 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756002c8a0 con 0x7f756815e730 2023-11-18T20:45:08.273 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.269+0000 7f75667fc640 1 -- 172.21.15.60:0/1138773200 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756002cc10 con 0x7f756815e730 2023-11-18T20:45:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.270+0000 7f756d8bb640 1 -- 172.21.15.60:0/1138773200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f756815e730 msgr2=0x7f756815aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.270+0000 7f756d8bb640 1 --2- 172.21.15.60:0/1138773200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f756815e730 0x7f756815aa20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7560009d90 tx=0x7f756002bd70 comp rx=0 tx=0).stop 2023-11-18T20:45:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.270+0000 7f756d8bb640 1 -- 172.21.15.60:0/1138773200 shutdown_connections 2023-11-18T20:45:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.270+0000 7f756d8bb640 1 --2- 172.21.15.60:0/1138773200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f756815e730 0x7f756815aa20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.270+0000 7f756d8bb640 1 --2- 172.21.15.60:0/1138773200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75680b1400 0x7f756815a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.276 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.270+0000 7f756d8bb640 1 -- 172.21.15.60:0/1138773200 >> 172.21.15.60:0/1138773200 conn(0x7f756806a4d0 msgr2=0x7f756806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:08.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.270+0000 7f756d8bb640 1 -- 172.21.15.60:0/1138773200 shutdown_connections 2023-11-18T20:45:08.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.270+0000 7f756d8bb640 1 -- 172.21.15.60:0/1138773200 wait complete. 2023-11-18T20:45:08.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.271+0000 7f756d8bb640 1 Processor -- start 2023-11-18T20:45:08.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.271+0000 7f756d8bb640 1 -- start start 2023-11-18T20:45:08.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f756d8bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75680b1400 0x7f75680add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f756d8bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7568143490 0x7f756813db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f756d8bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f756815e730 0x7f756813e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.277 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f756d8bb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756813caf0 con 0x7f75680b1400 2023-11-18T20:45:08.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f756d8bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756813cd90 con 0x7f756815e730 2023-11-18T20:45:08.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f756d8bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756813d070 con 0x7f7568143490 2023-11-18T20:45:08.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75680b1400 0x7f75680add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f7566ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75680b1400 0x7f75680add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56130/0 (socket says 172.21.15.60:56130) 2023-11-18T20:45:08.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f7566ffd640 1 -- 172.21.15.60:0/342388521 learned_addr learned my addr 172.21.15.60:0/342388521 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:08.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f755e7fc640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7568143490 0x7f756813db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f75677fe640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f756815e730 0x7f756813e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f7566ffd640 1 -- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7568143490 msgr2=0x7f756813db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f7566ffd640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7568143490 0x7f756813db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f7566ffd640 1 -- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f756815e730 msgr2=0x7f756813e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f7566ffd640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f756815e730 0x7f756813e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f7566ffd640 1 -- 172.21.15.60:0/342388521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7550009400 con 0x7f75680b1400 2023-11-18T20:45:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.272+0000 7f75677fe640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f756815e730 0x7f756813e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.273+0000 7f7566ffd640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75680b1400 0x7f75680add10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f755800c750 tx=0x7f755800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:08.279 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.273+0000 7f7564ff9640 1 -- 172.21.15.60:0/342388521 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558005ad0 con 0x7f75680b1400 2023-11-18T20:45:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.273+0000 7f7564ff9640 1 -- 172.21.15.60:0/342388521 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7558005c50 con 0x7f75680b1400 2023-11-18T20:45:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.273+0000 7f756d8bb640 1 -- 172.21.15.60:0/342388521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7560009a70 con 0x7f75680b1400 2023-11-18T20:45:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.273+0000 7f7564ff9640 1 -- 172.21.15.60:0/342388521 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558007070 con 0x7f75680b1400 2023-11-18T20:45:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.273+0000 7f756d8bb640 1 -- 172.21.15.60:0/342388521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756813d630 con 0x7f75680b1400 2023-11-18T20:45:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.274+0000 7f756d8bb640 1 -- 172.21.15.60:0/342388521 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7568069400 con 0x7f75680b1400 2023-11-18T20:45:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.274+0000 7f7564ff9640 1 -- 172.21.15.60:0/342388521 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f75580071d0 con 0x7f75680b1400 2023-11-18T20:45:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.275+0000 7f7564ff9640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f754806ae40 0x7f754806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.275+0000 7f7564ff9640 1 -- 172.21.15.60:0/342388521 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7558088fb0 con 0x7f75680b1400 2023-11-18T20:45:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.275+0000 7f755e7fc640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f754806ae40 0x7f754806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.276+0000 7f755e7fc640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f754806ae40 0x7f754806d300 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7568142320 tx=0x7f7550009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:08.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.279+0000 7f7564ff9640 1 -- 172.21.15.60:0/342388521 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75580bbe90 con 0x7f75680b1400 2023-11-18T20:45:08.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.452+0000 7f756d8bb640 1 -- 172.21.15.60:0/342388521 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75680cc560 con 0x7f754806ae40 2023-11-18T20:45:08.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.455+0000 7f7564ff9640 1 -- 172.21.15.60:0/342388521 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51156 (secure 0 0 0) 0x7f75680cc560 con 0x7f754806ae40 2023-11-18T20:45:08.457 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:08.457 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:45:08.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 -- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f754806ae40 msgr2=0x7f754806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.458 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f754806ae40 0x7f754806d300 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7568142320 tx=0x7f7550009190 comp rx=0 tx=0).stop 2023-11-18T20:45:08.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 -- 172.21.15.60:0/342388521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75680b1400 msgr2=0x7f75680add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75680b1400 0x7f75680add10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f755800c750 tx=0x7f755800cc10 comp rx=0 tx=0).stop 2023-11-18T20:45:08.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 -- 172.21.15.60:0/342388521 shutdown_connections 2023-11-18T20:45:08.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f754806ae40 0x7f754806d300 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f756815e730 0x7f756813e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7568143490 0x7f756813db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.459 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 --2- 172.21.15.60:0/342388521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75680b1400 0x7f75680add10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 -- 172.21.15.60:0/342388521 >> 172.21.15.60:0/342388521 conn(0x7f756806a4d0 msgr2=0x7f756806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:08.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 -- 172.21.15.60:0/342388521 shutdown_connections 2023-11-18T20:45:08.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.457+0000 7f756d8bb640 1 -- 172.21.15.60:0/342388521 wait complete. 2023-11-18T20:45:08.469 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2023-11-18T20:45:07.472590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238656,"kb_used_data":5860,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736064,"statfs":{"total":773094113280,"available":772849729536,"internally_reserved":0,"allocated":6000640,"data_stored":3176573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14933,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001086"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":191,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.877346+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:44:34.877346+0000","last_peered":"2023-11-18T20:44:34.877346+0000","last_clean":"2023-11-18T20:44:34.877346+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:44:34.877346+0000","last_undegraded":"2023-11-18T20:44:34.877346+0000","last_fullsized":"2023-11-18T20:44:34.877346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1196032,"data_stored":837836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.439}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60899999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71699999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.40999999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.40000000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.45600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72699999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73499999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.79800000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36899999999999999}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79800000000000004,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77600000000000002,"5min":0.876,"15min":0.876},"last":0.46400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":602112,"data_stored":247996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.36599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.497}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.80000000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.623}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75700000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54600000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76600000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":247344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.38600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.74299999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.002},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.78300000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.45900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.90900000000000003}]},{"osd":3,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91900000000000004,"5min":1.046,"15min":1.046},"last":0.86599999999999999}]},{"osd":4,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63200000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77000000000000002,"5min":0.875,"15min":0.875},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95999999999999996,"5min":1.099,"15min":1.099},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":247926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.317}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.39800000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71499999999999997,"5min":0.747,"15min":0.747},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.5}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.53100000000000003}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73299999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.752,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.48099999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44400000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.34699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58399999999999996}]},{"osd":7,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81699999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":622592,"data_stored":261501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.441,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.24199999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.317,"15min":0.315},"min":{"1min":0.26100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34300000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.316},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.32900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53100000000000003,"5min":0.995,"15min":0.995},"last":0.372}]},{"osd":4,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77700000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.41899999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48199999999999998}]},{"osd":6,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77000000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.45800000000000002}]},{"osd":7,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81299999999999994,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.46500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":261479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33800000000000002,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.37},{"interface":"front","average":{"1min":0.375,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.28000000000000003}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41699999999999998,"15min":0.42299999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54500000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.54600000000000004,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71999999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.52600000000000002}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.68999999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83899999999999997,"5min":1.859,"15min":1.859},"last":0.503},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.626}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.76100000000000001,"5min":1.464,"15min":1.464},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.68600000000000005,"15min":0.70099999999999996},"min":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.749,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.68799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":589824,"data_stored":241423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.38400000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.307},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48999999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.40400000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33300000000000002},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51300000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34999999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.432},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.57099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38800000000000001}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":1.105,"15min":1.105},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.55200000000000005}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70099999999999996}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72499999999999998,"5min":0.995,"15min":0.995},"last":0.58199999999999996}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76900000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1179648,"data_stored":831068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33500000000000002,"15min":0.33600000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.374,"15min":0.375},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":0.62,"15min":0.62},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.51300000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.37,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.33900000000000002}]},{"osd":3,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81100000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.497},{"interface":"front","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59899999999999998}]},{"osd":7,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.625},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.628},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95899999999999996,"5min":1.006,"15min":1.006},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:45:08.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:45:08.577 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.576+0000 7fab36789640 1 Processor -- start 2023-11-18T20:45:08.578 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.576+0000 7fab36789640 1 -- start start 2023-11-18T20:45:08.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab36789640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fab30140310 0x7fab301618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.579 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab36789640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fab3006abe0 0x7fab3006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab36789640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab301406f0 con 0x7fab301409d0 2023-11-18T20:45:08.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab36789640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab30161e20 con 0x7fab3006abe0 2023-11-18T20:45:08.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab36789640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab30143960 con 0x7fab30140310 2023-11-18T20:45:08.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab34d26640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fab3006abe0 0x7fab3006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.580 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab2f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fab301409d0 0x7fab3006a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45618/0 (socket says 172.21.15.60:45618) 2023-11-18T20:45:08.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab2f7fe640 1 -- 172.21.15.60:0/2850042236 learned_addr learned my addr 172.21.15.60:0/2850042236 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:08.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab2ffff640 1 --2- 172.21.15.60:0/2850042236 >> v2:172.21.15.100:3300/0 conn(0x7fab30140310 0x7fab301618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab34d26640 1 -- 172.21.15.60:0/2850042236 >> v2:172.21.15.100:3300/0 conn(0x7fab30140310 msgr2=0x7fab301618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:08.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab34d26640 1 --2- 172.21.15.60:0/2850042236 >> v2:172.21.15.100:3300/0 conn(0x7fab30140310 0x7fab301618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.581 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab34d26640 1 -- 172.21.15.60:0/2850042236 >> v1:172.21.15.60:6789/0 conn(0x7fab301409d0 legacy=0x7fab3006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.577+0000 7fab34d26640 1 -- 172.21.15.60:0/2850042236 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30142de0 con 0x7fab3006abe0 2023-11-18T20:45:08.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.578+0000 7fab34d26640 1 --2- 172.21.15.60:0/2850042236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab3006abe0 0x7fab3006efb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fab300b5840 tx=0x7fab2802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca424aa0635d13da server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:08.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.578+0000 7fab2effd640 1 -- 172.21.15.60:0/2850042236 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2802cbf0 con 0x7fab3006abe0 2023-11-18T20:45:08.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.578+0000 7fab2effd640 1 -- 172.21.15.60:0/2850042236 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2802cd50 con 0x7fab3006abe0 2023-11-18T20:45:08.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.578+0000 7fab2effd640 1 -- 172.21.15.60:0/2850042236 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28034660 con 0x7fab3006abe0 2023-11-18T20:45:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.579+0000 7fab36789640 1 -- 172.21.15.60:0/2850042236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab3006abe0 msgr2=0x7fab3006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.579+0000 7fab36789640 1 --2- 172.21.15.60:0/2850042236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab3006abe0 0x7fab3006efb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fab300b5840 tx=0x7fab2802be40 comp rx=0 tx=0).stop 2023-11-18T20:45:08.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.579+0000 7fab36789640 1 -- 172.21.15.60:0/2850042236 shutdown_connections 2023-11-18T20:45:08.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.579+0000 7fab36789640 1 --2- 172.21.15.60:0/2850042236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab3006abe0 0x7fab3006efb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.579+0000 7fab36789640 1 --2- 172.21.15.60:0/2850042236 >> v2:172.21.15.100:3300/0 conn(0x7fab30140310 0x7fab301618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.579+0000 7fab36789640 1 -- 172.21.15.60:0/2850042236 >> 172.21.15.60:0/2850042236 conn(0x7fab300b1e50 msgr2=0x7fab300b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:08.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.579+0000 7fab36789640 1 -- 172.21.15.60:0/2850042236 shutdown_connections 2023-11-18T20:45:08.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.579+0000 7fab36789640 1 -- 172.21.15.60:0/2850042236 wait complete. 2023-11-18T20:45:08.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.580+0000 7fab36789640 1 Processor -- start 2023-11-18T20:45:08.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.580+0000 7fab36789640 1 -- start start 2023-11-18T20:45:08.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab36789640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab3006abe0 0x7fab301581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab36789640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab30140310 0x7fab3015d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab36789640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab301409d0 0x7fab30143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab36789640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3015d610 con 0x7fab301409d0 2023-11-18T20:45:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab36789640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab30144200 con 0x7fab3006abe0 2023-11-18T20:45:08.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab36789640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab301444d0 con 0x7fab30140310 2023-11-18T20:45:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab34d26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab301409d0 0x7fab30143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab34d26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab301409d0 0x7fab30143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56138/0 (socket says 172.21.15.60:56138) 2023-11-18T20:45:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab34d26640 1 -- 172.21.15.60:0/4203463769 learned_addr learned my addr 172.21.15.60:0/4203463769 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab2ffff640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab3006abe0 0x7fab301581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab2f7fe640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab30140310 0x7fab3015d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab34d26640 1 -- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab30140310 msgr2=0x7fab3015d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab34d26640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab30140310 0x7fab3015d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab34d26640 1 -- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab3006abe0 msgr2=0x7fab301581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab34d26640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab3006abe0 0x7fab301581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab34d26640 1 -- 172.21.15.60:0/4203463769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28009a70 con 0x7fab301409d0 2023-11-18T20:45:08.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab2f7fe640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab30140310 0x7fab3015d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:45:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.581+0000 7fab34d26640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab301409d0 0x7fab30143cc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fab28009e80 tx=0x7fab28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.582+0000 7fab2cff9640 1 -- 172.21.15.60:0/4203463769 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28040650 con 0x7fab301409d0 2023-11-18T20:45:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.582+0000 7fab2cff9640 1 -- 172.21.15.60:0/4203463769 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab280407d0 con 0x7fab301409d0 2023-11-18T20:45:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.582+0000 7fab36789640 1 -- 172.21.15.60:0/4203463769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab301447b0 con 0x7fab301409d0 2023-11-18T20:45:08.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.582+0000 7fab2cff9640 1 -- 172.21.15.60:0/4203463769 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28040b80 con 0x7fab301409d0 2023-11-18T20:45:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.582+0000 7fab36789640 1 -- 172.21.15.60:0/4203463769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab30144d70 con 0x7fab301409d0 2023-11-18T20:45:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.583+0000 7fab36789640 1 -- 172.21.15.60:0/4203463769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab30069400 con 0x7fab301409d0 2023-11-18T20:45:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.583+0000 7fab2cff9640 1 -- 172.21.15.60:0/4203463769 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab28040ce0 con 0x7fab301409d0 2023-11-18T20:45:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.584+0000 7fab2cff9640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fab1006ae40 0x7fab1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.584+0000 7fab2cff9640 1 -- 172.21.15.60:0/4203463769 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fab280ac270 con 0x7fab301409d0 2023-11-18T20:45:08.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.584+0000 7fab2ffff640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fab1006ae40 0x7fab1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.584+0000 7fab2ffff640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fab1006ae40 0x7fab1006d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fab2000a970 tx=0x7fab20008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:08.593 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.588+0000 7fab2cff9640 1 -- 172.21.15.60:0/4203463769 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab28077620 con 0x7fab301409d0 2023-11-18T20:45:08.760 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.759+0000 7fab36789640 1 -- 172.21.15.60:0/4203463769 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab30143680 con 0x7fab1006ae40 2023-11-18T20:45:08.763 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.762+0000 7fab2cff9640 1 -- 172.21.15.60:0/4203463769 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51156 (secure 0 0 0) 0x7fab2000c050 con 0x7fab1006ae40 2023-11-18T20:45:08.763 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:45:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.763+0000 7fab36789640 1 -- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fab1006ae40 msgr2=0x7fab1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.763+0000 7fab36789640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fab1006ae40 0x7fab1006d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fab2000a970 tx=0x7fab20008040 comp rx=0 tx=0).stop 2023-11-18T20:45:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.763+0000 7fab36789640 1 -- 172.21.15.60:0/4203463769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab301409d0 msgr2=0x7fab30143cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.763+0000 7fab36789640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab301409d0 0x7fab30143cc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fab28009e80 tx=0x7fab28004a40 comp rx=0 tx=0).stop 2023-11-18T20:45:08.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.764+0000 7fab36789640 1 -- 172.21.15.60:0/4203463769 shutdown_connections 2023-11-18T20:45:08.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.764+0000 7fab36789640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fab1006ae40 0x7fab1006d300 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.764+0000 7fab36789640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab301409d0 0x7fab30143cc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.764+0000 7fab36789640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab30140310 0x7fab3015d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.764+0000 7fab36789640 1 --2- 172.21.15.60:0/4203463769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab3006abe0 0x7fab301581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.764+0000 7fab36789640 1 -- 172.21.15.60:0/4203463769 >> 172.21.15.60:0/4203463769 conn(0x7fab300b1e50 msgr2=0x7fab3015eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:08.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.764+0000 7fab36789640 1 -- 172.21.15.60:0/4203463769 shutdown_connections 2023-11-18T20:45:08.767 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.764+0000 7fab36789640 1 -- 172.21.15.60:0/4203463769 wait complete. 2023-11-18T20:45:08.775 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2023-11-18T20:45:07.472590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238656,"kb_used_data":5860,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736064,"statfs":{"total":773094113280,"available":772849729536,"internally_reserved":0,"allocated":6000640,"data_stored":3176573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14933,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001086"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":191,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.877346+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:44:34.877346+0000","last_peered":"2023-11-18T20:44:34.877346+0000","last_clean":"2023-11-18T20:44:34.877346+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:44:34.877346+0000","last_undegraded":"2023-11-18T20:44:34.877346+0000","last_fullsized":"2023-11-18T20:44:34.877346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1196032,"data_stored":837836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.439}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60899999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71699999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.40999999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.40000000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.45600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72699999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73499999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.79800000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36899999999999999}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79800000000000004,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77600000000000002,"5min":0.876,"15min":0.876},"last":0.46400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":602112,"data_stored":247996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.36599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.497}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.80000000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.623}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75700000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54600000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76600000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":247344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.38600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.74299999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.002},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.78300000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.45900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.90900000000000003}]},{"osd":3,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91900000000000004,"5min":1.046,"15min":1.046},"last":0.86599999999999999}]},{"osd":4,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63200000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77000000000000002,"5min":0.875,"15min":0.875},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95999999999999996,"5min":1.099,"15min":1.099},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":247926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.317}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.39800000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71499999999999997,"5min":0.747,"15min":0.747},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.5}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.53100000000000003}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73299999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.752,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.48099999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44400000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.34699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58399999999999996}]},{"osd":7,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81699999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":622592,"data_stored":261501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.441,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.24199999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.317,"15min":0.315},"min":{"1min":0.26100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34300000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.316},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.32900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53100000000000003,"5min":0.995,"15min":0.995},"last":0.372}]},{"osd":4,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77700000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.41899999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48199999999999998}]},{"osd":6,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77000000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.45800000000000002}]},{"osd":7,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81299999999999994,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.46500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":261479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33800000000000002,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.37},{"interface":"front","average":{"1min":0.375,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.28000000000000003}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41699999999999998,"15min":0.42299999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54500000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.54600000000000004,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71999999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.52600000000000002}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.68999999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83899999999999997,"5min":1.859,"15min":1.859},"last":0.503},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.626}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.76100000000000001,"5min":1.464,"15min":1.464},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.68600000000000005,"15min":0.70099999999999996},"min":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.749,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.68799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":589824,"data_stored":241423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.38400000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.307},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48999999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.40400000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33300000000000002},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51300000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34999999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.432},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.57099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38800000000000001}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":1.105,"15min":1.105},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.55200000000000005}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70099999999999996}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72499999999999998,"5min":0.995,"15min":0.995},"last":0.58199999999999996}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76900000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1179648,"data_stored":831068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33500000000000002,"15min":0.33600000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.374,"15min":0.375},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":0.62,"15min":0.62},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.51300000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.37,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.33900000000000002}]},{"osd":3,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81100000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.497},{"interface":"front","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59899999999999998}]},{"osd":7,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.625},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.628},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95899999999999996,"5min":1.006,"15min":1.006},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:45:08.777 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-18T20:45:08.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:45:08.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.881+0000 7f5be255a640 1 Processor -- start 2023-11-18T20:45:08.883 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.881+0000 7f5be255a640 1 -- start start 2023-11-18T20:45:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.882+0000 7f5be255a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc15d330 0x7f5bdc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.882+0000 7f5be255a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5bdc0d2910 0x7f5bdc0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.884 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.882+0000 7f5be255a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0b2b10 con 0x7f5bdc0d2d90 2023-11-18T20:45:08.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.882+0000 7f5be255a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc0d1590 con 0x7f5bdc0d2910 2023-11-18T20:45:08.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.882+0000 7f5be255a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0d1870 con 0x7f5bdc15d330 2023-11-18T20:45:08.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.883+0000 7f5bdbfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc15d330 0x7f5bdc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.883+0000 7f5bdb7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5bdc0d2910 0x7f5bdc0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.883+0000 7f5bdbfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc15d330 0x7f5bdc0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:57468/0 (socket says 172.21.15.60:57468) 2023-11-18T20:45:08.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.883+0000 7f5bdbfff640 1 -- 172.21.15.60:0/27724135 learned_addr learned my addr 172.21.15.60:0/27724135 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:08.885 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.883+0000 7f5be0af7640 1 --1- 172.21.15.60:0/27724135 >> v1:172.21.15.100:6789/0 conn(0x7f5bdc0d2d90 0x7f5bdc0b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:40988/0 (socket says 172.21.15.60:40988) 2023-11-18T20:45:08.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.883+0000 7f5bdbfff640 1 -- 172.21.15.60:0/27724135 >> v2:172.21.15.100:3300/0 conn(0x7f5bdc0d2910 msgr2=0x7f5bdc0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:08.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.883+0000 7f5bdbfff640 1 --2- 172.21.15.60:0/27724135 >> v2:172.21.15.100:3300/0 conn(0x7f5bdc0d2910 0x7f5bdc0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.883+0000 7f5bdbfff640 1 -- 172.21.15.60:0/27724135 >> v1:172.21.15.100:6789/0 conn(0x7f5bdc0d2d90 legacy=0x7f5bdc0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.883+0000 7f5bdbfff640 1 -- 172.21.15.60:0/27724135 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0d1b50 con 0x7f5bdc15d330 2023-11-18T20:45:08.886 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.883+0000 7f5bdb7fe640 1 --2- 172.21.15.60:0/27724135 >> v2:172.21.15.100:3300/0 conn(0x7f5bdc0d2910 0x7f5bdc0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:08.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.884+0000 7f5bdbfff640 1 --2- 172.21.15.60:0/27724135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc15d330 0x7f5bdc0b8b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009870 tx=0x7f5bcc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9422e376ac7dc73f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:08.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.884+0000 7f5bdaffd640 1 -- 172.21.15.60:0/27724135 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc031640 con 0x7f5bdc15d330 2023-11-18T20:45:08.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.884+0000 7f5bdaffd640 1 -- 172.21.15.60:0/27724135 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc0317a0 con 0x7f5bdc15d330 2023-11-18T20:45:08.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.884+0000 7f5bdaffd640 1 -- 172.21.15.60:0/27724135 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc031b30 con 0x7f5bdc15d330 2023-11-18T20:45:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.885+0000 7f5be255a640 1 -- 172.21.15.60:0/27724135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc15d330 msgr2=0x7f5bdc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.885+0000 7f5be255a640 1 --2- 172.21.15.60:0/27724135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc15d330 0x7f5bdc0b8b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009870 tx=0x7f5bcc02b8f0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.885+0000 7f5be255a640 1 -- 172.21.15.60:0/27724135 shutdown_connections 2023-11-18T20:45:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.885+0000 7f5be255a640 1 --2- 172.21.15.60:0/27724135 >> v2:172.21.15.100:3300/0 conn(0x7f5bdc0d2910 0x7f5bdc0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.885+0000 7f5be255a640 1 --2- 172.21.15.60:0/27724135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc15d330 0x7f5bdc0b8b80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.885+0000 7f5be255a640 1 -- 172.21.15.60:0/27724135 >> 172.21.15.60:0/27724135 conn(0x7f5bdc072020 msgr2=0x7f5bdc072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:08.890 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.885+0000 7f5be255a640 1 -- 172.21.15.60:0/27724135 shutdown_connections 2023-11-18T20:45:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.885+0000 7f5be255a640 1 -- 172.21.15.60:0/27724135 wait complete. 2023-11-18T20:45:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.886+0000 7f5be255a640 1 Processor -- start 2023-11-18T20:45:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.886+0000 7f5be255a640 1 -- start start 2023-11-18T20:45:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5be255a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bdc0d2910 0x7f5bdc1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5be255a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc0d2d90 0x7f5bdc0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5be255a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bdc15d330 0x7f5bdc0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5be255a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0c7c00 con 0x7f5bdc0d2910 2023-11-18T20:45:08.891 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5be255a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0cd580 con 0x7f5bdc15d330 2023-11-18T20:45:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5be255a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0ccce0 con 0x7f5bdc0d2d90 2023-11-18T20:45:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bdc0d2910 0x7f5bdc1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bdbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bdc0d2910 0x7f5bdc1561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56154/0 (socket says 172.21.15.60:56154) 2023-11-18T20:45:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bdbfff640 1 -- 172.21.15.60:0/2584814995 learned_addr learned my addr 172.21.15.60:0/2584814995 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bdb7fe640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc0d2d90 0x7f5bdc0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5be0af7640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bdc15d330 0x7f5bdc0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bdbfff640 1 -- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc0d2d90 msgr2=0x7f5bdc0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bdbfff640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc0d2d90 0x7f5bdc0cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bdbfff640 1 -- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bdc15d330 msgr2=0x7f5bdc0cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:08.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bdbfff640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bdc15d330 0x7f5bdc0cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:08.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bdbfff640 1 -- 172.21.15.60:0/2584814995 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd0009400 con 0x7f5bdc0d2910 2023-11-18T20:45:08.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5be0af7640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bdc15d330 0x7f5bdc0cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:45:08.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bdbfff640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bdc0d2910 0x7f5bdc1561f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc000c00 tx=0x7f5bcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:08.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.887+0000 7f5bd8ff9640 1 -- 172.21.15.60:0/2584814995 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc03a930 con 0x7f5bdc0d2910 2023-11-18T20:45:08.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.888+0000 7f5bd8ff9640 1 -- 172.21.15.60:0/2584814995 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc03aa90 con 0x7f5bdc0d2910 2023-11-18T20:45:08.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.888+0000 7f5be255a640 1 -- 172.21.15.60:0/2584814995 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc0094d0 con 0x7f5bdc0d2910 2023-11-18T20:45:08.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.888+0000 7f5bd8ff9640 1 -- 172.21.15.60:0/2584814995 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc03ae40 con 0x7f5bdc0d2910 2023-11-18T20:45:08.894 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.888+0000 7f5be255a640 1 -- 172.21.15.60:0/2584814995 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc0cd2a0 con 0x7f5bdc0d2910 2023-11-18T20:45:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.889+0000 7f5be255a640 1 -- 172.21.15.60:0/2584814995 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc069400 con 0x7f5bdc0d2910 2023-11-18T20:45:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.889+0000 7f5bd8ff9640 1 -- 172.21.15.60:0/2584814995 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5bcc02f050 con 0x7f5bdc0d2910 2023-11-18T20:45:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.889+0000 7f5bd8ff9640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5bc806ae40 0x7f5bc806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.890+0000 7f5bd8ff9640 1 -- 172.21.15.60:0/2584814995 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5bcc0ab0b0 con 0x7f5bdc0d2910 2023-11-18T20:45:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.890+0000 7f5bdb7fe640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5bc806ae40 0x7f5bc806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:08.895 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.890+0000 7f5bdb7fe640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5bc806ae40 0x7f5bc806d300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009ae0 tx=0x7f5bd0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:08.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:08.894+0000 7f5bd8ff9640 1 -- 172.21.15.60:0/2584814995 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5bcc0764f0 con 0x7f5bdc0d2910 2023-11-18T20:45:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.066+0000 7f5be255a640 1 -- 172.21.15.60:0/2584814995 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5bdc0d23f0 con 0x7f5bc806ae40 2023-11-18T20:45:09.069 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.068+0000 7f5bd8ff9640 1 -- 172.21.15.60:0/2584814995 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51156 (secure 0 0 0) 0x7f5bd0010930 con 0x7f5bc806ae40 2023-11-18T20:45:09.070 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:45:09.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.070+0000 7f5be255a640 1 -- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5bc806ae40 msgr2=0x7f5bc806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:09.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.070+0000 7f5be255a640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5bc806ae40 0x7f5bc806d300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009ae0 tx=0x7f5bd0008040 comp rx=0 tx=0).stop 2023-11-18T20:45:09.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.070+0000 7f5be255a640 1 -- 172.21.15.60:0/2584814995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bdc0d2910 msgr2=0x7f5bdc1561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:09.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.070+0000 7f5be255a640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bdc0d2910 0x7f5bdc1561f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc000c00 tx=0x7f5bcc0046c0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.072 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.070+0000 7f5be255a640 1 -- 172.21.15.60:0/2584814995 shutdown_connections 2023-11-18T20:45:09.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.070+0000 7f5be255a640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5bc806ae40 0x7f5bc806d300 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.070+0000 7f5be255a640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bdc15d330 0x7f5bdc0cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.070+0000 7f5be255a640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bdc0d2d90 0x7f5bdc0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.070+0000 7f5be255a640 1 --2- 172.21.15.60:0/2584814995 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bdc0d2910 0x7f5bdc1561f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.070+0000 7f5be255a640 1 -- 172.21.15.60:0/2584814995 >> 172.21.15.60:0/2584814995 conn(0x7f5bdc072020 msgr2=0x7f5bdc072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:09.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.071+0000 7f5be255a640 1 -- 172.21.15.60:0/2584814995 shutdown_connections 2023-11-18T20:45:09.073 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.071+0000 7f5be255a640 1 -- 172.21.15.60:0/2584814995 wait complete. 2023-11-18T20:45:09.082 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2023-11-18T20:45:07.472590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238656,"kb_used_data":5860,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736064,"statfs":{"total":773094113280,"available":772849729536,"internally_reserved":0,"allocated":6000640,"data_stored":3176573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14933,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001086"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":191,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-18T20:44:34.877346+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:44:34.877346+0000","last_peered":"2023-11-18T20:44:34.877346+0000","last_clean":"2023-11-18T20:44:34.877346+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:44:34.877346+0000","last_undegraded":"2023-11-18T20:44:34.877346+0000","last_fullsized":"2023-11-18T20:44:34.877346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1196032,"data_stored":837836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.439}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60899999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71699999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.40999999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.40000000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.45600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56599999999999995}]},{"osd":4,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72699999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73499999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.79800000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.36899999999999999}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79800000000000004,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77600000000000002,"5min":0.876,"15min":0.876},"last":0.46400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":602112,"data_stored":247996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.36599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.497}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.80000000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.623}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75700000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54600000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76600000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":247344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.38600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.74299999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.002},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.78300000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.45900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.90900000000000003}]},{"osd":3,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91900000000000004,"5min":1.046,"15min":1.046},"last":0.86599999999999999}]},{"osd":4,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63200000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77000000000000002,"5min":0.875,"15min":0.875},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95999999999999996,"5min":1.099,"15min":1.099},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":247926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.317}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.39800000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71499999999999997,"5min":0.747,"15min":0.747},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.5}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.53100000000000003}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73299999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.752,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.48099999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44400000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.34699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58399999999999996}]},{"osd":7,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81699999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":622592,"data_stored":261501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.441,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.24199999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.317,"15min":0.315},"min":{"1min":0.26100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34300000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.316},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.32900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53100000000000003,"5min":0.995,"15min":0.995},"last":0.372}]},{"osd":4,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77700000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.41899999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48199999999999998}]},{"osd":6,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77000000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.45800000000000002}]},{"osd":7,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81299999999999994,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.46500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":261479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33800000000000002,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.37},{"interface":"front","average":{"1min":0.375,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.28000000000000003}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41699999999999998,"15min":0.42299999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54500000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.54600000000000004,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71999999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.52600000000000002}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.68999999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83899999999999997,"5min":1.859,"15min":1.859},"last":0.503},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.626}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.76100000000000001,"5min":1.464,"15min":1.464},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.68600000000000005,"15min":0.70099999999999996},"min":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.749,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.68799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":589824,"data_stored":241423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.38400000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.307},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48999999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.40400000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33300000000000002},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51300000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34999999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.432},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.57099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38800000000000001}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":1.105,"15min":1.105},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.55200000000000005}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70099999999999996}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72499999999999998,"5min":0.995,"15min":0.995},"last":0.58199999999999996}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76900000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1179648,"data_stored":831068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33500000000000002,"15min":0.33600000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.374,"15min":0.375},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":0.62,"15min":0.62},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.51300000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.37,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.33900000000000002}]},{"osd":3,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40100000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81100000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.497},{"interface":"front","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59899999999999998}]},{"osd":7,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.625},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.628},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95899999999999996,"5min":1.006,"15min":1.006},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:45:09.084 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-18T20:45:09.190 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.189+0000 7ff4e1aaf640 1 Processor -- start 2023-11-18T20:45:09.191 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.189+0000 7ff4e1aaf640 1 -- start start 2023-11-18T20:45:09.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4e1aaf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4dc0ad640 0x7ff4dc06d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:09.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4e1aaf640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc0adae0 con 0x7ff4dc15d1d0 2023-11-18T20:45:09.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4e1aaf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc0addc0 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4e1aaf640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4dc0ae0a0 con 0x7ff4dc0ad640 2023-11-18T20:45:09.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4daffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4dc0ad640 0x7ff4dc06d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:09.192 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4db7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff4dc15d1d0 0x7ff4dc06b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45622/0 (socket says 172.21.15.60:45622) 2023-11-18T20:45:09.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4db7fe640 1 -- 172.21.15.60:0/1756743780 learned_addr learned my addr 172.21.15.60:0/1756743780 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:09.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4daffd640 1 -- 172.21.15.60:0/1756743780 >> v1:172.21.15.100:6789/0 conn(0x7ff4dc06bbf0 legacy=0x7ff4dc06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:09.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4daffd640 1 -- 172.21.15.60:0/1756743780 >> v1:172.21.15.60:6789/0 conn(0x7ff4dc15d1d0 legacy=0x7ff4dc06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:09.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4daffd640 1 -- 172.21.15.60:0/1756743780 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc0ae380 con 0x7ff4dc0ad640 2023-11-18T20:45:09.193 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.190+0000 7ff4da7fc640 1 -- 172.21.15.60:0/1756743780 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166093018 0 0) 0x7ff4dc0adae0 con 0x7ff4dc15d1d0 2023-11-18T20:45:09.194 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.191+0000 7ff4daffd640 1 --2- 172.21.15.60:0/1756743780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc0ad640 0x7ff4dc06d0b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00097a0 tx=0x7ff4d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee543ba1fef194a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.191+0000 7ff4da7fc640 1 -- 172.21.15.60:0/1756743780 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0031830 con 0x7ff4dc0ad640 2023-11-18T20:45:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.191+0000 7ff4da7fc640 1 -- 172.21.15.60:0/1756743780 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d0031990 con 0x7ff4dc0ad640 2023-11-18T20:45:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.191+0000 7ff4da7fc640 1 -- 172.21.15.60:0/1756743780 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0031d20 con 0x7ff4dc0ad640 2023-11-18T20:45:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.192+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/1756743780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc0ad640 msgr2=0x7ff4dc06d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.192+0000 7ff4e1aaf640 1 --2- 172.21.15.60:0/1756743780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc0ad640 0x7ff4dc06d0b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00097a0 tx=0x7ff4d002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.192+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/1756743780 shutdown_connections 2023-11-18T20:45:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.192+0000 7ff4e1aaf640 1 --2- 172.21.15.60:0/1756743780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc0ad640 0x7ff4dc06d0b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.192+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/1756743780 >> 172.21.15.60:0/1756743780 conn(0x7ff4dc070e70 msgr2=0x7ff4dc071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.192+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/1756743780 shutdown_connections 2023-11-18T20:45:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.192+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/1756743780 wait complete. 2023-11-18T20:45:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.193+0000 7ff4e1aaf640 1 Processor -- start 2023-11-18T20:45:09.196 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.193+0000 7ff4e1aaf640 1 -- start start 2023-11-18T20:45:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4e1aaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc06bbf0 0x7ff4dc143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:09.198 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4e1aaf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4dc0ad640 0x7ff4dc15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4e1aaf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4dc15d1d0 0x7ff4dc14fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4e1aaf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc1411d0 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4e1aaf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc159a50 con 0x7ff4dc15d1d0 2023-11-18T20:45:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4e1aaf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc158e40 con 0x7ff4dc0ad640 2023-11-18T20:45:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4db7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc06bbf0 0x7ff4dc143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4db7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc06bbf0 0x7ff4dc143710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56166/0 (socket says 172.21.15.60:56166) 2023-11-18T20:45:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4db7fe640 1 -- 172.21.15.60:0/3511174317 learned_addr learned my addr 172.21.15.60:0/3511174317 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4daffd640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4dc0ad640 0x7ff4dc15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4dbfff640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4dc15d1d0 0x7ff4dc14fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4db7fe640 1 -- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4dc0ad640 msgr2=0x7ff4dc15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4db7fe640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4dc0ad640 0x7ff4dc15d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4db7fe640 1 -- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4dc15d1d0 msgr2=0x7ff4dc14fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4db7fe640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4dc15d1d0 0x7ff4dc14fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4db7fe640 1 -- 172.21.15.60:0/3511174317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d0009400 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4dbfff640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4dc15d1d0 0x7ff4dc14fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.194+0000 7ff4db7fe640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc06bbf0 0x7ff4dc143710 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00c460 tx=0x7ff4cc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.195+0000 7ff4c3fff640 1 -- 172.21.15.60:0/3511174317 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0074e0 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.201 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.195+0000 7ff4c3fff640 1 -- 172.21.15.60:0/3511174317 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc007660 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.195+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/3511174317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc159120 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.195+0000 7ff4c3fff640 1 -- 172.21.15.60:0/3511174317 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc004a20 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.195+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/3511174317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc1596e0 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.196+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/3511174317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4dc159d30 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.196+0000 7ff4c3fff640 1 -- 172.21.15.60:0/3511174317 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4cc004b80 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.196+0000 7ff4c3fff640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff4c806ae40 0x7ff4c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:09.202 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.197+0000 7ff4c3fff640 1 -- 172.21.15.60:0/3511174317 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff4cc088520 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.197+0000 7ff4daffd640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff4c806ae40 0x7ff4c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.197+0000 7ff4daffd640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff4c806ae40 0x7ff4c806d300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0036000 tx=0x7ff4d003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:09.203 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.201+0000 7ff4c3fff640 1 -- 172.21.15.60:0/3511174317 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4cc053900 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.379+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/3511174317 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4dc069400 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.379+0000 7ff4c3fff640 1 -- 172.21.15.60:0/3511174317 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+12974 (secure 0 0 0) 0x7ff4cc018090 con 0x7ff4dc06bbf0 2023-11-18T20:45:09.381 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:09.381 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":52,"fsid":"ae4682cb-f89a-4edd-b6d6-55eae03fd1b8","created":"2023-11-18T20:42:13.323233+0000","modified":"2023-11-18T20:45:06.312438+0000","last_up_change":"2023-11-18T20:42:23.954469+0000","last_in_change":"2023-11-18T20:42:15.511939+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-18T20:42:25.491946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c9a2cddd-47db-4b41-a027-3a80d0dd460f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6801","nonce":2078754623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6803","nonce":2078754623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6807","nonce":2078754623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":2078754623},{"type":"v1","addr":"172.21.15.60:6805","nonce":2078754623}]},"public_addr":"172.21.15.60:6801/2078754623","cluster_addr":"172.21.15.60:6803/2078754623","heartbeat_back_addr":"172.21.15.60:6807/2078754623","heartbeat_front_addr":"172.21.15.60:6805/2078754623","state":["exists","up"]},{"osd":1,"uuid":"3ac66b30-eb29-41c3-bbf5-c9d2463d00b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6809","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6812","nonce":3839540220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6817","nonce":3839540220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6823","nonce":3839540220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3839540220},{"type":"v1","addr":"172.21.15.60:6821","nonce":3839540220}]},"public_addr":"172.21.15.60:6812/3839540220","cluster_addr":"172.21.15.60:6817/3839540220","heartbeat_back_addr":"172.21.15.60:6823/3839540220","heartbeat_front_addr":"172.21.15.60:6821/3839540220","state":["exists","up"]},{"osd":2,"uuid":"16e394a5-b7ad-4465-85af-63a9107958f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6810","nonce":479181725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6813","nonce":479181725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6819","nonce":479181725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":479181725},{"type":"v1","addr":"172.21.15.60:6816","nonce":479181725}]},"public_addr":"172.21.15.60:6810/479181725","cluster_addr":"172.21.15.60:6813/479181725","heartbeat_back_addr":"172.21.15.60:6819/479181725","heartbeat_front_addr":"172.21.15.60:6816/479181725","state":["exists","up"]},{"osd":3,"uuid":"babf5c8f-7384-4916-8d52-382f76dcd263","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6825","nonce":4034119923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6827","nonce":4034119923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6831","nonce":4034119923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":4034119923},{"type":"v1","addr":"172.21.15.60:6829","nonce":4034119923}]},"public_addr":"172.21.15.60:6825/4034119923","cluster_addr":"172.21.15.60:6827/4034119923","heartbeat_back_addr":"172.21.15.60:6831/4034119923","heartbeat_front_addr":"172.21.15.60:6829/4034119923","state":["exists","up"]},{"osd":4,"uuid":"c814556f-cab3-4263-9038-9b6dec6dda55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6801","nonce":454905857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6804","nonce":454905857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6811","nonce":454905857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":454905857},{"type":"v1","addr":"172.21.15.100:6808","nonce":454905857}]},"public_addr":"172.21.15.100:6801/454905857","cluster_addr":"172.21.15.100:6804/454905857","heartbeat_back_addr":"172.21.15.100:6811/454905857","heartbeat_front_addr":"172.21.15.100:6808/454905857","state":["exists","up"]},{"osd":5,"uuid":"bd5eb319-c72d-4451-a478-db504bd2fcf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6817","nonce":4243789067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6819","nonce":4243789067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6823","nonce":4243789067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4243789067},{"type":"v1","addr":"172.21.15.100:6821","nonce":4243789067}]},"public_addr":"172.21.15.100:6817/4243789067","cluster_addr":"172.21.15.100:6819/4243789067","heartbeat_back_addr":"172.21.15.100:6823/4243789067","heartbeat_front_addr":"172.21.15.100:6821/4243789067","state":["exists","up"]},{"osd":6,"uuid":"83e97732-ae79-4373-9313-65910a99e0d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6805","nonce":1390121871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6810","nonce":1390121871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6815","nonce":1390121871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1390121871},{"type":"v1","addr":"172.21.15.100:6813","nonce":1390121871}]},"public_addr":"172.21.15.100:6805/1390121871","cluster_addr":"172.21.15.100:6810/1390121871","heartbeat_back_addr":"172.21.15.100:6815/1390121871","heartbeat_front_addr":"172.21.15.100:6813/1390121871","state":["exists","up"]},{"osd":7,"uuid":"228baf66-5cfe-414c-aeea-92ebcd6cbcad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6825","nonce":1780039900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6827","nonce":1780039900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6831","nonce":1780039900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1780039900},{"type":"v1","addr":"172.21.15.100:6829","nonce":1780039900}]},"public_addr":"172.21.15.100:6825/1780039900","cluster_addr":"172.21.15.100:6827/1780039900","heartbeat_back_addr":"172.21.15.100:6831/1780039900","heartbeat_front_addr":"172.21.15.100:6829/1780039900","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.626842+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.646124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.716950+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.672175+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.707920+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.677333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-18T20:42:20.733809+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/3098423505":"2023-11-19T20:45:02.955595+0000","172.21.15.60:6834/3098423505":"2023-11-19T20:45:02.955595+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-18T20:45:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.381+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff4c806ae40 msgr2=0x7ff4c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.381+0000 7ff4e1aaf640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff4c806ae40 0x7ff4c806d300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0036000 tx=0x7ff4d003d040 comp rx=0 tx=0).stop 2023-11-18T20:45:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.381+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/3511174317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc06bbf0 msgr2=0x7ff4dc143710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.381+0000 7ff4e1aaf640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc06bbf0 0x7ff4dc143710 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00c460 tx=0x7ff4cc00c920 comp rx=0 tx=0).stop 2023-11-18T20:45:09.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.382+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/3511174317 shutdown_connections 2023-11-18T20:45:09.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.382+0000 7ff4e1aaf640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff4c806ae40 0x7ff4c806d300 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.382+0000 7ff4e1aaf640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4dc15d1d0 0x7ff4dc14fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.382+0000 7ff4e1aaf640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4dc0ad640 0x7ff4dc15d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.382+0000 7ff4e1aaf640 1 --2- 172.21.15.60:0/3511174317 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4dc06bbf0 0x7ff4dc143710 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:09.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.382+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/3511174317 >> 172.21.15.60:0/3511174317 conn(0x7ff4dc070e70 msgr2=0x7ff4dc0af780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:09.384 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.382+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/3511174317 shutdown_connections 2023-11-18T20:45:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:09.382+0000 7ff4e1aaf640 1 -- 172.21.15.60:0/3511174317 wait complete. 2023-11-18T20:45:09.537 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:45:09.539 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:45:10.395 INFO:tasks.ceph:Scrubbing osd.0 2023-11-18T20:45:10.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-18T20:45:10.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.519+0000 7ff2ea990640 1 Processor -- start 2023-11-18T20:45:10.520 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.519+0000 7ff2ea990640 1 -- start start 2023-11-18T20:45:10.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2ea990640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4160bb0 0x7ff2e40d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2ea990640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff2e40ce440 0x7ff2e40d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.522 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2ea990640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e40b3450 con 0x7ff2e40b3fc0 2023-11-18T20:45:10.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2ea990640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e40ce8b0 con 0x7ff2e40ce440 2023-11-18T20:45:10.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2ea990640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e40b45b0 con 0x7ff2e4160bb0 2023-11-18T20:45:10.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff2e40b3fc0 0x7ff2e40cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45624/0 (socket says 172.21.15.60:45624) 2023-11-18T20:45:10.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e37fe640 1 -- 172.21.15.60:0/3778546035 learned_addr learned my addr 172.21.15.60:0/3778546035 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:10.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e3fff640 1 --2- 172.21.15.60:0/3778546035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4160bb0 0x7ff2e40d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e8f2d640 1 --2- 172.21.15.60:0/3778546035 >> v2:172.21.15.100:3300/0 conn(0x7ff2e40ce440 0x7ff2e40d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.523 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e2ffd640 1 -- 172.21.15.60:0/3778546035 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373501649 0 0) 0x7ff2e40b3450 con 0x7ff2e40b3fc0 2023-11-18T20:45:10.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e2ffd640 1 -- 172.21.15.60:0/3778546035 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2cc003520 con 0x7ff2e40b3fc0 2023-11-18T20:45:10.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e3fff640 1 -- 172.21.15.60:0/3778546035 >> v2:172.21.15.100:3300/0 conn(0x7ff2e40ce440 msgr2=0x7ff2e40d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:10.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e3fff640 1 --2- 172.21.15.60:0/3778546035 >> v2:172.21.15.100:3300/0 conn(0x7ff2e40ce440 0x7ff2e40d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e3fff640 1 -- 172.21.15.60:0/3778546035 >> v1:172.21.15.60:6789/0 conn(0x7ff2e40b3fc0 legacy=0x7ff2e40cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e3fff640 1 -- 172.21.15.60:0/3778546035 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e40b4890 con 0x7ff2e4160bb0 2023-11-18T20:45:10.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.520+0000 7ff2e8f2d640 1 --2- 172.21.15.60:0/3778546035 >> v2:172.21.15.100:3300/0 conn(0x7ff2e40ce440 0x7ff2e40d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:10.524 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.521+0000 7ff2e3fff640 1 --2- 172.21.15.60:0/3778546035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4160bb0 0x7ff2e40d3a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40097a0 tx=0x7ff2d402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6023bf33152e089a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.521+0000 7ff2e2ffd640 1 -- 172.21.15.60:0/3778546035 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4031830 con 0x7ff2e4160bb0 2023-11-18T20:45:10.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.521+0000 7ff2e2ffd640 1 -- 172.21.15.60:0/3778546035 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d4031990 con 0x7ff2e4160bb0 2023-11-18T20:45:10.525 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.521+0000 7ff2e2ffd640 1 -- 172.21.15.60:0/3778546035 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4031d20 con 0x7ff2e4160bb0 2023-11-18T20:45:10.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.522+0000 7ff2ea990640 1 -- 172.21.15.60:0/3778546035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4160bb0 msgr2=0x7ff2e40d3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.522+0000 7ff2ea990640 1 --2- 172.21.15.60:0/3778546035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4160bb0 0x7ff2e40d3a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40097a0 tx=0x7ff2d402b8d0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.522+0000 7ff2ea990640 1 -- 172.21.15.60:0/3778546035 shutdown_connections 2023-11-18T20:45:10.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.522+0000 7ff2ea990640 1 --2- 172.21.15.60:0/3778546035 >> v2:172.21.15.100:3300/0 conn(0x7ff2e40ce440 0x7ff2e40d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.522+0000 7ff2ea990640 1 --2- 172.21.15.60:0/3778546035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4160bb0 0x7ff2e40d3a30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.528 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.522+0000 7ff2ea990640 1 -- 172.21.15.60:0/3778546035 >> 172.21.15.60:0/3778546035 conn(0x7ff2e409c590 msgr2=0x7ff2e40d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:10.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.522+0000 7ff2ea990640 1 -- 172.21.15.60:0/3778546035 shutdown_connections 2023-11-18T20:45:10.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.522+0000 7ff2ea990640 1 -- 172.21.15.60:0/3778546035 wait complete. 2023-11-18T20:45:10.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.523+0000 7ff2ea990640 1 Processor -- start 2023-11-18T20:45:10.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.524+0000 7ff2ea990640 1 -- start start 2023-11-18T20:45:10.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.524+0000 7ff2ea990640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e40b3fc0 0x7ff2e41f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.524+0000 7ff2ea990640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e40ce440 0x7ff2e40cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.529 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.524+0000 7ff2ea990640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e4160bb0 0x7ff2e4159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.524+0000 7ff2ea990640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e41f2e60 con 0x7ff2e40ce440 2023-11-18T20:45:10.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.524+0000 7ff2ea990640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e40cc6c0 con 0x7ff2e4160bb0 2023-11-18T20:45:10.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.524+0000 7ff2ea990640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e415a1c0 con 0x7ff2e40b3fc0 2023-11-18T20:45:10.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.524+0000 7ff2e37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e40ce440 0x7ff2e40cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e40ce440 0x7ff2e40cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56170/0 (socket says 172.21.15.60:56170) 2023-11-18T20:45:10.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e37fe640 1 -- 172.21.15.60:0/1441623740 learned_addr learned my addr 172.21.15.60:0/1441623740 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:10.530 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e3fff640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e40b3fc0 0x7ff2e41f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e8f2d640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e4160bb0 0x7ff2e4159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e37fe640 1 -- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e40b3fc0 msgr2=0x7ff2e41f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e37fe640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e40b3fc0 0x7ff2e41f2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e37fe640 1 -- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e4160bb0 msgr2=0x7ff2e4159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e37fe640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e4160bb0 0x7ff2e4159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e37fe640 1 -- 172.21.15.60:0/1441623740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc009a70 con 0x7ff2e40ce440 2023-11-18T20:45:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e37fe640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e40ce440 0x7ff2e40cc180 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800c500 tx=0x7ff2d800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.531 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e0ff9640 1 -- 172.21.15.60:0/1441623740 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8005ce0 con 0x7ff2e40ce440 2023-11-18T20:45:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e0ff9640 1 -- 172.21.15.60:0/1441623740 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d8005e40 con 0x7ff2e40ce440 2023-11-18T20:45:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d4009400 con 0x7ff2e40ce440 2023-11-18T20:45:10.532 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2e0ff9640 1 -- 172.21.15.60:0/1441623740 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80072b0 con 0x7ff2e40ce440 2023-11-18T20:45:10.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.525+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e41f6b60 con 0x7ff2e40ce440 2023-11-18T20:45:10.533 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.526+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff2e4160f90 con 0x7ff2e40ce440 2023-11-18T20:45:10.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.527+0000 7ff2e0ff9640 1 -- 172.21.15.60:0/1441623740 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff2d80074d0 con 0x7ff2e40ce440 2023-11-18T20:45:10.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.528+0000 7ff2e0ff9640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff2d006ae40 0x7ff2d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.528+0000 7ff2e0ff9640 1 -- 172.21.15.60:0/1441623740 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff2d80886b0 con 0x7ff2e40ce440 2023-11-18T20:45:10.534 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.528+0000 7ff2e0ff9640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] conn(0x7ff2d0070ca0 0x7ff2d00730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.528+0000 7ff2e3fff640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff2d006ae40 0x7ff2d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.528+0000 7ff2e0ff9640 1 -- 172.21.15.60:0/1441623740 --> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2d0073770 con 0x7ff2d0070ca0 2023-11-18T20:45:10.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.528+0000 7ff2e0ff9640 1 -- 172.21.15.60:0/1441623740 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2d808ad70 con 0x7ff2e40ce440 2023-11-18T20:45:10.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.528+0000 7ff2e8f2d640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] conn(0x7ff2d0070ca0 0x7ff2d00730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.528+0000 7ff2e3fff640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff2d006ae40 0x7ff2d006d300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4036000 tx=0x7ff2d403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.528+0000 7ff2e8f2d640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] conn(0x7ff2d0070ca0 0x7ff2d00730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.535 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.530+0000 7ff2e0ff9640 1 -- 172.21.15.60:0/1441623740 <== osd.0 v2:172.21.15.60:6800/2078754623 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff2d0073770 con 0x7ff2d0070ca0 2023-11-18T20:45:10.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.543+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 --> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff2e40b3980 con 0x7ff2d0070ca0 2023-11-18T20:45:10.544 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.543+0000 7ff2e0ff9640 1 -- 172.21.15.60:0/1441623740 <== osd.0 v2:172.21.15.60:6800/2078754623 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff2dc00f040 con 0x7ff2d0070ca0 2023-11-18T20:45:10.545 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:45:10.545 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-18T20:45:10.545 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:45:10.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.543+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 >> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] conn(0x7ff2d0070ca0 msgr2=0x7ff2d00730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.544+0000 7ff2ea990640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.60:6800/2078754623,v1:172.21.15.60:6801/2078754623] conn(0x7ff2d0070ca0 0x7ff2d00730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.546 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.544+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff2d006ae40 msgr2=0x7ff2d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.544+0000 7ff2ea990640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7ff2d006ae40 0x7ff2d006d300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4036000 tx=0x7ff2d403c040 comp rx=0 tx=0).stop 2023-11-18T20:45:10.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.544+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e40ce440 msgr2=0x7ff2e40cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.544+0000 7ff2ea990640 1 --2- 172.21.15.60:0/1441623740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e40ce440 0x7ff2e40cc180 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800c500 tx=0x7ff2d800c900 comp rx=0 tx=0).stop 2023-11-18T20:45:10.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.544+0000 7ff2e8f2d640 1 -- 172.21.15.60:0/1441623740 reap_dead start 2023-11-18T20:45:10.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.544+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 shutdown_connections 2023-11-18T20:45:10.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.544+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 >> 172.21.15.60:0/1441623740 conn(0x7ff2e409c590 msgr2=0x7ff2e40b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:10.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.544+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 shutdown_connections 2023-11-18T20:45:10.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.544+0000 7ff2ea990640 1 -- 172.21.15.60:0/1441623740 wait complete. 2023-11-18T20:45:10.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-18T20:45:10.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.664+0000 7f1bd2664640 1 Processor -- start 2023-11-18T20:45:10.665 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.664+0000 7f1bd2664640 1 -- start start 2023-11-18T20:45:10.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bd2664640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1bcc15d330 0x7f1bcc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bd2664640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1bcc0d2910 0x7f1bcc0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bd2664640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc0b2b10 con 0x7f1bcc0d3730 2023-11-18T20:45:10.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bd2664640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc0d3bc0 con 0x7f1bcc15d330 2023-11-18T20:45:10.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bd2664640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc0d1590 con 0x7f1bcc0d2910 2023-11-18T20:45:10.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bcbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1bcc15d330 0x7f1bcc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bcbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1bcc15d330 0x7f1bcc0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57924/0 (socket says 172.21.15.60:57924) 2023-11-18T20:45:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bcbfff640 1 -- 172.21.15.60:0/1452645830 learned_addr learned my addr 172.21.15.60:0/1452645830 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bcb7fe640 1 --2- 172.21.15.60:0/1452645830 >> v2:172.21.15.100:3300/0 conn(0x7f1bcc0d2910 0x7f1bcc0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bcbfff640 1 -- 172.21.15.60:0/1452645830 >> v2:172.21.15.100:3300/0 conn(0x7f1bcc0d2910 msgr2=0x7f1bcc0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bcbfff640 1 --2- 172.21.15.60:0/1452645830 >> v2:172.21.15.100:3300/0 conn(0x7f1bcc0d2910 0x7f1bcc0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bcbfff640 1 -- 172.21.15.60:0/1452645830 >> v1:172.21.15.100:6789/0 conn(0x7f1bcc0d3730 legacy=0x7f1bcc0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.665+0000 7f1bcbfff640 1 -- 172.21.15.60:0/1452645830 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0d1870 con 0x7f1bcc15d330 2023-11-18T20:45:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.666+0000 7f1bcbfff640 1 --2- 172.21.15.60:0/1452645830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc15d330 0x7f1bcc0b8b80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0d1540 tx=0x7f1bb402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c29054f01c91e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.666+0000 7f1bcaffd640 1 -- 172.21.15.60:0/1452645830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4031870 con 0x7f1bcc15d330 2023-11-18T20:45:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.666+0000 7f1bcaffd640 1 -- 172.21.15.60:0/1452645830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb40319d0 con 0x7f1bcc15d330 2023-11-18T20:45:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.666+0000 7f1bcaffd640 1 -- 172.21.15.60:0/1452645830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4031d60 con 0x7f1bcc15d330 2023-11-18T20:45:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.667+0000 7f1bd2664640 1 -- 172.21.15.60:0/1452645830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc15d330 msgr2=0x7f1bcc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.670 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.667+0000 7f1bd2664640 1 --2- 172.21.15.60:0/1452645830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc15d330 0x7f1bcc0b8b80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0d1540 tx=0x7f1bb402b910 comp rx=0 tx=0).stop 2023-11-18T20:45:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.667+0000 7f1bd2664640 1 -- 172.21.15.60:0/1452645830 shutdown_connections 2023-11-18T20:45:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.667+0000 7f1bd2664640 1 --2- 172.21.15.60:0/1452645830 >> v2:172.21.15.100:3300/0 conn(0x7f1bcc0d2910 0x7f1bcc0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.667+0000 7f1bd2664640 1 --2- 172.21.15.60:0/1452645830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc15d330 0x7f1bcc0b8b80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.667+0000 7f1bd2664640 1 -- 172.21.15.60:0/1452645830 >> 172.21.15.60:0/1452645830 conn(0x7f1bcc072020 msgr2=0x7f1bcc072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.667+0000 7f1bd2664640 1 -- 172.21.15.60:0/1452645830 shutdown_connections 2023-11-18T20:45:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.667+0000 7f1bd2664640 1 -- 172.21.15.60:0/1452645830 wait complete. 2023-11-18T20:45:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.668+0000 7f1bd2664640 1 Processor -- start 2023-11-18T20:45:10.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.668+0000 7f1bd2664640 1 -- start start 2023-11-18T20:45:10.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bd2664640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bcc0d2910 0x7f1bcc156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bd2664640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0d3730 0x7f1bcc15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bd2664640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc15d330 0x7f1bcc1602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bd2664640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc157d40 con 0x7f1bcc0d3730 2023-11-18T20:45:10.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bd2664640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc160810 con 0x7f1bcc15d330 2023-11-18T20:45:10.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bd2664640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0c8100 con 0x7f1bcc0d2910 2023-11-18T20:45:10.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bcb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0d3730 0x7f1bcc15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bcb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0d3730 0x7f1bcc15fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57928/0 (socket says 172.21.15.60:57928) 2023-11-18T20:45:10.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bcb7fe640 1 -- 172.21.15.60:0/3557985586 learned_addr learned my addr 172.21.15.60:0/3557985586 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:10.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bcbfff640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bcc0d2910 0x7f1bcc156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.669+0000 7f1bd0c01640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc15d330 0x7f1bcc1602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bcb7fe640 1 -- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bcc0d2910 msgr2=0x7f1bcc156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bcb7fe640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bcc0d2910 0x7f1bcc156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bcb7fe640 1 -- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc15d330 msgr2=0x7f1bcc1602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bcb7fe640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc15d330 0x7f1bcc1602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bcb7fe640 1 -- 172.21.15.60:0/3557985586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40094d0 con 0x7f1bcc0d3730 2023-11-18T20:45:10.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bcb7fe640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0d3730 0x7f1bcc15fd90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e9e0 tx=0x7f1bbc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/3557985586 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc009670 con 0x7f1bcc0d3730 2023-11-18T20:45:10.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/3557985586 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc00c9d0 con 0x7f1bcc0d3730 2023-11-18T20:45:10.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bd2664640 1 -- 172.21.15.60:0/3557985586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc157780 con 0x7f1bcc0d3730 2023-11-18T20:45:10.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/3557985586 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00cd80 con 0x7f1bcc0d3730 2023-11-18T20:45:10.676 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.670+0000 7f1bd2664640 1 -- 172.21.15.60:0/3557985586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc0c7d90 con 0x7f1bcc0d3730 2023-11-18T20:45:10.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.671+0000 7f1bd2664640 1 -- 172.21.15.60:0/3557985586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc0c7bf0 con 0x7f1bcc0d3730 2023-11-18T20:45:10.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.672+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/3557985586 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1bbc013470 con 0x7f1bcc0d3730 2023-11-18T20:45:10.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.672+0000 7f1bc8ff9640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1bac06ae40 0x7f1bac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.672+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/3557985586 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1bbc089440 con 0x7f1bcc0d3730 2023-11-18T20:45:10.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.672+0000 7f1bcbfff640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1bac06ae40 0x7f1bac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.673+0000 7f1bcbfff640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1bac06ae40 0x7f1bac06d300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4036000 tx=0x7f1bb403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.678 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.677+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/3557985586 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1bcc0c7bf0 con 0x7f1bcc0d3730 2023-11-18T20:45:10.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.855+0000 7f1bd2664640 1 -- 172.21.15.60:0/3557985586 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bcc0d2110 con 0x7f1bac06ae40 2023-11-18T20:45:10.857 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.855+0000 7f1bc8ff9640 1 -- 172.21.15.60:0/3557985586 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1bb4031bc0 con 0x7f1bac06ae40 2023-11-18T20:45:10.857 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 0 to deep-scrub 2023-11-18T20:45:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.857+0000 7f1bd2664640 1 -- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1bac06ae40 msgr2=0x7f1bac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.857+0000 7f1bd2664640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1bac06ae40 0x7f1bac06d300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4036000 tx=0x7f1bb403c040 comp rx=0 tx=0).stop 2023-11-18T20:45:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.857+0000 7f1bd2664640 1 -- 172.21.15.60:0/3557985586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0d3730 msgr2=0x7f1bcc15fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.857+0000 7f1bd2664640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0d3730 0x7f1bcc15fd90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e9e0 tx=0x7f1bbc00eea0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.858+0000 7f1bd2664640 1 -- 172.21.15.60:0/3557985586 shutdown_connections 2023-11-18T20:45:10.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.858+0000 7f1bd2664640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1bac06ae40 0x7f1bac06d300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.858+0000 7f1bd2664640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bcc15d330 0x7f1bcc1602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.858+0000 7f1bd2664640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bcc0d3730 0x7f1bcc15fd90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.858+0000 7f1bd2664640 1 --2- 172.21.15.60:0/3557985586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bcc0d2910 0x7f1bcc156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.858+0000 7f1bd2664640 1 -- 172.21.15.60:0/3557985586 >> 172.21.15.60:0/3557985586 conn(0x7f1bcc072020 msgr2=0x7f1bcc072400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:10.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.858+0000 7f1bd2664640 1 -- 172.21.15.60:0/3557985586 shutdown_connections 2023-11-18T20:45:10.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.858+0000 7f1bd2664640 1 -- 172.21.15.60:0/3557985586 wait complete. 2023-11-18T20:45:10.870 INFO:tasks.ceph:Scrubbing osd.1 2023-11-18T20:45:10.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-18T20:45:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.981+0000 7fbf15b9e640 1 Processor -- start 2023-11-18T20:45:10.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.981+0000 7fbf15b9e640 1 -- start start 2023-11-18T20:45:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.982+0000 7fbf15b9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf10071380 0x7fbf100cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.982+0000 7fbf15b9e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbf1006a4d0 0x7fbf1006a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.982+0000 7fbf15b9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1015b5d0 con 0x7fbf10144880 2023-11-18T20:45:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.982+0000 7fbf15b9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1006ae80 con 0x7fbf10071380 2023-11-18T20:45:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.982+0000 7fbf15b9e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf10071880 con 0x7fbf1006a4d0 2023-11-18T20:45:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.982+0000 7fbf0ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf10071380 0x7fbf100cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.982+0000 7fbf14b9c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbf10144880 0x7fbf1015c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57674/0 (socket says 172.21.15.60:57674) 2023-11-18T20:45:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.982+0000 7fbf14b9c640 1 -- 172.21.15.60:0/4172214293 learned_addr learned my addr 172.21.15.60:0/4172214293 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.982+0000 7fbf1539d640 1 --2- 172.21.15.60:0/4172214293 >> v2:172.21.15.100:3300/0 conn(0x7fbf1006a4d0 0x7fbf1006a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.983+0000 7fbf0ffff640 1 -- 172.21.15.60:0/4172214293 >> v2:172.21.15.100:3300/0 conn(0x7fbf1006a4d0 msgr2=0x7fbf1006a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.983+0000 7fbf0ffff640 1 --2- 172.21.15.60:0/4172214293 >> v2:172.21.15.100:3300/0 conn(0x7fbf1006a4d0 0x7fbf1006a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.983+0000 7fbf0ffff640 1 -- 172.21.15.60:0/4172214293 >> v1:172.21.15.60:6789/0 conn(0x7fbf10144880 legacy=0x7fbf1015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.983+0000 7fbf0ffff640 1 -- 172.21.15.60:0/4172214293 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10071b60 con 0x7fbf10071380 2023-11-18T20:45:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.983+0000 7fbf0f7fe640 1 -- 172.21.15.60:0/4172214293 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742428795 0 0) 0x7fbf1015b5d0 con 0x7fbf10144880 2023-11-18T20:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.983+0000 7fbf0ffff640 1 --2- 172.21.15.60:0/4172214293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf10071380 0x7fbf100cf620 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbf100afb90 tx=0x7fbefc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd9cc5c48891c9f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf0f7fe640 1 -- 172.21.15.60:0/4172214293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc031830 con 0x7fbf10071380 2023-11-18T20:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf0f7fe640 1 -- 172.21.15.60:0/4172214293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc031990 con 0x7fbf10071380 2023-11-18T20:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf0f7fe640 1 -- 172.21.15.60:0/4172214293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc031d20 con 0x7fbf10071380 2023-11-18T20:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf15b9e640 1 -- 172.21.15.60:0/4172214293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf10071380 msgr2=0x7fbf100cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf15b9e640 1 --2- 172.21.15.60:0/4172214293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf10071380 0x7fbf100cf620 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbf100afb90 tx=0x7fbefc02b8d0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf15b9e640 1 -- 172.21.15.60:0/4172214293 shutdown_connections 2023-11-18T20:45:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf15b9e640 1 --2- 172.21.15.60:0/4172214293 >> v2:172.21.15.100:3300/0 conn(0x7fbf1006a4d0 0x7fbf1006a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf15b9e640 1 --2- 172.21.15.60:0/4172214293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf10071380 0x7fbf100cf620 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf15b9e640 1 -- 172.21.15.60:0/4172214293 >> 172.21.15.60:0/4172214293 conn(0x7fbf1006b5e0 msgr2=0x7fbf1006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf15b9e640 1 -- 172.21.15.60:0/4172214293 shutdown_connections 2023-11-18T20:45:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.984+0000 7fbf15b9e640 1 -- 172.21.15.60:0/4172214293 wait complete. 2023-11-18T20:45:10.990 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.986+0000 7fbf15b9e640 1 Processor -- start 2023-11-18T20:45:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.986+0000 7fbf15b9e640 1 -- start start 2023-11-18T20:45:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.986+0000 7fbf15b9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf1006a4d0 0x7fbf1015a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.986+0000 7fbf15b9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf10071380 0x7fbf100b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.986+0000 7fbf15b9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf10144880 0x7fbf100c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf15b9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf10139140 con 0x7fbf10071380 2023-11-18T20:45:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf15b9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100c52c0 con 0x7fbf10144880 2023-11-18T20:45:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf15b9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100c55a0 con 0x7fbf1006a4d0 2023-11-18T20:45:10.991 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf0ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf10071380 0x7fbf100b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf14b9c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf1006a4d0 0x7fbf1015a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf1539d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf10144880 0x7fbf100c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf0ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf10071380 0x7fbf100b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57946/0 (socket says 172.21.15.60:57946) 2023-11-18T20:45:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf0ffff640 1 -- 172.21.15.60:0/3902684789 learned_addr learned my addr 172.21.15.60:0/3902684789 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf0ffff640 1 -- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf1006a4d0 msgr2=0x7fbf1015a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf0ffff640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf1006a4d0 0x7fbf1015a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf0ffff640 1 -- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf10144880 msgr2=0x7fbf100c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:10.992 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf0ffff640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf10144880 0x7fbf100c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf0ffff640 1 -- 172.21.15.60:0/3902684789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc009400 con 0x7fbf10071380 2023-11-18T20:45:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf1539d640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf10144880 0x7fbf100c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf14b9c640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf1006a4d0 0x7fbf1015a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf0ffff640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf10071380 0x7fbf100b3990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fbefc000c00 tx=0x7fbefc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.987+0000 7fbf0d7fa640 1 -- 172.21.15.60:0/3902684789 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc004230 con 0x7fbf10071380 2023-11-18T20:45:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.988+0000 7fbf0d7fa640 1 -- 172.21.15.60:0/3902684789 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc0043b0 con 0x7fbf10071380 2023-11-18T20:45:10.993 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.988+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf100c5880 con 0x7fbf10071380 2023-11-18T20:45:10.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.988+0000 7fbf0d7fa640 1 -- 172.21.15.60:0/3902684789 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc004520 con 0x7fbf10071380 2023-11-18T20:45:10.994 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.988+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf100c5e40 con 0x7fbf10071380 2023-11-18T20:45:10.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.988+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbf1015cd00 con 0x7fbf10071380 2023-11-18T20:45:10.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.989+0000 7fbf0d7fa640 1 -- 172.21.15.60:0/3902684789 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbefc03a8a0 con 0x7fbf10071380 2023-11-18T20:45:10.995 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.989+0000 7fbf0d7fa640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbef406ae40 0x7fbef406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.990+0000 7fbf0d7fa640 1 -- 172.21.15.60:0/3902684789 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbefc0ab9d0 con 0x7fbf10071380 2023-11-18T20:45:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.990+0000 7fbf14b9c640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbef406ae40 0x7fbef406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.990+0000 7fbf0d7fa640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] conn(0x7fbef4070ca0 0x7fbef40730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.990+0000 7fbf0d7fa640 1 -- 172.21.15.60:0/3902684789 --> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbef4073770 con 0x7fbef4070ca0 2023-11-18T20:45:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.990+0000 7fbf0d7fa640 1 -- 172.21.15.60:0/3902684789 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbefc0b02c0 con 0x7fbf10071380 2023-11-18T20:45:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.990+0000 7fbf1539d640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] conn(0x7fbef4070ca0 0x7fbef40730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:10.996 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.990+0000 7fbf14b9c640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbef406ae40 0x7fbef406d300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbf04000f60 tx=0x7fbf04007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.990+0000 7fbf1539d640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] conn(0x7fbef4070ca0 0x7fbef40730c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:10.997 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:10.991+0000 7fbf0d7fa640 1 -- 172.21.15.60:0/3902684789 <== osd.1 v2:172.21.15.60:6809/3839540220 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fbef4073770 con 0x7fbef4070ca0 2023-11-18T20:45:11.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.004+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 --> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbf10072120 con 0x7fbef4070ca0 2023-11-18T20:45:11.005 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.004+0000 7fbf0d7fa640 1 -- 172.21.15.60:0/3902684789 <== osd.1 v2:172.21.15.60:6809/3839540220 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbf0800f040 con 0x7fbef4070ca0 2023-11-18T20:45:11.006 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:45:11.006 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-18T20:45:11.006 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:45:11.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.004+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 >> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] conn(0x7fbef4070ca0 msgr2=0x7fbef40730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.004+0000 7fbf15b9e640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.60:6809/3839540220,v1:172.21.15.60:6812/3839540220] conn(0x7fbef4070ca0 0x7fbef40730c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.004+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbef406ae40 msgr2=0x7fbef406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.004+0000 7fbf15b9e640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbef406ae40 0x7fbef406d300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbf04000f60 tx=0x7fbf04007d50 comp rx=0 tx=0).stop 2023-11-18T20:45:11.007 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.005+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf10071380 msgr2=0x7fbf100b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.005+0000 7fbf15b9e640 1 --2- 172.21.15.60:0/3902684789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf10071380 0x7fbf100b3990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fbefc000c00 tx=0x7fbefc0046c0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.005+0000 7fbf1539d640 1 -- 172.21.15.60:0/3902684789 reap_dead start 2023-11-18T20:45:11.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.005+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 shutdown_connections 2023-11-18T20:45:11.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.005+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 >> 172.21.15.60:0/3902684789 conn(0x7fbf1006b5e0 msgr2=0x7fbf100b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:11.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.005+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 shutdown_connections 2023-11-18T20:45:11.008 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.005+0000 7fbf15b9e640 1 -- 172.21.15.60:0/3902684789 wait complete. 2023-11-18T20:45:11.016 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-18T20:45:11.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.123+0000 7fdc10197640 1 Processor -- start 2023-11-18T20:45:11.124 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.123+0000 7fdc10197640 1 -- start start 2023-11-18T20:45:11.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.124+0000 7fdc10197640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d2910 0x7fdc080d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.124+0000 7fdc10197640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdc080d3730 0x7fdc080b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.124+0000 7fdc10197640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc080b2b10 con 0x7fdc0815d330 2023-11-18T20:45:11.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.124+0000 7fdc10197640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc080d3b50 con 0x7fdc080d3730 2023-11-18T20:45:11.126 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.124+0000 7fdc10197640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080d1590 con 0x7fdc080d2910 2023-11-18T20:45:11.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.124+0000 7fdc0d732640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d2910 0x7fdc080d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.124+0000 7fdc0e734640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdc080d3730 0x7fdc080b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.124+0000 7fdc0d732640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d2910 0x7fdc080d31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:33454/0 (socket says 172.21.15.60:33454) 2023-11-18T20:45:11.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.124+0000 7fdc0d732640 1 -- 172.21.15.60:0/635474702 learned_addr learned my addr 172.21.15.60:0/635474702 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:11.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.124+0000 7fdc0df33640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdc0815d330 0x7fdc080b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:46436/0 (socket says 172.21.15.60:46436) 2023-11-18T20:45:11.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.125+0000 7fdc0cf31640 1 -- 172.21.15.60:0/635474702 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99622985 0 0) 0x7fdc080b2b10 con 0x7fdc0815d330 2023-11-18T20:45:11.127 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.125+0000 7fdc0cf31640 1 -- 172.21.15.60:0/635474702 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbf0003520 con 0x7fdc0815d330 2023-11-18T20:45:11.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.125+0000 7fdc0d732640 1 -- 172.21.15.60:0/635474702 >> v2:172.21.15.100:3300/0 conn(0x7fdc080d3730 msgr2=0x7fdc080b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:11.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.125+0000 7fdc0d732640 1 --2- 172.21.15.60:0/635474702 >> v2:172.21.15.100:3300/0 conn(0x7fdc080d3730 0x7fdc080b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.125+0000 7fdc0d732640 1 -- 172.21.15.60:0/635474702 >> v1:172.21.15.100:6789/0 conn(0x7fdc0815d330 legacy=0x7fdc080b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.125+0000 7fdc0d732640 1 -- 172.21.15.60:0/635474702 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc080d1870 con 0x7fdc080d2910 2023-11-18T20:45:11.128 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.125+0000 7fdc0e734640 1 --2- 172.21.15.60:0/635474702 >> v2:172.21.15.100:3300/0 conn(0x7fdc080d3730 0x7fdc080b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:11.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.125+0000 7fdc0d732640 1 --2- 172.21.15.60:0/635474702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d2910 0x7fdc080d31f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009870 tx=0x7fdbfc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ce36f29dbb593c server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc0cf31640 1 -- 172.21.15.60:0/635474702 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc031640 con 0x7fdc080d2910 2023-11-18T20:45:11.131 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc0cf31640 1 -- 172.21.15.60:0/635474702 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbfc0317a0 con 0x7fdc080d2910 2023-11-18T20:45:11.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc0cf31640 1 -- 172.21.15.60:0/635474702 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc031b30 con 0x7fdc080d2910 2023-11-18T20:45:11.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc10197640 1 -- 172.21.15.60:0/635474702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d2910 msgr2=0x7fdc080d31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc10197640 1 --2- 172.21.15.60:0/635474702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d2910 0x7fdc080d31f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009870 tx=0x7fdbfc02b8f0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc10197640 1 -- 172.21.15.60:0/635474702 shutdown_connections 2023-11-18T20:45:11.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc10197640 1 --2- 172.21.15.60:0/635474702 >> v2:172.21.15.100:3300/0 conn(0x7fdc080d3730 0x7fdc080b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc10197640 1 --2- 172.21.15.60:0/635474702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d2910 0x7fdc080d31f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc10197640 1 -- 172.21.15.60:0/635474702 >> 172.21.15.60:0/635474702 conn(0x7fdc08072020 msgr2=0x7fdc08072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:11.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc10197640 1 -- 172.21.15.60:0/635474702 shutdown_connections 2023-11-18T20:45:11.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.126+0000 7fdc10197640 1 -- 172.21.15.60:0/635474702 wait complete. 2023-11-18T20:45:11.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.127+0000 7fdc10197640 1 Processor -- start 2023-11-18T20:45:11.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.127+0000 7fdc10197640 1 -- start start 2023-11-18T20:45:11.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc10197640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc080d2910 0x7fdc080ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc10197640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d3730 0x7fdc08152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc10197640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0815d330 0x7fdc080cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.133 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc10197640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080cc820 con 0x7fdc0815d330 2023-11-18T20:45:11.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc10197640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc08152c80 con 0x7fdc080d2910 2023-11-18T20:45:11.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc10197640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080cbef0 con 0x7fdc080d3730 2023-11-18T20:45:11.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc0e734640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0815d330 0x7fdc080cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc0e734640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0815d330 0x7fdc080cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57962/0 (socket says 172.21.15.60:57962) 2023-11-18T20:45:11.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc0e734640 1 -- 172.21.15.60:0/849221353 learned_addr learned my addr 172.21.15.60:0/849221353 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:11.134 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc0d732640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d3730 0x7fdc08152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.128+0000 7fdc0df33640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc080d2910 0x7fdc080ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc0e734640 1 -- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d3730 msgr2=0x7fdc08152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc0e734640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d3730 0x7fdc08152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc0e734640 1 -- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc080d2910 msgr2=0x7fdc080ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc0e734640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc080d2910 0x7fdc080ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc0e734640 1 -- 172.21.15.60:0/849221353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf8009400 con 0x7fdc0815d330 2023-11-18T20:45:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc0e734640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0815d330 0x7fdc080cccb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800e880 tx=0x7fdbf800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc067fc640 1 -- 172.21.15.60:0/849221353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8009660 con 0x7fdc0815d330 2023-11-18T20:45:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc067fc640 1 -- 172.21.15.60:0/849221353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf8004020 con 0x7fdc0815d330 2023-11-18T20:45:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc10197640 1 -- 172.21.15.60:0/849221353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc0094d0 con 0x7fdc0815d330 2023-11-18T20:45:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc067fc640 1 -- 172.21.15.60:0/849221353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf80043b0 con 0x7fdc0815d330 2023-11-18T20:45:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.129+0000 7fdc10197640 1 -- 172.21.15.60:0/849221353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc080cc4b0 con 0x7fdc0815d330 2023-11-18T20:45:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.130+0000 7fdc10197640 1 -- 172.21.15.60:0/849221353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc080cc990 con 0x7fdc0815d330 2023-11-18T20:45:11.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.131+0000 7fdc067fc640 1 -- 172.21.15.60:0/849221353 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdbf800c940 con 0x7fdc0815d330 2023-11-18T20:45:11.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.131+0000 7fdc067fc640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdbf406ae40 0x7fdbf406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.137 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.131+0000 7fdc067fc640 1 -- 172.21.15.60:0/849221353 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdbf8089490 con 0x7fdc0815d330 2023-11-18T20:45:11.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.131+0000 7fdc0df33640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdbf406ae40 0x7fdbf406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.132+0000 7fdc0df33640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdbf406ae40 0x7fdbf406d300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdc00001560 tx=0x7fdc00001700 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.138 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.135+0000 7fdc067fc640 1 -- 172.21.15.60:0/849221353 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdbf80548f0 con 0x7fdc0815d330 2023-11-18T20:45:11.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.315+0000 7fdc10197640 1 -- 172.21.15.60:0/849221353 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc080d2110 con 0x7fdbf406ae40 2023-11-18T20:45:11.317 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.316+0000 7fdc067fc640 1 -- 172.21.15.60:0/849221353 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdc00004de0 con 0x7fdbf406ae40 2023-11-18T20:45:11.317 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 1 to deep-scrub 2023-11-18T20:45:11.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.318+0000 7fdc10197640 1 -- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdbf406ae40 msgr2=0x7fdbf406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.318+0000 7fdc10197640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdbf406ae40 0x7fdbf406d300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdc00001560 tx=0x7fdc00001700 comp rx=0 tx=0).stop 2023-11-18T20:45:11.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.318+0000 7fdc10197640 1 -- 172.21.15.60:0/849221353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0815d330 msgr2=0x7fdc080cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.318+0000 7fdc10197640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0815d330 0x7fdc080cccb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800e880 tx=0x7fdbf800ed40 comp rx=0 tx=0).stop 2023-11-18T20:45:11.320 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.318+0000 7fdc10197640 1 -- 172.21.15.60:0/849221353 shutdown_connections 2023-11-18T20:45:11.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.318+0000 7fdc10197640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fdbf406ae40 0x7fdbf406d300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.318+0000 7fdc10197640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0815d330 0x7fdc080cccb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.318+0000 7fdc10197640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc080d3730 0x7fdc08152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.318+0000 7fdc10197640 1 --2- 172.21.15.60:0/849221353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc080d2910 0x7fdc080ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.319+0000 7fdc10197640 1 -- 172.21.15.60:0/849221353 >> 172.21.15.60:0/849221353 conn(0x7fdc08072020 msgr2=0x7fdc080d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:11.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.319+0000 7fdc10197640 1 -- 172.21.15.60:0/849221353 shutdown_connections 2023-11-18T20:45:11.321 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.319+0000 7fdc10197640 1 -- 172.21.15.60:0/849221353 wait complete. 2023-11-18T20:45:11.331 INFO:tasks.ceph:Scrubbing osd.2 2023-11-18T20:45:11.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-18T20:45:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.440+0000 7f436f6cf640 1 Processor -- start 2023-11-18T20:45:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.440+0000 7f436f6cf640 1 -- start start 2023-11-18T20:45:11.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.441+0000 7f436f6cf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43680ce440 0x7f43680d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.442 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.441+0000 7f436f6cf640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43680b3450 con 0x7f43680b3fc0 2023-11-18T20:45:11.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.441+0000 7f436f6cf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43680b45b0 con 0x7f4368160bb0 2023-11-18T20:45:11.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.441+0000 7f436f6cf640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43680d4140 con 0x7f43680ce440 2023-11-18T20:45:11.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.441+0000 7f436dc6c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f43680ce440 0x7f43680d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.441+0000 7f436cc6a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f43680b3fc0 0x7f43680cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57682/0 (socket says 172.21.15.60:57682) 2023-11-18T20:45:11.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.441+0000 7f436cc6a640 1 -- 172.21.15.60:0/1497237705 learned_addr learned my addr 172.21.15.60:0/1497237705 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:11.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.441+0000 7f436dc6c640 1 -- 172.21.15.60:0/1497237705 >> v1:172.21.15.100:6789/0 conn(0x7f4368160bb0 legacy=0x7f43680d3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.443 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.441+0000 7f436dc6c640 1 -- 172.21.15.60:0/1497237705 >> v1:172.21.15.60:6789/0 conn(0x7f43680b3fc0 legacy=0x7f43680cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.444 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.441+0000 7f436dc6c640 1 -- 172.21.15.60:0/1497237705 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680b4890 con 0x7f43680ce440 2023-11-18T20:45:11.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.442+0000 7f436dc6c640 1 --2- 172.21.15.60:0/1497237705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680ce440 0x7f43680d1120 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4360009d90 tx=0x7f436002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3737cff1858b6c3f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.442+0000 7f435ffff640 1 -- 172.21.15.60:0/1497237705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436002cd00 con 0x7f43680ce440 2023-11-18T20:45:11.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.442+0000 7f435ffff640 1 -- 172.21.15.60:0/1497237705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436002ce60 con 0x7f43680ce440 2023-11-18T20:45:11.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.442+0000 7f435ffff640 1 -- 172.21.15.60:0/1497237705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360034660 con 0x7f43680ce440 2023-11-18T20:45:11.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.443+0000 7f436f6cf640 1 -- 172.21.15.60:0/1497237705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680ce440 msgr2=0x7f43680d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.445 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.443+0000 7f436f6cf640 1 --2- 172.21.15.60:0/1497237705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680ce440 0x7f43680d1120 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4360009d90 tx=0x7f436002bf50 comp rx=0 tx=0).stop 2023-11-18T20:45:11.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.443+0000 7f436f6cf640 1 -- 172.21.15.60:0/1497237705 shutdown_connections 2023-11-18T20:45:11.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.443+0000 7f436f6cf640 1 --2- 172.21.15.60:0/1497237705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43680ce440 0x7f43680d1120 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.443+0000 7f436f6cf640 1 -- 172.21.15.60:0/1497237705 >> 172.21.15.60:0/1497237705 conn(0x7f436809c590 msgr2=0x7f43680d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:11.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.443+0000 7f436f6cf640 1 -- 172.21.15.60:0/1497237705 shutdown_connections 2023-11-18T20:45:11.446 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.443+0000 7f436f6cf640 1 -- 172.21.15.60:0/1497237705 wait complete. 2023-11-18T20:45:11.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.444+0000 7f436f6cf640 1 Processor -- start 2023-11-18T20:45:11.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.444+0000 7f436f6cf640 1 -- start start 2023-11-18T20:45:11.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436f6cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43680b3fc0 0x7f43680cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436f6cf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43680ce440 0x7f43680cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436f6cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4368160bb0 0x7f43680b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436f6cf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436815c6c0 con 0x7f4368160bb0 2023-11-18T20:45:11.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436f6cf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43680ca2f0 con 0x7f43680b3fc0 2023-11-18T20:45:11.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436f6cf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43680ca5d0 con 0x7f43680ce440 2023-11-18T20:45:11.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436dc6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4368160bb0 0x7f43680b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436dc6c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4368160bb0 0x7f43680b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57976/0 (socket says 172.21.15.60:57976) 2023-11-18T20:45:11.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436dc6c640 1 -- 172.21.15.60:0/332854127 learned_addr learned my addr 172.21.15.60:0/332854127 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:11.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436d46b640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43680b3fc0 0x7f43680cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436cc6a640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43680ce440 0x7f43680cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436dc6c640 1 -- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43680ce440 msgr2=0x7f43680cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436dc6c640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43680ce440 0x7f43680cb8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436dc6c640 1 -- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43680b3fc0 msgr2=0x7f43680cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436dc6c640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43680b3fc0 0x7f43680cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.450 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.445+0000 7f436dc6c640 1 -- 172.21.15.60:0/332854127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360009a70 con 0x7f4368160bb0 2023-11-18T20:45:11.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.446+0000 7f436dc6c640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4368160bb0 0x7f43680b6850 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4360009d90 tx=0x7f4360004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.446+0000 7f435dffb640 1 -- 172.21.15.60:0/332854127 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436003d600 con 0x7f4368160bb0 2023-11-18T20:45:11.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.446+0000 7f435dffb640 1 -- 172.21.15.60:0/332854127 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436003d780 con 0x7f4368160bb0 2023-11-18T20:45:11.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.446+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43680ca8b0 con 0x7f4368160bb0 2023-11-18T20:45:11.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.446+0000 7f435dffb640 1 -- 172.21.15.60:0/332854127 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436003db30 con 0x7f4368160bb0 2023-11-18T20:45:11.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.446+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43680cae70 con 0x7f4368160bb0 2023-11-18T20:45:11.451 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.447+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f43680ce870 con 0x7f4368160bb0 2023-11-18T20:45:11.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.447+0000 7f435dffb640 1 -- 172.21.15.60:0/332854127 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f436003b040 con 0x7f4368160bb0 2023-11-18T20:45:11.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.448+0000 7f435dffb640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f434c06ae40 0x7f434c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.448+0000 7f435dffb640 1 -- 172.21.15.60:0/332854127 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f43600abfc0 con 0x7f4368160bb0 2023-11-18T20:45:11.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.448+0000 7f435dffb640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] conn(0x7f434c070ca0 0x7f434c0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.448+0000 7f435dffb640 1 -- 172.21.15.60:0/332854127 --> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f434c073770 con 0x7f434c070ca0 2023-11-18T20:45:11.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.448+0000 7f435dffb640 1 -- 172.21.15.60:0/332854127 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43600ac410 con 0x7f4368160bb0 2023-11-18T20:45:11.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.448+0000 7f436d46b640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f434c06ae40 0x7f434c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.448+0000 7f436cc6a640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] conn(0x7f434c070ca0 0x7f434c0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.448+0000 7f436cc6a640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] conn(0x7f434c070ca0 0x7f434c0730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.448+0000 7f436d46b640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f434c06ae40 0x7f434c06d300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4358000f60 tx=0x7f4358009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.449+0000 7f435dffb640 1 -- 172.21.15.60:0/332854127 <== osd.2 v2:172.21.15.60:6808/479181725 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f434c073770 con 0x7f434c070ca0 2023-11-18T20:45:11.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.462+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 --> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f43680b3980 con 0x7f434c070ca0 2023-11-18T20:45:11.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.462+0000 7f435dffb640 1 -- 172.21.15.60:0/332854127 <== osd.2 v2:172.21.15.60:6808/479181725 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4364001830 con 0x7f434c070ca0 2023-11-18T20:45:11.464 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:45:11.464 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-18T20:45:11.464 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:45:11.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 >> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] conn(0x7f434c070ca0 msgr2=0x7f434c0730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436f6cf640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.60:6808/479181725,v1:172.21.15.60:6810/479181725] conn(0x7f434c070ca0 0x7f434c0730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f434c06ae40 msgr2=0x7f434c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436f6cf640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f434c06ae40 0x7f434c06d300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4358000f60 tx=0x7f4358009040 comp rx=0 tx=0).stop 2023-11-18T20:45:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4368160bb0 msgr2=0x7f43680b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436f6cf640 1 --2- 172.21.15.60:0/332854127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4368160bb0 0x7f43680b6850 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4360009d90 tx=0x7f4360004a40 comp rx=0 tx=0).stop 2023-11-18T20:45:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436dc6c640 1 -- 172.21.15.60:0/332854127 reap_dead start 2023-11-18T20:45:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 shutdown_connections 2023-11-18T20:45:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 >> 172.21.15.60:0/332854127 conn(0x7f436809c590 msgr2=0x7f43680d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 shutdown_connections 2023-11-18T20:45:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.463+0000 7f436f6cf640 1 -- 172.21.15.60:0/332854127 wait complete. 2023-11-18T20:45:11.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-18T20:45:11.582 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.581+0000 7fa7dea51640 1 Processor -- start 2023-11-18T20:45:11.583 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.581+0000 7fa7dea51640 1 -- start start 2023-11-18T20:45:11.586 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7dea51640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80d3900 0x7fa7d80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7dea51640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d80b2b10 con 0x7fa7d815d330 2023-11-18T20:45:11.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7dea51640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d80d1590 con 0x7fa7d80d2910 2023-11-18T20:45:11.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7dea51640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80d1870 con 0x7fa7d80d3900 2023-11-18T20:45:11.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7d7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa7d815d330 0x7fa7d80b8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57688/0 (socket says 172.21.15.60:57688) 2023-11-18T20:45:11.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7d7fff640 1 -- 172.21.15.60:0/2625769381 learned_addr learned my addr 172.21.15.60:0/2625769381 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:11.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832327215 0 0) 0x7fa7d80b2b10 con 0x7fa7d815d330 2023-11-18T20:45:11.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c0003520 con 0x7fa7d815d330 2023-11-18T20:45:11.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7dcfee640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80d3900 0x7fa7d80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014298553 0 0) 0x7fa7c0003520 con 0x7fa7d815d330 2023-11-18T20:45:11.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80d3900 msgr2=0x7fa7d80b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:11.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7d6ffd640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80d3900 0x7fa7d80b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 >> v1:172.21.15.100:6789/0 conn(0x7fa7d80d2910 legacy=0x7fa7d80d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d80d1b50 con 0x7fa7d815d330 2023-11-18T20:45:11.588 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.582+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fa7cc001670 con 0x7fa7d815d330 2023-11-18T20:45:11.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 >> v1:172.21.15.60:6789/0 conn(0x7fa7d815d330 legacy=0x7fa7d80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d6ffd640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c00039a0 0x7fa7c0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d6ffd640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c0004350 0x7fa7c0004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d6ffd640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80d3900 0x7fa7c0005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80d1870 con 0x7fa7c0004350 2023-11-18T20:45:11.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c000ba30 con 0x7fa7c00039a0 2023-11-18T20:45:11.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c000bbf0 con 0x7fa7d80d3900 2023-11-18T20:45:11.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d7fff640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c00039a0 0x7fa7c0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d77fe640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c0004350 0x7fa7c0004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7dcfee640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80d3900 0x7fa7c0005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d7fff640 1 -- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80d3900 msgr2=0x7fa7c0005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:11.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d7fff640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80d3900 0x7fa7c0005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d7fff640 1 -- 172.21.15.60:0/2625769381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c0004350 msgr2=0x7fa7c0004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:11.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d7fff640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c0004350 0x7fa7c0004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d7fff640 1 -- 172.21.15.60:0/2625769381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c0003520 con 0x7fa7c00039a0 2023-11-18T20:45:11.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.583+0000 7fa7d77fe640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7c0004350 0x7fa7c0004820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:11.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.584+0000 7fa7d7fff640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c00039a0 0x7fa7c0003d80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00c210 tx=0x7fa7c00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e5cf41177c4346a server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.584+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc02cae0 con 0x7fa7c00039a0 2023-11-18T20:45:11.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.584+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc02cc60 con 0x7fa7c00039a0 2023-11-18T20:45:11.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.584+0000 7fa7d6ffd640 1 -- 172.21.15.60:0/2625769381 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0058d0 con 0x7fa7c00039a0 2023-11-18T20:45:11.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.585+0000 7fa7dea51640 1 -- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c00039a0 msgr2=0x7fa7c0003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.585+0000 7fa7dea51640 1 --2- 172.21.15.60:0/2625769381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c00039a0 0x7fa7c0003d80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00c210 tx=0x7fa7c00057f0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.585+0000 7fa7dcfee640 1 -- 172.21.15.60:0/2625769381 reap_dead start 2023-11-18T20:45:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.585+0000 7fa7dea51640 1 -- 172.21.15.60:0/2625769381 shutdown_connections 2023-11-18T20:45:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.585+0000 7fa7dea51640 1 -- 172.21.15.60:0/2625769381 >> 172.21.15.60:0/2625769381 conn(0x7fa7d8072020 msgr2=0x7fa7d8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.585+0000 7fa7dea51640 1 -- 172.21.15.60:0/2625769381 shutdown_connections 2023-11-18T20:45:11.592 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.585+0000 7fa7dea51640 1 -- 172.21.15.60:0/2625769381 wait complete. 2023-11-18T20:45:11.594 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.586+0000 7fa7dea51640 1 Processor -- start 2023-11-18T20:45:11.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.586+0000 7fa7dea51640 1 -- start start 2023-11-18T20:45:11.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dea51640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c00039a0 0x7fa7d8154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dea51640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80b3d60 0x7fa7d80c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.595 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dea51640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80b41c0 0x7fa7d80c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dea51640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80ca330 con 0x7fa7d80b41c0 2023-11-18T20:45:11.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dea51640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80ca610 con 0x7fa7c00039a0 2023-11-18T20:45:11.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dea51640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80ca8f0 con 0x7fa7d80b3d60 2023-11-18T20:45:11.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dcfee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80b41c0 0x7fa7d80c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.596 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dcfee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80b41c0 0x7fa7d80c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58002/0 (socket says 172.21.15.60:58002) 2023-11-18T20:45:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dcfee640 1 -- 172.21.15.60:0/1866505274 learned_addr learned my addr 172.21.15.60:0/1866505274 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7d77fe640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80b3d60 0x7fa7d80c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7d7fff640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c00039a0 0x7fa7d8154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dcfee640 1 -- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80b3d60 msgr2=0x7fa7d80c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dcfee640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80b3d60 0x7fa7d80c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dcfee640 1 -- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c00039a0 msgr2=0x7fa7d8154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dcfee640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c00039a0 0x7fa7d8154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.597 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dcfee640 1 -- 172.21.15.60:0/1866505274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d80cabd0 con 0x7fa7d80b41c0 2023-11-18T20:45:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.587+0000 7fa7dcfee640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80b41c0 0x7fa7d80c5920 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa7d808c510 tx=0x7fa7d000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.588+0000 7fa7d4ff9640 1 -- 172.21.15.60:0/1866505274 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000e040 con 0x7fa7d80b41c0 2023-11-18T20:45:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.588+0000 7fa7d4ff9640 1 -- 172.21.15.60:0/1866505274 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d000f040 con 0x7fa7d80b41c0 2023-11-18T20:45:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.588+0000 7fa7dea51640 1 -- 172.21.15.60:0/1866505274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d80caeb0 con 0x7fa7d80b41c0 2023-11-18T20:45:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.588+0000 7fa7d4ff9640 1 -- 172.21.15.60:0/1866505274 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0015670 con 0x7fa7d80b41c0 2023-11-18T20:45:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.588+0000 7fa7dea51640 1 -- 172.21.15.60:0/1866505274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d8155300 con 0x7fa7d80b41c0 2023-11-18T20:45:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.589+0000 7fa7dea51640 1 -- 172.21.15.60:0/1866505274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d80cb3d0 con 0x7fa7d80b41c0 2023-11-18T20:45:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.590+0000 7fa7d4ff9640 1 -- 172.21.15.60:0/1866505274 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7d00157d0 con 0x7fa7d80b41c0 2023-11-18T20:45:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.590+0000 7fa7d4ff9640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7c406ae70 0x7fa7c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.590+0000 7fa7d4ff9640 1 -- 172.21.15.60:0/1866505274 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa7d00897b0 con 0x7fa7d80b41c0 2023-11-18T20:45:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.590+0000 7fa7d7fff640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7c406ae70 0x7fa7c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.599 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.591+0000 7fa7d7fff640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7c406ae70 0x7fa7c406d330 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc034000 tx=0x7fa7cc009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.600 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.594+0000 7fa7d4ff9640 1 -- 172.21.15.60:0/1866505274 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7d80cb3d0 con 0x7fa7d80b41c0 2023-11-18T20:45:11.772 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.771+0000 7fa7dea51640 1 -- 172.21.15.60:0/1866505274 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7d80d23f0 con 0x7fa7c406ae70 2023-11-18T20:45:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.771+0000 7fa7d4ff9640 1 -- 172.21.15.60:0/1866505274 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa7cc02c960 con 0x7fa7c406ae70 2023-11-18T20:45:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 2 to deep-scrub 2023-11-18T20:45:11.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 -- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7c406ae70 msgr2=0x7fa7c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7c406ae70 0x7fa7c406d330 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc034000 tx=0x7fa7cc009030 comp rx=0 tx=0).stop 2023-11-18T20:45:11.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 -- 172.21.15.60:0/1866505274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80b41c0 msgr2=0x7fa7d80c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80b41c0 0x7fa7d80c5920 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa7d808c510 tx=0x7fa7d000f580 comp rx=0 tx=0).stop 2023-11-18T20:45:11.775 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 -- 172.21.15.60:0/1866505274 shutdown_connections 2023-11-18T20:45:11.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fa7c406ae70 0x7fa7c406d330 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80b41c0 0x7fa7d80c5920 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7d80b3d60 0x7fa7d80c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 --2- 172.21.15.60:0/1866505274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7c00039a0 0x7fa7d8154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 -- 172.21.15.60:0/1866505274 >> 172.21.15.60:0/1866505274 conn(0x7fa7d8072020 msgr2=0x7fa7d815d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:11.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.773+0000 7fa7dea51640 1 -- 172.21.15.60:0/1866505274 shutdown_connections 2023-11-18T20:45:11.776 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.774+0000 7fa7dea51640 1 -- 172.21.15.60:0/1866505274 wait complete. 2023-11-18T20:45:11.785 INFO:tasks.ceph:Scrubbing osd.3 2023-11-18T20:45:11.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-18T20:45:11.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.895+0000 7f5f96c15640 1 Processor -- start 2023-11-18T20:45:11.896 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.895+0000 7f5f96c15640 1 -- start start 2023-11-18T20:45:11.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f96c15640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f900b4970 0x7f5f9015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.898 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f96c15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f9006f0f0 0x7f5f9006baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f96c15640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f900b0aa0 con 0x7f5f90161020 2023-11-18T20:45:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f96c15640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9015e0d0 con 0x7f5f900b4970 2023-11-18T20:45:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f96c15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9015da30 con 0x7f5f9006f0f0 2023-11-18T20:45:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f949b1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f900b4970 0x7f5f9015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f87fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5f90161020 0x7f5f9006e9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57690/0 (socket says 172.21.15.60:57690) 2023-11-18T20:45:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f87fff640 1 -- 172.21.15.60:0/705264328 learned_addr learned my addr 172.21.15.60:0/705264328 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f951b2640 1 --2- 172.21.15.60:0/705264328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f9006f0f0 0x7f5f9006baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.899 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f949b1640 1 -- 172.21.15.60:0/705264328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f9006f0f0 msgr2=0x7f5f9006baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:11.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f949b1640 1 --2- 172.21.15.60:0/705264328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f9006f0f0 0x7f5f9006baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f949b1640 1 -- 172.21.15.60:0/705264328 >> v1:172.21.15.60:6789/0 conn(0x7f5f90161020 legacy=0x7f5f9006e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f949b1640 1 -- 172.21.15.60:0/705264328 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f90161a10 con 0x7f5f900b4970 2023-11-18T20:45:11.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.896+0000 7f5f877fe640 1 -- 172.21.15.60:0/705264328 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358690985 0 0) 0x7f5f900b0aa0 con 0x7f5f90161020 2023-11-18T20:45:11.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.897+0000 7f5f949b1640 1 --2- 172.21.15.60:0/705264328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f900b4970 0x7f5f9015e4f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0097d0 tx=0x7f5f8c02b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9533adfe12fbaa02 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.897+0000 7f5f877fe640 1 -- 172.21.15.60:0/705264328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c031770 con 0x7f5f900b4970 2023-11-18T20:45:11.900 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.897+0000 7f5f877fe640 1 -- 172.21.15.60:0/705264328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c0318d0 con 0x7f5f900b4970 2023-11-18T20:45:11.901 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.897+0000 7f5f877fe640 1 -- 172.21.15.60:0/705264328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c031c60 con 0x7f5f900b4970 2023-11-18T20:45:11.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.898+0000 7f5f96c15640 1 -- 172.21.15.60:0/705264328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f900b4970 msgr2=0x7f5f9015e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.898+0000 7f5f96c15640 1 --2- 172.21.15.60:0/705264328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f900b4970 0x7f5f9015e4f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0097d0 tx=0x7f5f8c02b810 comp rx=0 tx=0).stop 2023-11-18T20:45:11.903 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.898+0000 7f5f96c15640 1 -- 172.21.15.60:0/705264328 shutdown_connections 2023-11-18T20:45:11.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.898+0000 7f5f96c15640 1 --2- 172.21.15.60:0/705264328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f9006f0f0 0x7f5f9006baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.898+0000 7f5f96c15640 1 --2- 172.21.15.60:0/705264328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f900b4970 0x7f5f9015e4f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.898+0000 7f5f96c15640 1 -- 172.21.15.60:0/705264328 >> 172.21.15.60:0/705264328 conn(0x7f5f9006a6b0 msgr2=0x7f5f9006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:11.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.898+0000 7f5f96c15640 1 -- 172.21.15.60:0/705264328 shutdown_connections 2023-11-18T20:45:11.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.898+0000 7f5f96c15640 1 -- 172.21.15.60:0/705264328 wait complete. 2023-11-18T20:45:11.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.899+0000 7f5f96c15640 1 Processor -- start 2023-11-18T20:45:11.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.899+0000 7f5f96c15640 1 -- start start 2023-11-18T20:45:11.904 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f96c15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f9006f0f0 0x7f5f90155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f96c15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f900b4970 0x7f5f900b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f96c15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f90161020 0x7f5f900b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f96c15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f901560c0 con 0x7f5f900b4970 2023-11-18T20:45:11.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f96c15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900b0bd0 con 0x7f5f90161020 2023-11-18T20:45:11.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f96c15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900b0eb0 con 0x7f5f9006f0f0 2023-11-18T20:45:11.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f900b4970 0x7f5f900b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.905 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f900b4970 0x7f5f900b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58022/0 (socket says 172.21.15.60:58022) 2023-11-18T20:45:11.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f87fff640 1 -- 172.21.15.60:0/76231973 learned_addr learned my addr 172.21.15.60:0/76231973 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:11.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f951b2640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f90161020 0x7f5f900b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f949b1640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f9006f0f0 0x7f5f90155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f87fff640 1 -- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f9006f0f0 msgr2=0x7f5f90155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f87fff640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f9006f0f0 0x7f5f90155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f87fff640 1 -- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f90161020 msgr2=0x7f5f900b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f87fff640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f90161020 0x7f5f900b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.906 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f87fff640 1 -- 172.21.15.60:0/76231973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c009400 con 0x7f5f900b4970 2023-11-18T20:45:11.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f949b1640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f9006f0f0 0x7f5f90155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:11.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.900+0000 7f5f951b2640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f90161020 0x7f5f900b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:11.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.901+0000 7f5f87fff640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f900b4970 0x7f5f900b1c70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000c430 tx=0x7f5f8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.901+0000 7f5f857fa640 1 -- 172.21.15.60:0/76231973 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f800074e0 con 0x7f5f900b4970 2023-11-18T20:45:11.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.901+0000 7f5f857fa640 1 -- 172.21.15.60:0/76231973 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f80007660 con 0x7f5f900b4970 2023-11-18T20:45:11.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.901+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f900b1190 con 0x7f5f900b4970 2023-11-18T20:45:11.907 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.901+0000 7f5f857fa640 1 -- 172.21.15.60:0/76231973 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80004a20 con 0x7f5f900b4970 2023-11-18T20:45:11.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.901+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f900b1750 con 0x7f5f900b4970 2023-11-18T20:45:11.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.902+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5f9015dcf0 con 0x7f5f900b4970 2023-11-18T20:45:11.908 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.902+0000 7f5f857fa640 1 -- 172.21.15.60:0/76231973 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f80004b80 con 0x7f5f900b4970 2023-11-18T20:45:11.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.903+0000 7f5f857fa640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f7c06ae40 0x7f5f7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.903+0000 7f5f857fa640 1 -- 172.21.15.60:0/76231973 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5f80089090 con 0x7f5f900b4970 2023-11-18T20:45:11.909 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.903+0000 7f5f949b1640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f7c06ae40 0x7f5f7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.903+0000 7f5f857fa640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] conn(0x7f5f7c070ca0 0x7f5f7c0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:11.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.903+0000 7f5f857fa640 1 -- 172.21.15.60:0/76231973 --> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f7c073770 con 0x7f5f7c070ca0 2023-11-18T20:45:11.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.903+0000 7f5f857fa640 1 -- 172.21.15.60:0/76231973 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f8008b750 con 0x7f5f900b4970 2023-11-18T20:45:11.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.903+0000 7f5f951b2640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] conn(0x7f5f7c070ca0 0x7f5f7c0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:11.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.903+0000 7f5f949b1640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f7c06ae40 0x7f5f7c06d300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c036000 tx=0x7f5f8c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.903+0000 7f5f951b2640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] conn(0x7f5f7c070ca0 0x7f5f7c0730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:11.910 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.904+0000 7f5f857fa640 1 -- 172.21.15.60:0/76231973 <== osd.3 v2:172.21.15.60:6824/4034119923 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5f7c073770 con 0x7f5f7c070ca0 2023-11-18T20:45:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.917+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 --> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5f900b1970 con 0x7f5f7c070ca0 2023-11-18T20:45:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.917+0000 7f5f857fa640 1 -- 172.21.15.60:0/76231973 <== osd.3 v2:172.21.15.60:6824/4034119923 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5f900b1970 con 0x7f5f7c070ca0 2023-11-18T20:45:11.919 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:45:11.919 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-18T20:45:11.919 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:45:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 >> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] conn(0x7f5f7c070ca0 msgr2=0x7f5f7c0730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f96c15640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.60:6824/4034119923,v1:172.21.15.60:6825/4034119923] conn(0x7f5f7c070ca0 0x7f5f7c0730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f7c06ae40 msgr2=0x7f5f7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f96c15640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f5f7c06ae40 0x7f5f7c06d300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c036000 tx=0x7f5f8c03d040 comp rx=0 tx=0).stop 2023-11-18T20:45:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f900b4970 msgr2=0x7f5f900b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f96c15640 1 --2- 172.21.15.60:0/76231973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f900b4970 0x7f5f900b1c70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000c430 tx=0x7f5f8000c8f0 comp rx=0 tx=0).stop 2023-11-18T20:45:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f951b2640 1 -- 172.21.15.60:0/76231973 reap_dead start 2023-11-18T20:45:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 shutdown_connections 2023-11-18T20:45:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 >> 172.21.15.60:0/76231973 conn(0x7f5f9006a6b0 msgr2=0x7f5f9006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 shutdown_connections 2023-11-18T20:45:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:11.918+0000 7f5f96c15640 1 -- 172.21.15.60:0/76231973 wait complete. 2023-11-18T20:45:11.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-18T20:45:12.036 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.035+0000 7fe3bfa09640 1 Processor -- start 2023-11-18T20:45:12.037 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.035+0000 7fe3bfa09640 1 -- start start 2023-11-18T20:45:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.036+0000 7fe3bfa09640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe3b80d2910 0x7fe3b80d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.036+0000 7fe3bfa09640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3b80d3730 0x7fe3b80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.036+0000 7fe3bfa09640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b80b2b10 con 0x7fe3b815d330 2023-11-18T20:45:12.038 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.036+0000 7fe3bfa09640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b80d3bc0 con 0x7fe3b80d3730 2023-11-18T20:45:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.036+0000 7fe3bfa09640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b80d1590 con 0x7fe3b80d2910 2023-11-18T20:45:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.036+0000 7fe3bdfa6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3b80d3730 0x7fe3b80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.036+0000 7fe3bdfa6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe3b80d3730 0x7fe3b80b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58026/0 (socket says 172.21.15.60:58026) 2023-11-18T20:45:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.036+0000 7fe3bdfa6640 1 -- 172.21.15.60:0/1986862384 learned_addr learned my addr 172.21.15.60:0/1986862384 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.037+0000 7fe3bcfa4640 1 --2- 172.21.15.60:0/1986862384 >> v2:172.21.15.100:3300/0 conn(0x7fe3b80d2910 0x7fe3b80d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.037+0000 7fe3bdfa6640 1 -- 172.21.15.60:0/1986862384 >> v2:172.21.15.100:3300/0 conn(0x7fe3b80d2910 msgr2=0x7fe3b80d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.037+0000 7fe3bdfa6640 1 --2- 172.21.15.60:0/1986862384 >> v2:172.21.15.100:3300/0 conn(0x7fe3b80d2910 0x7fe3b80d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.039 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.037+0000 7fe3bdfa6640 1 -- 172.21.15.60:0/1986862384 >> v1:172.21.15.100:6789/0 conn(0x7fe3b815d330 legacy=0x7fe3b80b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.037+0000 7fe3bdfa6640 1 -- 172.21.15.60:0/1986862384 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80d1870 con 0x7fe3b80d3730 2023-11-18T20:45:12.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.037+0000 7fe3bdfa6640 1 --2- 172.21.15.60:0/1986862384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d3730 0x7fe3b80b6390 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80d1540 tx=0x7fe3a802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d046070a89dbcea9 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3b7fff640 1 -- 172.21.15.60:0/1986862384 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a802ca80 con 0x7fe3b80d3730 2023-11-18T20:45:12.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3b7fff640 1 -- 172.21.15.60:0/1986862384 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a802cbe0 con 0x7fe3b80d3730 2023-11-18T20:45:12.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3b7fff640 1 -- 172.21.15.60:0/1986862384 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a80343f0 con 0x7fe3b80d3730 2023-11-18T20:45:12.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1986862384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d3730 msgr2=0x7fe3b80b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3bfa09640 1 --2- 172.21.15.60:0/1986862384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d3730 0x7fe3b80b6390 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80d1540 tx=0x7fe3a802bee0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1986862384 shutdown_connections 2023-11-18T20:45:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3bfa09640 1 --2- 172.21.15.60:0/1986862384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d3730 0x7fe3b80b6390 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3bfa09640 1 --2- 172.21.15.60:0/1986862384 >> v2:172.21.15.100:3300/0 conn(0x7fe3b80d2910 0x7fe3b80d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1986862384 >> 172.21.15.60:0/1986862384 conn(0x7fe3b8072020 msgr2=0x7fe3b8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1986862384 shutdown_connections 2023-11-18T20:45:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.038+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1986862384 wait complete. 2023-11-18T20:45:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.039+0000 7fe3bfa09640 1 Processor -- start 2023-11-18T20:45:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.039+0000 7fe3bfa09640 1 -- start start 2023-11-18T20:45:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bfa09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d2910 0x7fe3b80c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bfa09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3b80d3730 0x7fe3b81453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bfa09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b815d330 0x7fe3b80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bfa09640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8145930 con 0x7fe3b80d2910 2023-11-18T20:45:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bfa09640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80c8080 con 0x7fe3b815d330 2023-11-18T20:45:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bfa09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80c8360 con 0x7fe3b80d3730 2023-11-18T20:45:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bd7a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d2910 0x7fe3b80c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bd7a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d2910 0x7fe3b80c9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58034/0 (socket says 172.21.15.60:58034) 2023-11-18T20:45:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bd7a5640 1 -- 172.21.15.60:0/1042457279 learned_addr learned my addr 172.21.15.60:0/1042457279 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bcfa4640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3b80d3730 0x7fe3b81453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.040+0000 7fe3bdfa6640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b815d330 0x7fe3b80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3bd7a5640 1 -- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3b80d3730 msgr2=0x7fe3b81453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3bd7a5640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3b80d3730 0x7fe3b81453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3bd7a5640 1 -- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b815d330 msgr2=0x7fe3b80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3bd7a5640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b815d330 0x7fe3b80cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3bd7a5640 1 -- 172.21.15.60:0/1042457279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a8009b30 con 0x7fe3b80d2910 2023-11-18T20:45:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3bcfa4640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3b80d3730 0x7fe3b81453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3bd7a5640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d2910 0x7fe3b80c9120 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c450 tx=0x7fe3b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3b5ffb640 1 -- 172.21.15.60:0/1042457279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00052d0 con 0x7fe3b80d2910 2023-11-18T20:45:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3b5ffb640 1 -- 172.21.15.60:0/1042457279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b0005430 con 0x7fe3b80d2910 2023-11-18T20:45:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1042457279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b80c8640 con 0x7fe3b80d2910 2023-11-18T20:45:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3b5ffb640 1 -- 172.21.15.60:0/1042457279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0012870 con 0x7fe3b80d2910 2023-11-18T20:45:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.041+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1042457279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b80c8c00 con 0x7fe3b80d2910 2023-11-18T20:45:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.042+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1042457279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b8069400 con 0x7fe3b80d2910 2023-11-18T20:45:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.043+0000 7fe3b5ffb640 1 -- 172.21.15.60:0/1042457279 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3b00129d0 con 0x7fe3b80d2910 2023-11-18T20:45:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.043+0000 7fe3b5ffb640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3a406ae70 0x7fe3a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.043+0000 7fe3b5ffb640 1 -- 172.21.15.60:0/1042457279 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe3b0088d80 con 0x7fe3b80d2910 2023-11-18T20:45:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.043+0000 7fe3bcfa4640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3a406ae70 0x7fe3a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.044+0000 7fe3bcfa4640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3a406ae70 0x7fe3a406d330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe3b815df00 tx=0x7fe3ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.047+0000 7fe3b5ffb640 1 -- 172.21.15.60:0/1042457279 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3b00541c0 con 0x7fe3b80d2910 2023-11-18T20:45:12.223 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.222+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1042457279 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3b80d2110 con 0x7fe3a406ae70 2023-11-18T20:45:12.224 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.223+0000 7fe3b5ffb640 1 -- 172.21.15.60:0/1042457279 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe3ac011980 con 0x7fe3a406ae70 2023-11-18T20:45:12.224 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 3 to deep-scrub 2023-11-18T20:45:12.226 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3a406ae70 msgr2=0x7fe3a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3a406ae70 0x7fe3a406d330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe3b815df00 tx=0x7fe3ac008040 comp rx=0 tx=0).stop 2023-11-18T20:45:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1042457279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d2910 msgr2=0x7fe3b80c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d2910 0x7fe3b80c9120 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c450 tx=0x7fe3b000c910 comp rx=0 tx=0).stop 2023-11-18T20:45:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1042457279 shutdown_connections 2023-11-18T20:45:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe3a406ae70 0x7fe3a406d330 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3b815d330 0x7fe3b80cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.227 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3b80d3730 0x7fe3b81453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 --2- 172.21.15.60:0/1042457279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3b80d2910 0x7fe3b80c9120 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1042457279 >> 172.21.15.60:0/1042457279 conn(0x7fe3b8072020 msgr2=0x7fe3b80b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:12.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1042457279 shutdown_connections 2023-11-18T20:45:12.228 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.225+0000 7fe3bfa09640 1 -- 172.21.15.60:0/1042457279 wait complete. 2023-11-18T20:45:12.237 INFO:tasks.ceph:Scrubbing osd.4 2023-11-18T20:45:12.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-18T20:45:12.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.346+0000 7f92893ce640 1 Processor -- start 2023-11-18T20:45:12.348 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.346+0000 7f92893ce640 1 -- start start 2023-11-18T20:45:12.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.347+0000 7f92893ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9284143300 0x7f928415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.347+0000 7f92893ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b1180 0x7f9284071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.347+0000 7f92893ce640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92840b3ca0 con 0x7f92840b0cf0 2023-11-18T20:45:12.351 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.347+0000 7f92893ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928415e9f0 con 0x7f9284143300 2023-11-18T20:45:12.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.347+0000 7f92893ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92840b1610 con 0x7f92840b1180 2023-11-18T20:45:12.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.347+0000 7f92827fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f92840b0cf0 0x7f92840714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57696/0 (socket says 172.21.15.60:57696) 2023-11-18T20:45:12.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.347+0000 7f92827fc640 1 -- 172.21.15.60:0/1102341143 learned_addr learned my addr 172.21.15.60:0/1102341143 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:12.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9282ffd640 1 --2- 172.21.15.60:0/1102341143 >> v2:172.21.15.100:3300/0 conn(0x7f9284143300 0x7f928415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448089417 0 0) 0x7f92840b3ca0 con 0x7f92840b0cf0 2023-11-18T20:45:12.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9270005730 con 0x7f92840b0cf0 2023-11-18T20:45:12.352 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f92837fe640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b1180 0x7f9284071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2827944614 0 0) 0x7f9270005730 con 0x7f92840b0cf0 2023-11-18T20:45:12.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b1180 msgr2=0x7f9284071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b1180 0x7f9284071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 >> v2:172.21.15.100:3300/0 conn(0x7f9284143300 msgr2=0x7f928415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 --2- 172.21.15.60:0/1102341143 >> v2:172.21.15.100:3300/0 conn(0x7f9284143300 0x7f928415e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.353 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840cdb70 con 0x7f92840b0cf0 2023-11-18T20:45:12.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f926c001670 con 0x7f92840b0cf0 2023-11-18T20:45:12.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 >> v1:172.21.15.60:6789/0 conn(0x7f92840b0cf0 legacy=0x7f92840714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9270005b80 0x7f9270005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b1180 0x7f92700069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9270006f10 0x7f9270007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928415e9f0 con 0x7f9270006f10 2023-11-18T20:45:12.354 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92840b1610 con 0x7f9270005b80 2023-11-18T20:45:12.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927000db90 con 0x7f92840b1180 2023-11-18T20:45:12.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f92837fe640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9270006f10 0x7f9270007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f9282ffd640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9270005b80 0x7f9270005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f92827fc640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b1180 0x7f92700069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f92837fe640 1 -- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b1180 msgr2=0x7f92700069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f92837fe640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b1180 0x7f92700069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.355 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f92837fe640 1 -- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9270005b80 msgr2=0x7f9270005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f92837fe640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9270005b80 0x7f9270005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.348+0000 7f92837fe640 1 -- 172.21.15.60:0/1102341143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270005730 con 0x7f9270006f10 2023-11-18T20:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.349+0000 7f92837fe640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9270006f10 0x7f9270007390 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9274009fc0 tx=0x7f9270007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b29951f21559db7 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.349+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927402cc20 con 0x7f9270006f10 2023-11-18T20:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.349+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927400f040 con 0x7f9270006f10 2023-11-18T20:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.349+0000 7f9281ffb640 1 -- 172.21.15.60:0/1102341143 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274034440 con 0x7f9270006f10 2023-11-18T20:45:12.356 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.350+0000 7f92893ce640 1 -- 172.21.15.60:0/1102341143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9270006f10 msgr2=0x7f9270007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.350+0000 7f92893ce640 1 --2- 172.21.15.60:0/1102341143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9270006f10 0x7f9270007390 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9274009fc0 tx=0x7f9270007a00 comp rx=0 tx=0).stop 2023-11-18T20:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.350+0000 7f92837fe640 1 -- 172.21.15.60:0/1102341143 reap_dead start 2023-11-18T20:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.350+0000 7f92893ce640 1 -- 172.21.15.60:0/1102341143 shutdown_connections 2023-11-18T20:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.350+0000 7f92893ce640 1 -- 172.21.15.60:0/1102341143 >> 172.21.15.60:0/1102341143 conn(0x7f928406a4d0 msgr2=0x7f928406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.350+0000 7f92893ce640 1 -- 172.21.15.60:0/1102341143 shutdown_connections 2023-11-18T20:45:12.357 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.350+0000 7f92893ce640 1 -- 172.21.15.60:0/1102341143 wait complete. 2023-11-18T20:45:12.360 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.351+0000 7f92893ce640 1 Processor -- start 2023-11-18T20:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.351+0000 7f92893ce640 1 -- start start 2023-11-18T20:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f92893ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284136130 0x7f928413fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f92893ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92841f2f10 0x7f9284139960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f92893ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9284139ea0 0x7f92840707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f92893ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9284136510 con 0x7f9284136130 2023-11-18T20:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f92893ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92841f3470 con 0x7f92841f2f10 2023-11-18T20:45:12.361 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f92893ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92841f3750 con 0x7f9284139ea0 2023-11-18T20:45:12.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f9282ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284136130 0x7f928413fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f9282ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284136130 0x7f928413fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58054/0 (socket says 172.21.15.60:58054) 2023-11-18T20:45:12.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f9282ffd640 1 -- 172.21.15.60:0/4277195297 learned_addr learned my addr 172.21.15.60:0/4277195297 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:12.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f92827fc640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92841f2f10 0x7f9284139960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f92837fe640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9284139ea0 0x7f92840707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f9282ffd640 1 -- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9284139ea0 msgr2=0x7f92840707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f9282ffd640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9284139ea0 0x7f92840707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.362 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f9282ffd640 1 -- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92841f2f10 msgr2=0x7f9284139960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f9282ffd640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92841f2f10 0x7f9284139960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f9282ffd640 1 -- 172.21.15.60:0/4277195297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92841f3a30 con 0x7f9284136130 2023-11-18T20:45:12.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.352+0000 7f92827fc640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92841f2f10 0x7f9284139960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:45:12.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.353+0000 7f9282ffd640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284136130 0x7f928413fe90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f927c011960 tx=0x7f927c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.363 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.353+0000 7f927b7fe640 1 -- 172.21.15.60:0/4277195297 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00faf0 con 0x7f9284136130 2023-11-18T20:45:12.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.353+0000 7f927b7fe640 1 -- 172.21.15.60:0/4277195297 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c00fc50 con 0x7f9284136130 2023-11-18T20:45:12.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.353+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92841f3d10 con 0x7f9284136130 2023-11-18T20:45:12.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.353+0000 7f927b7fe640 1 -- 172.21.15.60:0/4277195297 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c014440 con 0x7f9284136130 2023-11-18T20:45:12.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.353+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9284070ee0 con 0x7f9284136130 2023-11-18T20:45:12.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.354+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f92841f38c0 con 0x7f9284136130 2023-11-18T20:45:12.364 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.354+0000 7f927b7fe640 1 -- 172.21.15.60:0/4277195297 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f927c01d400 con 0x7f9284136130 2023-11-18T20:45:12.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.355+0000 7f927b7fe640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f926406ae40 0x7f926406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.355+0000 7f927b7fe640 1 -- 172.21.15.60:0/4277195297 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f927c08a450 con 0x7f9284136130 2023-11-18T20:45:12.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.355+0000 7f927b7fe640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] conn(0x7f9264070ca0 0x7f92640730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.355+0000 7f927b7fe640 1 -- 172.21.15.60:0/4277195297 --> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9264073770 con 0x7f9264070ca0 2023-11-18T20:45:12.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.355+0000 7f927b7fe640 1 -- 172.21.15.60:0/4277195297 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f927c08a8a0 con 0x7f9284136130 2023-11-18T20:45:12.365 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.355+0000 7f92827fc640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f926406ae40 0x7f926406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.355+0000 7f92837fe640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] conn(0x7f9264070ca0 0x7f92640730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.356+0000 7f92837fe640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] conn(0x7f9264070ca0 0x7f92640730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.356+0000 7f92827fc640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f926406ae40 0x7f926406d300 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f926c0017a0 tx=0x7f926c001c50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.366 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.356+0000 7f927b7fe640 1 -- 172.21.15.60:0/4277195297 <== osd.4 v2:172.21.15.100:6800/454905857 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9264073770 con 0x7f9264070ca0 2023-11-18T20:45:12.370 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.369+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 --> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f92840ce410 con 0x7f9264070ca0 2023-11-18T20:45:12.371 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f927b7fe640 1 -- 172.21.15.60:0/4277195297 <== osd.4 v2:172.21.15.100:6800/454905857 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f927402cb40 con 0x7f9264070ca0 2023-11-18T20:45:12.371 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:45:12.372 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-18T20:45:12.372 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:45:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] conn(0x7f9264070ca0 msgr2=0x7f92640730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92893ce640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:6800/454905857,v1:172.21.15.100:6801/454905857] conn(0x7f9264070ca0 0x7f92640730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f926406ae40 msgr2=0x7f926406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92893ce640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f926406ae40 0x7f926406d300 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f926c0017a0 tx=0x7f926c001c50 comp rx=0 tx=0).stop 2023-11-18T20:45:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284136130 msgr2=0x7f928413fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92893ce640 1 --2- 172.21.15.60:0/4277195297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284136130 0x7f928413fe90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f927c011960 tx=0x7f927c011e20 comp rx=0 tx=0).stop 2023-11-18T20:45:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92837fe640 1 -- 172.21.15.60:0/4277195297 reap_dead start 2023-11-18T20:45:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 shutdown_connections 2023-11-18T20:45:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 >> 172.21.15.60:0/4277195297 conn(0x7f928406a4d0 msgr2=0x7f928413cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 shutdown_connections 2023-11-18T20:45:12.374 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.370+0000 7f92893ce640 1 -- 172.21.15.60:0/4277195297 wait complete. 2023-11-18T20:45:12.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-18T20:45:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.488+0000 7f1dcf390640 1 Processor -- start 2023-11-18T20:45:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.488+0000 7f1dcf390640 1 -- start start 2023-11-18T20:45:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcf390640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1dc8150560 0x7f1dc80b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcf390640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1dc80b3390 0x7f1dc815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcf390640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc8150940 0x7f1dc815d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcf390640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc80b49f0 con 0x7f1dc8150560 2023-11-18T20:45:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcf390640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc80b3de0 con 0x7f1dc80b3390 2023-11-18T20:45:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcf390640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b40c0 con 0x7f1dc8150940 2023-11-18T20:45:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcd12c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1dc8150560 0x7f1dc80b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcd12c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1dc8150560 0x7f1dc80b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58062/0 (socket says 172.21.15.60:58062) 2023-11-18T20:45:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcd12c640 1 -- 172.21.15.60:0/850155911 learned_addr learned my addr 172.21.15.60:0/850155911 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcd92d640 1 --2- 172.21.15.60:0/850155911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc8150940 0x7f1dc815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcc92b640 1 --2- 172.21.15.60:0/850155911 >> v2:172.21.15.100:3300/0 conn(0x7f1dc80b3390 0x7f1dc815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcd12c640 1 -- 172.21.15.60:0/850155911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc8150940 msgr2=0x7f1dc815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcd12c640 1 --2- 172.21.15.60:0/850155911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc8150940 0x7f1dc815d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcd12c640 1 -- 172.21.15.60:0/850155911 >> v2:172.21.15.100:3300/0 conn(0x7f1dc80b3390 msgr2=0x7f1dc815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcd12c640 1 --2- 172.21.15.60:0/850155911 >> v2:172.21.15.100:3300/0 conn(0x7f1dc80b3390 0x7f1dc815d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.489+0000 7f1dcd12c640 1 -- 172.21.15.60:0/850155911 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc80b43a0 con 0x7f1dc8150560 2023-11-18T20:45:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.490+0000 7f1dcd12c640 1 --2- 172.21.15.60:0/850155911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc8150560 0x7f1dc80b2e50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8072810 tx=0x7f1dc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f06b887e89acf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.490+0000 7f1dbffff640 1 -- 172.21.15.60:0/850155911 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0031870 con 0x7f1dc8150560 2023-11-18T20:45:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.490+0000 7f1dbffff640 1 -- 172.21.15.60:0/850155911 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc00319d0 con 0x7f1dc8150560 2023-11-18T20:45:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.490+0000 7f1dbffff640 1 -- 172.21.15.60:0/850155911 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0031d60 con 0x7f1dc8150560 2023-11-18T20:45:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.491+0000 7f1dcf390640 1 -- 172.21.15.60:0/850155911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc8150560 msgr2=0x7f1dc80b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.491+0000 7f1dcf390640 1 --2- 172.21.15.60:0/850155911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc8150560 0x7f1dc80b2e50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8072810 tx=0x7f1dc002b910 comp rx=0 tx=0).stop 2023-11-18T20:45:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.491+0000 7f1dcf390640 1 -- 172.21.15.60:0/850155911 shutdown_connections 2023-11-18T20:45:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.491+0000 7f1dcf390640 1 --2- 172.21.15.60:0/850155911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc8150940 0x7f1dc815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.491+0000 7f1dcf390640 1 --2- 172.21.15.60:0/850155911 >> v2:172.21.15.100:3300/0 conn(0x7f1dc80b3390 0x7f1dc815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.491+0000 7f1dcf390640 1 --2- 172.21.15.60:0/850155911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc8150560 0x7f1dc80b2e50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.491+0000 7f1dcf390640 1 -- 172.21.15.60:0/850155911 >> 172.21.15.60:0/850155911 conn(0x7f1dc806a4d0 msgr2=0x7f1dc806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:12.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.491+0000 7f1dcf390640 1 -- 172.21.15.60:0/850155911 shutdown_connections 2023-11-18T20:45:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.491+0000 7f1dcf390640 1 -- 172.21.15.60:0/850155911 wait complete. 2023-11-18T20:45:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.492+0000 7f1dcf390640 1 Processor -- start 2023-11-18T20:45:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.492+0000 7f1dcf390640 1 -- start start 2023-11-18T20:45:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcf390640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc80b3390 0x7f1dc8159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcf390640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc8150560 0x7f1dc8159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcf390640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc80cb270 0x7f1dc80cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcf390640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc815c050 con 0x7f1dc80b3390 2023-11-18T20:45:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcf390640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc815c330 con 0x7f1dc8150560 2023-11-18T20:45:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcf390640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc815c610 con 0x7f1dc80cb270 2023-11-18T20:45:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcd12c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc80b3390 0x7f1dc8159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcd12c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc80b3390 0x7f1dc8159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58066/0 (socket says 172.21.15.60:58066) 2023-11-18T20:45:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcd12c640 1 -- 172.21.15.60:0/1150957508 learned_addr learned my addr 172.21.15.60:0/1150957508 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcc92b640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc8150560 0x7f1dc8159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcd92d640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc80cb270 0x7f1dc80cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcd12c640 1 -- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc80cb270 msgr2=0x7f1dc80cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcd12c640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc80cb270 0x7f1dc80cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcd12c640 1 -- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc8150560 msgr2=0x7f1dc8159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcd12c640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc8150560 0x7f1dc8159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcd12c640 1 -- 172.21.15.60:0/1150957508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00094d0 con 0x7f1dc80b3390 2023-11-18T20:45:12.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.493+0000 7f1dcd92d640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc80cb270 0x7f1dc80cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:12.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.494+0000 7f1dcd12c640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc80b3390 0x7f1dc8159a20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1dc002b950 tx=0x7f1dc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.499 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.494+0000 7f1dbdffb640 1 -- 172.21.15.60:0/1150957508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0031860 con 0x7f1dc80b3390 2023-11-18T20:45:12.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.494+0000 7f1dbdffb640 1 -- 172.21.15.60:0/1150957508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc00443f0 con 0x7f1dc80b3390 2023-11-18T20:45:12.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.494+0000 7f1dcf390640 1 -- 172.21.15.60:0/1150957508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc8159220 con 0x7f1dc80b3390 2023-11-18T20:45:12.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.494+0000 7f1dbdffb640 1 -- 172.21.15.60:0/1150957508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00447a0 con 0x7f1dc80b3390 2023-11-18T20:45:12.500 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.494+0000 7f1dcf390640 1 -- 172.21.15.60:0/1150957508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc81e5900 con 0x7f1dc80b3390 2023-11-18T20:45:12.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.495+0000 7f1dcf390640 1 -- 172.21.15.60:0/1150957508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc8159740 con 0x7f1dc80b3390 2023-11-18T20:45:12.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.495+0000 7f1dbdffb640 1 -- 172.21.15.60:0/1150957508 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1dc004d450 con 0x7f1dc80b3390 2023-11-18T20:45:12.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.496+0000 7f1dbdffb640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1dac06ae40 0x7f1dac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.501 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.496+0000 7f1dbdffb640 1 -- 172.21.15.60:0/1150957508 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1dc002d040 con 0x7f1dc80b3390 2023-11-18T20:45:12.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.496+0000 7f1dcc92b640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1dac06ae40 0x7f1dac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.496+0000 7f1dcc92b640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1dac06ae40 0x7f1dac06d300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1db800aac0 tx=0x7f1db8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.502 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.500+0000 7f1dbdffb640 1 -- 172.21.15.60:0/1150957508 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1dc0076e10 con 0x7f1dc80b3390 2023-11-18T20:45:12.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.676+0000 7f1dcf390640 1 -- 172.21.15.60:0/1150957508 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dc80b4e80 con 0x7f1dac06ae40 2023-11-18T20:45:12.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.676+0000 7f1dbdffb640 1 -- 172.21.15.60:0/1150957508 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1db800c040 con 0x7f1dac06ae40 2023-11-18T20:45:12.678 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 4 to deep-scrub 2023-11-18T20:45:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.678+0000 7f1dcf390640 1 -- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1dac06ae40 msgr2=0x7f1dac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1dac06ae40 0x7f1dac06d300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1db800aac0 tx=0x7f1db8008040 comp rx=0 tx=0).stop 2023-11-18T20:45:12.680 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 -- 172.21.15.60:0/1150957508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc80b3390 msgr2=0x7f1dc8159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc80b3390 0x7f1dc8159a20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1dc002b950 tx=0x7f1dc00046c0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 -- 172.21.15.60:0/1150957508 shutdown_connections 2023-11-18T20:45:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f1dac06ae40 0x7f1dac06d300 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1dc80cb270 0x7f1dc80cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc8150560 0x7f1dc8159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 --2- 172.21.15.60:0/1150957508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dc80b3390 0x7f1dc8159a20 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 -- 172.21.15.60:0/1150957508 >> 172.21.15.60:0/1150957508 conn(0x7f1dc806a4d0 msgr2=0x7f1dc8071160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 -- 172.21.15.60:0/1150957508 shutdown_connections 2023-11-18T20:45:12.681 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.679+0000 7f1dcf390640 1 -- 172.21.15.60:0/1150957508 wait complete. 2023-11-18T20:45:12.690 INFO:tasks.ceph:Scrubbing osd.5 2023-11-18T20:45:12.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-18T20:45:12.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.801+0000 7f24150e0640 1 Processor -- start 2023-11-18T20:45:12.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.801+0000 7f24150e0640 1 -- start start 2023-11-18T20:45:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f24150e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24100b4a80 0x7f24100ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f24150e0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f241015b5d0 0x7f241015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f24150e0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24100ced30 con 0x7f2410144960 2023-11-18T20:45:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f24150e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24100cd510 con 0x7f24100b4a80 2023-11-18T20:45:12.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f24150e0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241015bf60 con 0x7f241015b5d0 2023-11-18T20:45:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f240e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24100b4a80 0x7f24100ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f240e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24100b4a80 0x7f24100ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58082/0 (socket says 172.21.15.60:58082) 2023-11-18T20:45:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f240e59c640 1 -- 172.21.15.60:0/475359567 learned_addr learned my addr 172.21.15.60:0/475359567 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f240f59e640 1 --2- 172.21.15.60:0/475359567 >> v2:172.21.15.100:3300/0 conn(0x7f241015b5d0 0x7f241015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f240e59c640 1 -- 172.21.15.60:0/475359567 >> v2:172.21.15.100:3300/0 conn(0x7f241015b5d0 msgr2=0x7f241015ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f240e59c640 1 --2- 172.21.15.60:0/475359567 >> v2:172.21.15.100:3300/0 conn(0x7f241015b5d0 0x7f241015ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f240e59c640 1 -- 172.21.15.60:0/475359567 >> v1:172.21.15.100:6789/0 conn(0x7f2410144960 legacy=0x7f24100b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.802+0000 7f240e59c640 1 -- 172.21.15.60:0/475359567 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241015c220 con 0x7f24100b4a80 2023-11-18T20:45:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.803+0000 7f240e59c640 1 --2- 172.21.15.60:0/475359567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24100b4a80 0x7f24100ccfd0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f24100cef80 tx=0x7f240402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90520c3db3fd61db server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.803+0000 7f240dd9b640 1 -- 172.21.15.60:0/475359567 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404030860 con 0x7f24100b4a80 2023-11-18T20:45:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.803+0000 7f240dd9b640 1 -- 172.21.15.60:0/475359567 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24040309c0 con 0x7f24100b4a80 2023-11-18T20:45:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.803+0000 7f240dd9b640 1 -- 172.21.15.60:0/475359567 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404030d50 con 0x7f24100b4a80 2023-11-18T20:45:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.804+0000 7f24150e0640 1 -- 172.21.15.60:0/475359567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24100b4a80 msgr2=0x7f24100ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.804+0000 7f24150e0640 1 --2- 172.21.15.60:0/475359567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24100b4a80 0x7f24100ccfd0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f24100cef80 tx=0x7f240402b9a0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.804+0000 7f24150e0640 1 -- 172.21.15.60:0/475359567 shutdown_connections 2023-11-18T20:45:12.809 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.804+0000 7f24150e0640 1 --2- 172.21.15.60:0/475359567 >> v2:172.21.15.100:3300/0 conn(0x7f241015b5d0 0x7f241015ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.804+0000 7f24150e0640 1 --2- 172.21.15.60:0/475359567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24100b4a80 0x7f24100ccfd0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.804+0000 7f24150e0640 1 -- 172.21.15.60:0/475359567 >> 172.21.15.60:0/475359567 conn(0x7f241006a4d0 msgr2=0x7f241006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.804+0000 7f24150e0640 1 -- 172.21.15.60:0/475359567 shutdown_connections 2023-11-18T20:45:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.804+0000 7f24150e0640 1 -- 172.21.15.60:0/475359567 wait complete. 2023-11-18T20:45:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.805+0000 7f24150e0640 1 Processor -- start 2023-11-18T20:45:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.805+0000 7f24150e0640 1 -- start start 2023-11-18T20:45:12.810 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f24150e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24100b4a80 0x7f24100cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f24150e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2410144960 0x7f24100b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f24150e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f241015b5d0 0x7f24100b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f24150e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2410158d00 con 0x7f24100b4a80 2023-11-18T20:45:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f24150e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2410157280 con 0x7f241015b5d0 2023-11-18T20:45:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f24150e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24100b3c50 con 0x7f2410144960 2023-11-18T20:45:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f240ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24100b4a80 0x7f24100cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f240ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24100b4a80 0x7f24100cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58088/0 (socket says 172.21.15.60:58088) 2023-11-18T20:45:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f240ed9d640 1 -- 172.21.15.60:0/3769408735 learned_addr learned my addr 172.21.15.60:0/3769408735 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f240e59c640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2410144960 0x7f24100b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.806+0000 7f240f59e640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f241015b5d0 0x7f24100b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f240ed9d640 1 -- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2410144960 msgr2=0x7f24100b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f240ed9d640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2410144960 0x7f24100b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f240ed9d640 1 -- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f241015b5d0 msgr2=0x7f24100b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f240ed9d640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f241015b5d0 0x7f24100b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f240ed9d640 1 -- 172.21.15.60:0/3769408735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24040094d0 con 0x7f24100b4a80 2023-11-18T20:45:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f240ed9d640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24100b4a80 0x7f24100cac70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f240800c450 tx=0x7f240800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f23f77fe640 1 -- 172.21.15.60:0/3769408735 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24080052d0 con 0x7f24100b4a80 2023-11-18T20:45:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f23f77fe640 1 -- 172.21.15.60:0/3769408735 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2408005430 con 0x7f24100b4a80 2023-11-18T20:45:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24101f2da0 con 0x7f24100b4a80 2023-11-18T20:45:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f23f77fe640 1 -- 172.21.15.60:0/3769408735 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408012870 con 0x7f24100b4a80 2023-11-18T20:45:12.813 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.807+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2410157770 con 0x7f24100b4a80 2023-11-18T20:45:12.814 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.808+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24100707c0 con 0x7f24100b4a80 2023-11-18T20:45:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.808+0000 7f23f77fe640 1 -- 172.21.15.60:0/3769408735 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24080129d0 con 0x7f24100b4a80 2023-11-18T20:45:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.809+0000 7f23f77fe640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f23fc06ae40 0x7f23fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.809+0000 7f23f77fe640 1 -- 172.21.15.60:0/3769408735 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2408088d00 con 0x7f24100b4a80 2023-11-18T20:45:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.809+0000 7f23f77fe640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] conn(0x7f23fc070ca0 0x7f23fc0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.809+0000 7f23f77fe640 1 -- 172.21.15.60:0/3769408735 --> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23fc073770 con 0x7f23fc070ca0 2023-11-18T20:45:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.809+0000 7f23f77fe640 1 -- 172.21.15.60:0/3769408735 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f240808b3c0 con 0x7f24100b4a80 2023-11-18T20:45:12.815 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.809+0000 7f240e59c640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f23fc06ae40 0x7f23fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.809+0000 7f240f59e640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] conn(0x7f23fc070ca0 0x7f23fc0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.810+0000 7f240e59c640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f23fc06ae40 0x7f23fc06d300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2404036000 tx=0x7f2404008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.810+0000 7f240f59e640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] conn(0x7f23fc070ca0 0x7f23fc0730c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.816 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.811+0000 7f23f77fe640 1 -- 172.21.15.60:0/3769408735 <== osd.5 v2:172.21.15.100:6816/4243789067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f23fc073770 con 0x7f23fc070ca0 2023-11-18T20:45:12.824 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.824+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 --> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f24100cc7d0 con 0x7f23fc070ca0 2023-11-18T20:45:12.825 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.824+0000 7f23f77fe640 1 -- 172.21.15.60:0/3769408735 <== osd.5 v2:172.21.15.100:6816/4243789067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f24000083a0 con 0x7f23fc070ca0 2023-11-18T20:45:12.826 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:45:12.826 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-18T20:45:12.826 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:45:12.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] conn(0x7f23fc070ca0 msgr2=0x7f23fc0730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f24150e0640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:6816/4243789067,v1:172.21.15.100:6817/4243789067] conn(0x7f23fc070ca0 0x7f23fc0730c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f23fc06ae40 msgr2=0x7f23fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f24150e0640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f23fc06ae40 0x7f23fc06d300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2404036000 tx=0x7f2404008030 comp rx=0 tx=0).stop 2023-11-18T20:45:12.827 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24100b4a80 msgr2=0x7f24100cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f24150e0640 1 --2- 172.21.15.60:0/3769408735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24100b4a80 0x7f24100cac70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f240800c450 tx=0x7f240800c910 comp rx=0 tx=0).stop 2023-11-18T20:45:12.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f240f59e640 1 -- 172.21.15.60:0/3769408735 reap_dead start 2023-11-18T20:45:12.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 shutdown_connections 2023-11-18T20:45:12.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 >> 172.21.15.60:0/3769408735 conn(0x7f241006a4d0 msgr2=0x7f241006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:12.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 shutdown_connections 2023-11-18T20:45:12.828 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.825+0000 7f24150e0640 1 -- 172.21.15.60:0/3769408735 wait complete. 2023-11-18T20:45:12.836 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-18T20:45:12.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.942+0000 7fc839464640 1 Processor -- start 2023-11-18T20:45:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.942+0000 7fc839464640 1 -- start start 2023-11-18T20:45:12.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.943+0000 7fc839464640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc83415d330 0x7fc8340b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.943+0000 7fc839464640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8340b2b10 con 0x7fc8340d2910 2023-11-18T20:45:12.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.943+0000 7fc839464640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8340d1590 con 0x7fc8340d2d90 2023-11-18T20:45:12.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.943+0000 7fc839464640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8340d1870 con 0x7fc83415d330 2023-11-18T20:45:12.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.943+0000 7fc8327fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc8340d2910 0x7fc8340d31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57700/0 (socket says 172.21.15.60:57700) 2023-11-18T20:45:12.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.943+0000 7fc8327fc640 1 -- 172.21.15.60:0/1844617814 learned_addr learned my addr 172.21.15.60:0/1844617814 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:12.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.943+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892796122 0 0) 0x7fc8340b2b10 con 0x7fc8340d2910 2023-11-18T20:45:12.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc832ffd640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc83415d330 0x7fc8340b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc820003520 con 0x7fc8340d2910 2023-11-18T20:45:12.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147641465 0 0) 0x7fc8340d1590 con 0x7fc8340d2d90 2023-11-18T20:45:12.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8340b2b10 con 0x7fc8340d2d90 2023-11-18T20:45:12.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1680931109 0 0) 0x7fc820003520 con 0x7fc8340d2910 2023-11-18T20:45:12.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc83415d330 msgr2=0x7fc8340b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc83415d330 0x7fc8340b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 >> v1:172.21.15.100:6789/0 conn(0x7fc8340d2d90 legacy=0x7fc8340b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8340d1b50 con 0x7fc8340d2910 2023-11-18T20:45:12.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7fc81c001670 con 0x7fc8340d2910 2023-11-18T20:45:12.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 >> v1:172.21.15.60:6789/0 conn(0x7fc8340d2910 legacy=0x7fc8340d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc820003b30 0x7fc820003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8200047c0 0x7fc820004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc83415d330 0x7fc8200051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8340d1870 con 0x7fc820003b30 2023-11-18T20:45:12.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82000bb10 con 0x7fc8200047c0 2023-11-18T20:45:12.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82000bca0 con 0x7fc83415d330 2023-11-18T20:45:12.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc832ffd640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc820003b30 0x7fc820003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc8337fe640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc83415d330 0x7fc8200051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc8327fc640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8200047c0 0x7fc820004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc832ffd640 1 -- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc83415d330 msgr2=0x7fc8200051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.950 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc832ffd640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc83415d330 0x7fc8200051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc832ffd640 1 -- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8200047c0 msgr2=0x7fc820004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:12.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc832ffd640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8200047c0 0x7fc820004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc832ffd640 1 -- 172.21.15.60:0/1844617814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820003520 con 0x7fc820003b30 2023-11-18T20:45:12.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.944+0000 7fc8337fe640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc83415d330 0x7fc8200051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:12.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.945+0000 7fc832ffd640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc820003b30 0x7fc820003f30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009af0 tx=0x7fc8200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af8b0974e415215 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.945+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c02c6b0 con 0x7fc820003b30 2023-11-18T20:45:12.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.945+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c02c810 con 0x7fc820003b30 2023-11-18T20:45:12.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.945+0000 7fc831ffb640 1 -- 172.21.15.60:0/1844617814 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c02cba0 con 0x7fc820003b30 2023-11-18T20:45:12.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.946+0000 7fc839464640 1 -- 172.21.15.60:0/1844617814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc820003b30 msgr2=0x7fc820003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.946+0000 7fc839464640 1 --2- 172.21.15.60:0/1844617814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc820003b30 0x7fc820003f30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009af0 tx=0x7fc8200057f0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.946+0000 7fc8337fe640 1 -- 172.21.15.60:0/1844617814 reap_dead start 2023-11-18T20:45:12.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.946+0000 7fc839464640 1 -- 172.21.15.60:0/1844617814 shutdown_connections 2023-11-18T20:45:12.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.946+0000 7fc839464640 1 -- 172.21.15.60:0/1844617814 >> 172.21.15.60:0/1844617814 conn(0x7fc834072020 msgr2=0x7fc834072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:12.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.946+0000 7fc839464640 1 -- 172.21.15.60:0/1844617814 shutdown_connections 2023-11-18T20:45:12.952 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.946+0000 7fc839464640 1 -- 172.21.15.60:0/1844617814 wait complete. 2023-11-18T20:45:12.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.947+0000 7fc839464640 1 Processor -- start 2023-11-18T20:45:12.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.947+0000 7fc839464640 1 -- start start 2023-11-18T20:45:12.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc839464640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc820003b30 0x7fc8340c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc839464640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc834156c10 0x7fc8340c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc839464640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc83415e340 0x7fc83415e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc839464640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8340b7450 con 0x7fc834156c10 2023-11-18T20:45:12.955 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc839464640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83415ed10 con 0x7fc83415e340 2023-11-18T20:45:12.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc839464640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8340b7140 con 0x7fc820003b30 2023-11-18T20:45:12.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc8327fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc834156c10 0x7fc8340c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc8327fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc834156c10 0x7fc8340c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58110/0 (socket says 172.21.15.60:58110) 2023-11-18T20:45:12.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc8327fc640 1 -- 172.21.15.60:0/349947998 learned_addr learned my addr 172.21.15.60:0/349947998 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:12.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc8327fc640 1 -- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc820003b30 msgr2=0x7fc8340c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc8327fc640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc820003b30 0x7fc8340c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc8327fc640 1 -- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc83415e340 msgr2=0x7fc83415e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:12.956 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc8327fc640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc83415e340 0x7fc83415e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc8327fc640 1 -- 172.21.15.60:0/349947998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8341f2b30 con 0x7fc834156c10 2023-11-18T20:45:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc8327fc640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc834156c10 0x7fc8340c7af0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00e700 tx=0x7fc81c00eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc82b7fe640 1 -- 172.21.15.60:0/349947998 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00c5a0 con 0x7fc834156c10 2023-11-18T20:45:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.948+0000 7fc82b7fe640 1 -- 172.21.15.60:0/349947998 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c00c700 con 0x7fc834156c10 2023-11-18T20:45:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.949+0000 7fc839464640 1 -- 172.21.15.60:0/349947998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8340c92a0 con 0x7fc834156c10 2023-11-18T20:45:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.949+0000 7fc82b7fe640 1 -- 172.21.15.60:0/349947998 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00cab0 con 0x7fc834156c10 2023-11-18T20:45:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.949+0000 7fc839464640 1 -- 172.21.15.60:0/349947998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8340c9860 con 0x7fc834156c10 2023-11-18T20:45:12.957 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.949+0000 7fc839464640 1 -- 172.21.15.60:0/349947998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8340c9a20 con 0x7fc834156c10 2023-11-18T20:45:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.950+0000 7fc82b7fe640 1 -- 172.21.15.60:0/349947998 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc81c00cc10 con 0x7fc834156c10 2023-11-18T20:45:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.950+0000 7fc82b7fe640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc81406ae40 0x7fc81406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.950+0000 7fc82b7fe640 1 -- 172.21.15.60:0/349947998 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc81c089110 con 0x7fc834156c10 2023-11-18T20:45:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.951+0000 7fc832ffd640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc81406ae40 0x7fc81406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.951+0000 7fc832ffd640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc81406ae40 0x7fc81406d300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc82c036000 tx=0x7fc82c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:12.958 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:12.955+0000 7fc82b7fe640 1 -- 172.21.15.60:0/349947998 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc81c054550 con 0x7fc834156c10 2023-11-18T20:45:13.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.130+0000 7fc839464640 1 -- 172.21.15.60:0/349947998 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8340d23f0 con 0x7fc81406ae40 2023-11-18T20:45:13.132 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.131+0000 7fc82b7fe640 1 -- 172.21.15.60:0/349947998 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc82c004040 con 0x7fc81406ae40 2023-11-18T20:45:13.132 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 5 to deep-scrub 2023-11-18T20:45:13.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 -- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc81406ae40 msgr2=0x7fc81406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc81406ae40 0x7fc81406d300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc82c036000 tx=0x7fc82c02e000 comp rx=0 tx=0).stop 2023-11-18T20:45:13.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 -- 172.21.15.60:0/349947998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc834156c10 msgr2=0x7fc8340c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc834156c10 0x7fc8340c7af0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00e700 tx=0x7fc81c00eb00 comp rx=0 tx=0).stop 2023-11-18T20:45:13.135 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 -- 172.21.15.60:0/349947998 shutdown_connections 2023-11-18T20:45:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fc81406ae40 0x7fc81406d300 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc83415e340 0x7fc83415e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc834156c10 0x7fc8340c7af0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 --2- 172.21.15.60:0/349947998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc820003b30 0x7fc8340c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 -- 172.21.15.60:0/349947998 >> 172.21.15.60:0/349947998 conn(0x7fc834072020 msgr2=0x7fc8340b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 -- 172.21.15.60:0/349947998 shutdown_connections 2023-11-18T20:45:13.136 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.133+0000 7fc839464640 1 -- 172.21.15.60:0/349947998 wait complete. 2023-11-18T20:45:13.145 INFO:tasks.ceph:Scrubbing osd.6 2023-11-18T20:45:13.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-18T20:45:13.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.254+0000 7f32b8183640 1 Processor -- start 2023-11-18T20:45:13.255 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.254+0000 7f32b8183640 1 -- start start 2023-11-18T20:45:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b8183640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32b0143300 0x7f32b015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b8183640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32b00b0cf0 0x7f32b00b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b8183640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b00b1690 con 0x7f32b00714a0 2023-11-18T20:45:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b8183640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b00b3ca0 con 0x7f32b00b0cf0 2023-11-18T20:45:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b8183640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b015e9f0 con 0x7f32b0143300 2023-11-18T20:45:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b571e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32b00b0cf0 0x7f32b00b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b5f1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32b0143300 0x7f32b015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b6720640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f32b00714a0 0x7f32b0071930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:46476/0 (socket says 172.21.15.60:46476) 2023-11-18T20:45:13.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b571e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32b00b0cf0 0x7f32b00b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:60878/0 (socket says 172.21.15.60:60878) 2023-11-18T20:45:13.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b6720640 1 -- 172.21.15.60:0/3359949532 learned_addr learned my addr 172.21.15.60:0/3359949532 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:13.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b571e640 1 -- 172.21.15.60:0/3359949532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32b0143300 msgr2=0x7f32b015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:13.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b571e640 1 --2- 172.21.15.60:0/3359949532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32b0143300 0x7f32b015e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.258 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b571e640 1 -- 172.21.15.60:0/3359949532 >> v1:172.21.15.100:6789/0 conn(0x7f32b00714a0 legacy=0x7f32b0071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.256+0000 7f32b571e640 1 -- 172.21.15.60:0/3359949532 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00cdb70 con 0x7f32b00b0cf0 2023-11-18T20:45:13.259 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.255+0000 7f32b4f1d640 1 -- 172.21.15.60:0/3359949532 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012218019 0 0) 0x7f32b00b1690 con 0x7f32b00714a0 2023-11-18T20:45:13.261 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.256+0000 7f32b571e640 1 --2- 172.21.15.60:0/3359949532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32b00b0cf0 0x7f32b00b1150 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f32a00097a0 tx=0x7f32a002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2348685540524f0f server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.256+0000 7f32b4f1d640 1 -- 172.21.15.60:0/3359949532 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0031830 con 0x7f32b00b0cf0 2023-11-18T20:45:13.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.257+0000 7f32b4f1d640 1 -- 172.21.15.60:0/3359949532 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a0031990 con 0x7f32b00b0cf0 2023-11-18T20:45:13.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.257+0000 7f32b4f1d640 1 -- 172.21.15.60:0/3359949532 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0031d20 con 0x7f32b00b0cf0 2023-11-18T20:45:13.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.257+0000 7f32b8183640 1 -- 172.21.15.60:0/3359949532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32b00b0cf0 msgr2=0x7f32b00b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.257+0000 7f32b8183640 1 --2- 172.21.15.60:0/3359949532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32b00b0cf0 0x7f32b00b1150 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f32a00097a0 tx=0x7f32a002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.257+0000 7f32b8183640 1 -- 172.21.15.60:0/3359949532 shutdown_connections 2023-11-18T20:45:13.262 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.257+0000 7f32b8183640 1 --2- 172.21.15.60:0/3359949532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32b00b0cf0 0x7f32b00b1150 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.257+0000 7f32b8183640 1 --2- 172.21.15.60:0/3359949532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32b0143300 0x7f32b015e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.257+0000 7f32b8183640 1 -- 172.21.15.60:0/3359949532 >> 172.21.15.60:0/3359949532 conn(0x7f32b006a4d0 msgr2=0x7f32b006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:13.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.257+0000 7f32b8183640 1 -- 172.21.15.60:0/3359949532 shutdown_connections 2023-11-18T20:45:13.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.257+0000 7f32b8183640 1 -- 172.21.15.60:0/3359949532 wait complete. 2023-11-18T20:45:13.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.258+0000 7f32b8183640 1 Processor -- start 2023-11-18T20:45:13.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.258+0000 7f32b8183640 1 -- start start 2023-11-18T20:45:13.263 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b8183640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b00714a0 0x7f32b00acc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b8183640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32b00b0cf0 0x7f32b0157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b8183640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32b0143300 0x7f32b0157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b8183640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00ad180 con 0x7f32b00714a0 2023-11-18T20:45:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b8183640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00adfc0 con 0x7f32b00b0cf0 2023-11-18T20:45:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b8183640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00ad3b0 con 0x7f32b0143300 2023-11-18T20:45:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b5f1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b00714a0 0x7f32b00acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b5f1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b00714a0 0x7f32b00acc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58126/0 (socket says 172.21.15.60:58126) 2023-11-18T20:45:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b5f1f640 1 -- 172.21.15.60:0/4171307447 learned_addr learned my addr 172.21.15.60:0/4171307447 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:13.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b571e640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32b00b0cf0 0x7f32b0157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.259+0000 7f32b6720640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32b0143300 0x7f32b0157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32b5f1f640 1 -- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32b0143300 msgr2=0x7f32b0157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32b5f1f640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32b0143300 0x7f32b0157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32b5f1f640 1 -- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32b00b0cf0 msgr2=0x7f32b0157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32b5f1f640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32b00b0cf0 0x7f32b0157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32b5f1f640 1 -- 172.21.15.60:0/4171307447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0009400 con 0x7f32b00714a0 2023-11-18T20:45:13.265 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32b5f1f640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b00714a0 0x7f32b00acc40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00e9b0 tx=0x7f32ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32a67fc640 1 -- 172.21.15.60:0/4171307447 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac009670 con 0x7f32b00714a0 2023-11-18T20:45:13.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32a67fc640 1 -- 172.21.15.60:0/4171307447 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac00c9a0 con 0x7f32b00714a0 2023-11-18T20:45:13.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32a67fc640 1 -- 172.21.15.60:0/4171307447 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00cd50 con 0x7f32b00714a0 2023-11-18T20:45:13.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b00ad690 con 0x7f32b00714a0 2023-11-18T20:45:13.266 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.260+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b00adc50 con 0x7f32b00714a0 2023-11-18T20:45:13.267 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.261+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32b00cc2d0 con 0x7f32b00714a0 2023-11-18T20:45:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.261+0000 7f32a67fc640 1 -- 172.21.15.60:0/4171307447 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32ac013470 con 0x7f32b00714a0 2023-11-18T20:45:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.262+0000 7f32a67fc640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f329406ae40 0x7f329406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.262+0000 7f32a67fc640 1 -- 172.21.15.60:0/4171307447 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f32ac089500 con 0x7f32b00714a0 2023-11-18T20:45:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.262+0000 7f32a67fc640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] conn(0x7f3294070ca0 0x7f32940730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.262+0000 7f32a67fc640 1 -- 172.21.15.60:0/4171307447 --> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3294073770 con 0x7f3294070ca0 2023-11-18T20:45:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.262+0000 7f32a67fc640 1 -- 172.21.15.60:0/4171307447 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32ac08bbc0 con 0x7f32b00714a0 2023-11-18T20:45:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.262+0000 7f32b571e640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f329406ae40 0x7f329406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.262+0000 7f32b6720640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] conn(0x7f3294070ca0 0x7f32940730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.263+0000 7f32b571e640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f329406ae40 0x7f329406d300 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f32a0036000 tx=0x7f32a003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.263+0000 7f32b6720640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] conn(0x7f3294070ca0 0x7f32940730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.263+0000 7f32a67fc640 1 -- 172.21.15.60:0/4171307447 <== osd.6 v2:172.21.15.100:6802/1390121871 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3294073770 con 0x7f3294070ca0 2023-11-18T20:45:13.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 --> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f32b0070ab0 con 0x7f3294070ca0 2023-11-18T20:45:13.278 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32a67fc640 1 -- 172.21.15.60:0/4171307447 <== osd.6 v2:172.21.15.100:6802/1390121871 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f32b0070ab0 con 0x7f3294070ca0 2023-11-18T20:45:13.278 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:45:13.279 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-18T20:45:13.279 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:45:13.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] conn(0x7f3294070ca0 msgr2=0x7f32940730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:6802/1390121871,v1:172.21.15.100:6805/1390121871] conn(0x7f3294070ca0 0x7f32940730c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f329406ae40 msgr2=0x7f329406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f329406ae40 0x7f329406d300 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f32a0036000 tx=0x7f32a003c040 comp rx=0 tx=0).stop 2023-11-18T20:45:13.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b00714a0 msgr2=0x7f32b00acc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.280 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 --2- 172.21.15.60:0/4171307447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32b00714a0 0x7f32b00acc40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00e9b0 tx=0x7f32ac00ee70 comp rx=0 tx=0).stop 2023-11-18T20:45:13.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b6720640 1 -- 172.21.15.60:0/4171307447 reap_dead start 2023-11-18T20:45:13.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 shutdown_connections 2023-11-18T20:45:13.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 >> 172.21.15.60:0/4171307447 conn(0x7f32b006a4d0 msgr2=0x7f32b01598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:13.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 shutdown_connections 2023-11-18T20:45:13.281 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.277+0000 7f32b8183640 1 -- 172.21.15.60:0/4171307447 wait complete. 2023-11-18T20:45:13.289 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-18T20:45:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.394+0000 7f01ca62b640 1 Processor -- start 2023-11-18T20:45:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.395+0000 7f01ca62b640 1 -- start start 2023-11-18T20:45:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.395+0000 7f01ca62b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c40b1400 0x7f01c415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.397 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.395+0000 7f01ca62b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f01c415e730 0x7f01c415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.395+0000 7f01ca62b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c415e2a0 con 0x7f01c4143490 2023-11-18T20:45:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.395+0000 7f01ca62b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c415b520 con 0x7f01c415e730 2023-11-18T20:45:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.395+0000 7f01ca62b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4143870 con 0x7f01c40b1400 2023-11-18T20:45:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.395+0000 7f01c3fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f01c4143490 0x7f01c40b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57708/0 (socket says 172.21.15.60:57708) 2023-11-18T20:45:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.395+0000 7f01c3fff640 1 -- 172.21.15.60:0/2300781534 learned_addr learned my addr 172.21.15.60:0/2300781534 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.396+0000 7f01c37fe640 1 --2- 172.21.15.60:0/2300781534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c40b1400 0x7f01c415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.398 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.396+0000 7f01c8bc8640 1 --2- 172.21.15.60:0/2300781534 >> v2:172.21.15.100:3300/0 conn(0x7f01c415e730 0x7f01c415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.396+0000 7f01c2ffd640 1 -- 172.21.15.60:0/2300781534 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225197397 0 0) 0x7f01c415e2a0 con 0x7f01c4143490 2023-11-18T20:45:13.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.396+0000 7f01c2ffd640 1 -- 172.21.15.60:0/2300781534 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ac003520 con 0x7f01c4143490 2023-11-18T20:45:13.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.396+0000 7f01c37fe640 1 -- 172.21.15.60:0/2300781534 >> v2:172.21.15.100:3300/0 conn(0x7f01c415e730 msgr2=0x7f01c415aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:13.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.396+0000 7f01c37fe640 1 --2- 172.21.15.60:0/2300781534 >> v2:172.21.15.100:3300/0 conn(0x7f01c415e730 0x7f01c415aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.396+0000 7f01c37fe640 1 -- 172.21.15.60:0/2300781534 >> v1:172.21.15.60:6789/0 conn(0x7f01c4143490 legacy=0x7f01c40b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.399 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.396+0000 7f01c37fe640 1 -- 172.21.15.60:0/2300781534 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c415b240 con 0x7f01c40b1400 2023-11-18T20:45:13.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01c37fe640 1 --2- 172.21.15.60:0/2300781534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c40b1400 0x7f01c415a4e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f01b80097a0 tx=0x7f01b802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a9146a1055e674 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01c2ffd640 1 -- 172.21.15.60:0/2300781534 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8031830 con 0x7f01c40b1400 2023-11-18T20:45:13.403 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01c2ffd640 1 -- 172.21.15.60:0/2300781534 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b8031990 con 0x7f01c40b1400 2023-11-18T20:45:13.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01c2ffd640 1 -- 172.21.15.60:0/2300781534 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8031d20 con 0x7f01c40b1400 2023-11-18T20:45:13.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01ca62b640 1 -- 172.21.15.60:0/2300781534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c40b1400 msgr2=0x7f01c415a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01ca62b640 1 --2- 172.21.15.60:0/2300781534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c40b1400 0x7f01c415a4e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f01b80097a0 tx=0x7f01b802b8d0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01ca62b640 1 -- 172.21.15.60:0/2300781534 shutdown_connections 2023-11-18T20:45:13.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01ca62b640 1 --2- 172.21.15.60:0/2300781534 >> v2:172.21.15.100:3300/0 conn(0x7f01c415e730 0x7f01c415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.404 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01ca62b640 1 --2- 172.21.15.60:0/2300781534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c40b1400 0x7f01c415a4e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01ca62b640 1 -- 172.21.15.60:0/2300781534 >> 172.21.15.60:0/2300781534 conn(0x7f01c406a4d0 msgr2=0x7f01c406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:13.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.397+0000 7f01ca62b640 1 -- 172.21.15.60:0/2300781534 shutdown_connections 2023-11-18T20:45:13.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.398+0000 7f01ca62b640 1 -- 172.21.15.60:0/2300781534 wait complete. 2023-11-18T20:45:13.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.399+0000 7f01ca62b640 1 Processor -- start 2023-11-18T20:45:13.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.399+0000 7f01ca62b640 1 -- start start 2023-11-18T20:45:13.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.399+0000 7f01ca62b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c40b1400 0x7f01c41420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.399+0000 7f01ca62b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4143490 0x7f01c40c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.405 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01ca62b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c415e730 0x7f01c4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01ca62b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c413fd70 con 0x7f01c40b1400 2023-11-18T20:45:13.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01ca62b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4134ed0 con 0x7f01c4143490 2023-11-18T20:45:13.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01ca62b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c41351b0 con 0x7f01c415e730 2023-11-18T20:45:13.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c40b1400 0x7f01c41420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c40b1400 0x7f01c41420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58142/0 (socket says 172.21.15.60:58142) 2023-11-18T20:45:13.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c3fff640 1 -- 172.21.15.60:0/1642440302 learned_addr learned my addr 172.21.15.60:0/1642440302 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:13.406 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c37fe640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4143490 0x7f01c40c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c8bc8640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c415e730 0x7f01c4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c3fff640 1 -- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c415e730 msgr2=0x7f01c4070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c3fff640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c415e730 0x7f01c4070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c3fff640 1 -- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4143490 msgr2=0x7f01c40c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c3fff640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4143490 0x7f01c40c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c3fff640 1 -- 172.21.15.60:0/1642440302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b8009400 con 0x7f01c40b1400 2023-11-18T20:45:13.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c3fff640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c40b1400 0x7f01c41420d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c500 tx=0x7f01b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.407 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c0ff9640 1 -- 172.21.15.60:0/1642440302 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4005ce0 con 0x7f01c40b1400 2023-11-18T20:45:13.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c0ff9640 1 -- 172.21.15.60:0/1642440302 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b4005e40 con 0x7f01c40b1400 2023-11-18T20:45:13.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01ca62b640 1 -- 172.21.15.60:0/1642440302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c4135490 con 0x7f01c40b1400 2023-11-18T20:45:13.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01c0ff9640 1 -- 172.21.15.60:0/1642440302 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4007330 con 0x7f01c40b1400 2023-11-18T20:45:13.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.400+0000 7f01ca62b640 1 -- 172.21.15.60:0/1642440302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c4142610 con 0x7f01c40b1400 2023-11-18T20:45:13.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.401+0000 7f01ca62b640 1 -- 172.21.15.60:0/1642440302 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c41329a0 con 0x7f01c40b1400 2023-11-18T20:45:13.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.402+0000 7f01c0ff9640 1 -- 172.21.15.60:0/1642440302 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01b4007550 con 0x7f01c40b1400 2023-11-18T20:45:13.408 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.402+0000 7f01c0ff9640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f01b006ae40 0x7f01b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.402+0000 7f01c0ff9640 1 -- 172.21.15.60:0/1642440302 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f01b4088870 con 0x7f01c40b1400 2023-11-18T20:45:13.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.402+0000 7f01c37fe640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f01b006ae40 0x7f01b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.403+0000 7f01c37fe640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f01b006ae40 0x7f01b006d300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f01b8036000 tx=0x7f01b803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.409 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.407+0000 7f01c0ff9640 1 -- 172.21.15.60:0/1642440302 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01b4053e20 con 0x7f01c40b1400 2023-11-18T20:45:13.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.585+0000 7f01ca62b640 1 -- 172.21.15.60:0/1642440302 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f01c415af60 con 0x7f01b006ae40 2023-11-18T20:45:13.587 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.586+0000 7f01c0ff9640 1 -- 172.21.15.60:0/1642440302 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f01b8031b00 con 0x7f01b006ae40 2023-11-18T20:45:13.587 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 6 to deep-scrub 2023-11-18T20:45:13.589 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 -- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f01b006ae40 msgr2=0x7f01b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f01b006ae40 0x7f01b006d300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f01b8036000 tx=0x7f01b803c040 comp rx=0 tx=0).stop 2023-11-18T20:45:13.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 -- 172.21.15.60:0/1642440302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c40b1400 msgr2=0x7f01c41420d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c40b1400 0x7f01c41420d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c500 tx=0x7f01b400c9c0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 -- 172.21.15.60:0/1642440302 shutdown_connections 2023-11-18T20:45:13.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f01b006ae40 0x7f01b006d300 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01c415e730 0x7f01c4070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.590 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01c4143490 0x7f01c40c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 --2- 172.21.15.60:0/1642440302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01c40b1400 0x7f01c41420d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 -- 172.21.15.60:0/1642440302 >> 172.21.15.60:0/1642440302 conn(0x7f01c406a4d0 msgr2=0x7f01c40b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 -- 172.21.15.60:0/1642440302 shutdown_connections 2023-11-18T20:45:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.588+0000 7f01ca62b640 1 -- 172.21.15.60:0/1642440302 wait complete. 2023-11-18T20:45:13.600 INFO:tasks.ceph:Scrubbing osd.7 2023-11-18T20:45:13.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-18T20:45:13.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.710+0000 7fe1f8c90640 1 Processor -- start 2023-11-18T20:45:13.711 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.710+0000 7fe1f8c90640 1 -- start start 2023-11-18T20:45:13.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1f8c90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 0x7fe1f40b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1f8c90640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1f40b3e60 0x7fe1f415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1f8c90640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f40b42c0 con 0x7fe1f415a7f0 2023-11-18T20:45:13.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1f8c90640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f40b4560 con 0x7fe1f40b3e60 2023-11-18T20:45:13.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1f8c90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40b4840 con 0x7fe1f4159ed0 2023-11-18T20:45:13.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1f259c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 0x7fe1f40b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.713 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1ebfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1f40b3e60 0x7fe1f415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1f259c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 0x7fe1f40b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:33650/0 (socket says 172.21.15.60:33650) 2023-11-18T20:45:13.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1f259c640 1 -- 172.21.15.60:0/1980989065 learned_addr learned my addr 172.21.15.60:0/1980989065 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:13.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1ebfff640 1 -- 172.21.15.60:0/1980989065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 msgr2=0x7fe1f40b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:13.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1ebfff640 1 --2- 172.21.15.60:0/1980989065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 0x7fe1f40b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.711+0000 7fe1ebfff640 1 -- 172.21.15.60:0/1980989065 >> v1:172.21.15.100:6789/0 conn(0x7fe1f415a7f0 legacy=0x7fe1f4070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.712+0000 7fe1ebfff640 1 -- 172.21.15.60:0/1980989065 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f40b4b20 con 0x7fe1f40b3e60 2023-11-18T20:45:13.714 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.712+0000 7fe1f259c640 1 --2- 172.21.15.60:0/1980989065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 0x7fe1f40b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:13.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.712+0000 7fe1ebfff640 1 --2- 172.21.15.60:0/1980989065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f40b3e60 0x7fe1f415a2b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0097a0 tx=0x7fe1dc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c748c77fd92b396 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f1d9b640 1 -- 172.21.15.60:0/1980989065 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc031830 con 0x7fe1f40b3e60 2023-11-18T20:45:13.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f1d9b640 1 -- 172.21.15.60:0/1980989065 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc031990 con 0x7fe1f40b3e60 2023-11-18T20:45:13.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f1d9b640 1 -- 172.21.15.60:0/1980989065 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc031d20 con 0x7fe1f40b3e60 2023-11-18T20:45:13.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f8c90640 1 -- 172.21.15.60:0/1980989065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f40b3e60 msgr2=0x7fe1f415a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.716 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f8c90640 1 --2- 172.21.15.60:0/1980989065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f40b3e60 0x7fe1f415a2b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0097a0 tx=0x7fe1dc02b8d0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f8c90640 1 -- 172.21.15.60:0/1980989065 shutdown_connections 2023-11-18T20:45:13.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f8c90640 1 --2- 172.21.15.60:0/1980989065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f40b3e60 0x7fe1f415a2b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f8c90640 1 --2- 172.21.15.60:0/1980989065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 0x7fe1f40b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f8c90640 1 -- 172.21.15.60:0/1980989065 >> 172.21.15.60:0/1980989065 conn(0x7fe1f406a4d0 msgr2=0x7fe1f406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:13.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f8c90640 1 -- 172.21.15.60:0/1980989065 shutdown_connections 2023-11-18T20:45:13.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.713+0000 7fe1f8c90640 1 -- 172.21.15.60:0/1980989065 wait complete. 2023-11-18T20:45:13.717 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.714+0000 7fe1f8c90640 1 Processor -- start 2023-11-18T20:45:13.718 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.714+0000 7fe1f8c90640 1 -- start start 2023-11-18T20:45:13.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.715+0000 7fe1f8c90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f40b3e60 0x7fe1f414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.715+0000 7fe1f8c90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 0x7fe1f41e3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.715+0000 7fe1f8c90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f415a7f0 0x7fe1f4151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.715+0000 7fe1f8c90640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f414af00 con 0x7fe1f40b3e60 2023-11-18T20:45:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.715+0000 7fe1f8c90640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f41e37a0 con 0x7fe1f415a7f0 2023-11-18T20:45:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.715+0000 7fe1f8c90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f41dfde0 con 0x7fe1f4159ed0 2023-11-18T20:45:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.715+0000 7fe1f259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f40b3e60 0x7fe1f414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.715+0000 7fe1f259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f40b3e60 0x7fe1f414a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58144/0 (socket says 172.21.15.60:58144) 2023-11-18T20:45:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.715+0000 7fe1f259c640 1 -- 172.21.15.60:0/409161541 learned_addr learned my addr 172.21.15.60:0/409161541 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1ebfff640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 0x7fe1f41e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1f2d9d640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f415a7f0 0x7fe1f4151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1f259c640 1 -- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 msgr2=0x7fe1f41e3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1f259c640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1f4159ed0 0x7fe1f41e3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1f259c640 1 -- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f415a7f0 msgr2=0x7fe1f4151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1f259c640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f415a7f0 0x7fe1f4151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1f259c640 1 -- 172.21.15.60:0/409161541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e400b760 con 0x7fe1f40b3e60 2023-11-18T20:45:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1f2d9d640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f415a7f0 0x7fe1f4151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:45:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1f259c640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f40b3e60 0x7fe1f414a9c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400dc70 tx=0x7fe1e40094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1eaffd640 1 -- 172.21.15.60:0/409161541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400de80 con 0x7fe1f40b3e60 2023-11-18T20:45:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1eaffd640 1 -- 172.21.15.60:0/409161541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e4009e10 con 0x7fe1f40b3e60 2023-11-18T20:45:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc009400 con 0x7fe1f40b3e60 2023-11-18T20:45:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1eaffd640 1 -- 172.21.15.60:0/409161541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4017610 con 0x7fe1f40b3e60 2023-11-18T20:45:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.716+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f41e03a0 con 0x7fe1f40b3e60 2023-11-18T20:45:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.717+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1f40b50e0 con 0x7fe1f40b3e60 2023-11-18T20:45:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.717+0000 7fe1eaffd640 1 -- 172.21.15.60:0/409161541 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe1e4017770 con 0x7fe1f40b3e60 2023-11-18T20:45:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.718+0000 7fe1eaffd640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe1d406ae40 0x7fe1d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.718+0000 7fe1eaffd640 1 -- 172.21.15.60:0/409161541 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe1e400b050 con 0x7fe1f40b3e60 2023-11-18T20:45:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.718+0000 7fe1eaffd640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] conn(0x7fe1d4070ca0 0x7fe1d40730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.718+0000 7fe1eaffd640 1 -- 172.21.15.60:0/409161541 --> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1d4073770 con 0x7fe1d4070ca0 2023-11-18T20:45:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.718+0000 7fe1eaffd640 1 -- 172.21.15.60:0/409161541 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1e408ba90 con 0x7fe1f40b3e60 2023-11-18T20:45:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.718+0000 7fe1ebfff640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe1d406ae40 0x7fe1d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.718+0000 7fe1f2d9d640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] conn(0x7fe1d4070ca0 0x7fe1d40730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.719+0000 7fe1ebfff640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe1d406ae40 0x7fe1d406d300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc036000 tx=0x7fe1dc03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.719+0000 7fe1f2d9d640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] conn(0x7fe1d4070ca0 0x7fe1d40730c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.720+0000 7fe1eaffd640 1 -- 172.21.15.60:0/409161541 <== osd.7 v2:172.21.15.100:6824/1780039900 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe1d4073770 con 0x7fe1d4070ca0 2023-11-18T20:45:13.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.733+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 --> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe1f40b12a0 con 0x7fe1d4070ca0 2023-11-18T20:45:13.734 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.733+0000 7fe1eaffd640 1 -- 172.21.15.60:0/409161541 <== osd.7 v2:172.21.15.100:6824/1780039900 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe1f40b12a0 con 0x7fe1d4070ca0 2023-11-18T20:45:13.735 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-18T20:45:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-18T20:45:13.735 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-18T20:45:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] conn(0x7fe1d4070ca0 msgr2=0x7fe1d40730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.736 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f8c90640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:6824/1780039900,v1:172.21.15.100:6825/1780039900] conn(0x7fe1d4070ca0 0x7fe1d40730c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe1d406ae40 msgr2=0x7fe1d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f8c90640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fe1d406ae40 0x7fe1d406d300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc036000 tx=0x7fe1dc03c040 comp rx=0 tx=0).stop 2023-11-18T20:45:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f40b3e60 msgr2=0x7fe1f414a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f8c90640 1 --2- 172.21.15.60:0/409161541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1f40b3e60 0x7fe1f414a9c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400dc70 tx=0x7fe1e40094f0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f2d9d640 1 -- 172.21.15.60:0/409161541 reap_dead start 2023-11-18T20:45:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 shutdown_connections 2023-11-18T20:45:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 >> 172.21.15.60:0/409161541 conn(0x7fe1f406a4d0 msgr2=0x7fe1f4153960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 shutdown_connections 2023-11-18T20:45:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.734+0000 7fe1f8c90640 1 -- 172.21.15.60:0/409161541 wait complete. 2023-11-18T20:45:13.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-18T20:45:13.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.851+0000 7f6d5207c640 1 Processor -- start 2023-11-18T20:45:13.852 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.851+0000 7f6d5207c640 1 -- start start 2023-11-18T20:45:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d5207c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 0x7f6d4c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d5207c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6d4c1409d0 0x7f6d4c06a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d5207c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d4c1428d0 0x7f6d4c06aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d5207c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4c161e20 con 0x7f6d4c1428d0 2023-11-18T20:45:13.854 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d5207c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4c1406f0 con 0x7f6d4c1409d0 2023-11-18T20:45:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d5207c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c143960 con 0x7f6d4c140310 2023-11-18T20:45:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d4bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d4c1428d0 0x7f6d4c06aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d4bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d4c1428d0 0x7f6d4c06aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58154/0 (socket says 172.21.15.60:58154) 2023-11-18T20:45:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d4bfff640 1 -- 172.21.15.60:0/2237307571 learned_addr learned my addr 172.21.15.60:0/2237307571 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d4affd640 1 --2- 172.21.15.60:0/2237307571 >> v2:172.21.15.100:3300/0 conn(0x7f6d4c1409d0 0x7f6d4c06a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d4b7fe640 1 --2- 172.21.15.60:0/2237307571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 0x7f6d4c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.855 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d4bfff640 1 -- 172.21.15.60:0/2237307571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 msgr2=0x7f6d4c1618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d4bfff640 1 --2- 172.21.15.60:0/2237307571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 0x7f6d4c1618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d4bfff640 1 -- 172.21.15.60:0/2237307571 >> v2:172.21.15.100:3300/0 conn(0x7f6d4c1409d0 msgr2=0x7f6d4c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d4bfff640 1 --2- 172.21.15.60:0/2237307571 >> v2:172.21.15.100:3300/0 conn(0x7f6d4c1409d0 0x7f6d4c06a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.856 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.852+0000 7f6d4bfff640 1 -- 172.21.15.60:0/2237307571 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c142de0 con 0x7f6d4c1428d0 2023-11-18T20:45:13.860 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.853+0000 7f6d4bfff640 1 --2- 172.21.15.60:0/2237307571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1428d0 0x7f6d4c06aa10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c06e630 tx=0x7f6d4402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f80ef0bb599b671 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.853+0000 7f6d4a7fc640 1 -- 172.21.15.60:0/2237307571 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4402c9e0 con 0x7f6d4c1428d0 2023-11-18T20:45:13.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.853+0000 7f6d4a7fc640 1 -- 172.21.15.60:0/2237307571 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4402cb40 con 0x7f6d4c1428d0 2023-11-18T20:45:13.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.853+0000 7f6d4a7fc640 1 -- 172.21.15.60:0/2237307571 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d440343f0 con 0x7f6d4c1428d0 2023-11-18T20:45:13.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.854+0000 7f6d5207c640 1 -- 172.21.15.60:0/2237307571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1428d0 msgr2=0x7f6d4c06aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.861 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.854+0000 7f6d5207c640 1 --2- 172.21.15.60:0/2237307571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1428d0 0x7f6d4c06aa10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c06e630 tx=0x7f6d4402be40 comp rx=0 tx=0).stop 2023-11-18T20:45:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.854+0000 7f6d5207c640 1 -- 172.21.15.60:0/2237307571 shutdown_connections 2023-11-18T20:45:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.854+0000 7f6d5207c640 1 --2- 172.21.15.60:0/2237307571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1428d0 0x7f6d4c06aa10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.854+0000 7f6d5207c640 1 --2- 172.21.15.60:0/2237307571 >> v2:172.21.15.100:3300/0 conn(0x7f6d4c1409d0 0x7f6d4c06a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.862 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.854+0000 7f6d5207c640 1 --2- 172.21.15.60:0/2237307571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 0x7f6d4c1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.854+0000 7f6d5207c640 1 -- 172.21.15.60:0/2237307571 >> 172.21.15.60:0/2237307571 conn(0x7f6d4c0b1e50 msgr2=0x7f6d4c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.854+0000 7f6d5207c640 1 -- 172.21.15.60:0/2237307571 shutdown_connections 2023-11-18T20:45:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.854+0000 7f6d5207c640 1 -- 172.21.15.60:0/2237307571 wait complete. 2023-11-18T20:45:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.855+0000 7f6d5207c640 1 Processor -- start 2023-11-18T20:45:13.863 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.855+0000 7f6d5207c640 1 -- start start 2023-11-18T20:45:13.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d5207c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 0x7f6d4c146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d5207c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1409d0 0x7f6d4c1f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d5207c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c1428d0 0x7f6d4c131da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.864 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d5207c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c1f73c0 con 0x7f6d4c1409d0 2023-11-18T20:45:13.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d5207c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c144340 con 0x7f6d4c1428d0 2023-11-18T20:45:13.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d5207c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c143d80 con 0x7f6d4c140310 2023-11-18T20:45:13.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1409d0 0x7f6d4c1f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1409d0 0x7f6d4c1f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58170/0 (socket says 172.21.15.60:58170) 2023-11-18T20:45:13.865 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4affd640 1 -- 172.21.15.60:0/1669595896 learned_addr learned my addr 172.21.15.60:0/1669595896 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:13.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4b7fe640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 0x7f6d4c146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4bfff640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c1428d0 0x7f6d4c131da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4affd640 1 -- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 msgr2=0x7f6d4c146a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4affd640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 0x7f6d4c146a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.866 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4affd640 1 -- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c1428d0 msgr2=0x7f6d4c131da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:13.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4affd640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c1428d0 0x7f6d4c131da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:13.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4affd640 1 -- 172.21.15.60:0/1669595896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d44009b30 con 0x7f6d4c1409d0 2023-11-18T20:45:13.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4b7fe640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 0x7f6d4c146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:45:13.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.856+0000 7f6d4bfff640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c1428d0 0x7f6d4c131da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-18T20:45:13.867 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.857+0000 7f6d4affd640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1409d0 0x7f6d4c1f6e80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000e840 tx=0x7f6d4000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.857+0000 7f6d33fff640 1 -- 172.21.15.60:0/1669595896 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d400094f0 con 0x7f6d4c1409d0 2023-11-18T20:45:13.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.857+0000 7f6d33fff640 1 -- 172.21.15.60:0/1669595896 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4000c940 con 0x7f6d4c1409d0 2023-11-18T20:45:13.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.857+0000 7f6d5207c640 1 -- 172.21.15.60:0/1669595896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c144060 con 0x7f6d4c1409d0 2023-11-18T20:45:13.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.857+0000 7f6d33fff640 1 -- 172.21.15.60:0/1669595896 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4000ccd0 con 0x7f6d4c1409d0 2023-11-18T20:45:13.868 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.857+0000 7f6d5207c640 1 -- 172.21.15.60:0/1669595896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c131880 con 0x7f6d4c1409d0 2023-11-18T20:45:13.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.858+0000 7f6d5207c640 1 -- 172.21.15.60:0/1669595896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c069400 con 0x7f6d4c1409d0 2023-11-18T20:45:13.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.858+0000 7f6d33fff640 1 -- 172.21.15.60:0/1669595896 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d40013470 con 0x7f6d4c1409d0 2023-11-18T20:45:13.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.859+0000 7f6d33fff640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6d3806ae40 0x7f6d3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:13.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.859+0000 7f6d33fff640 1 -- 172.21.15.60:0/1669595896 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6d400893a0 con 0x7f6d4c1409d0 2023-11-18T20:45:13.869 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.859+0000 7f6d4b7fe640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6d3806ae40 0x7f6d3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.860+0000 7f6d4b7fe640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6d3806ae40 0x7f6d3806d300 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0099a0 tx=0x7f6d3c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:13.863+0000 7f6d33fff640 1 -- 172.21.15.60:0/1669595896 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d400547e0 con 0x7f6d4c1409d0 2023-11-18T20:45:14.041 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.039+0000 7f6d5207c640 1 -- 172.21.15.60:0/1669595896 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d4c143680 con 0x7f6d3806ae40 2023-11-18T20:45:14.042 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.040+0000 7f6d33fff640 1 -- 172.21.15.60:0/1669595896 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6d3c004020 con 0x7f6d3806ae40 2023-11-18T20:45:14.042 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 7 to deep-scrub 2023-11-18T20:45:14.043 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 -- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6d3806ae40 msgr2=0x7f6d3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:14.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6d3806ae40 0x7f6d3806d300 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0099a0 tx=0x7f6d3c008040 comp rx=0 tx=0).stop 2023-11-18T20:45:14.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 -- 172.21.15.60:0/1669595896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1409d0 msgr2=0x7f6d4c1f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:14.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1409d0 0x7f6d4c1f6e80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000e840 tx=0x7f6d4000ed00 comp rx=0 tx=0).stop 2023-11-18T20:45:14.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 -- 172.21.15.60:0/1669595896 shutdown_connections 2023-11-18T20:45:14.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f6d3806ae40 0x7f6d3806d300 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d4c1428d0 0x7f6d4c131da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.044 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d4c1409d0 0x7f6d4c1f6e80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 --2- 172.21.15.60:0/1669595896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4c140310 0x7f6d4c146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 -- 172.21.15.60:0/1669595896 >> 172.21.15.60:0/1669595896 conn(0x7f6d4c0b1e50 msgr2=0x7f6d4c06edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:14.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 -- 172.21.15.60:0/1669595896 shutdown_connections 2023-11-18T20:45:14.045 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.042+0000 7f6d5207c640 1 -- 172.21.15.60:0/1669595896 wait complete. 2023-11-18T20:45:14.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:45:14.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.159+0000 7f9ef8a7c640 1 Processor -- start 2023-11-18T20:45:14.161 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.159+0000 7f9ef8a7c640 1 -- start start 2023-11-18T20:45:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.160+0000 7f9ef8a7c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ef415e730 0x7f9ef415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.160+0000 7f9ef8a7c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef415e2a0 con 0x7f9ef4143490 2023-11-18T20:45:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.160+0000 7f9ef8a7c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef415b240 con 0x7f9ef40b1400 2023-11-18T20:45:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.160+0000 7f9ef8a7c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef415b520 con 0x7f9ef415e730 2023-11-18T20:45:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.161+0000 7f9ef2d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ef415e730 0x7f9ef415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.161+0000 7f9ef259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9ef4143490 0x7f9ef40b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57716/0 (socket says 172.21.15.60:57716) 2023-11-18T20:45:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.161+0000 7f9ef259c640 1 -- 172.21.15.60:0/852478818 learned_addr learned my addr 172.21.15.60:0/852478818 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.161+0000 7f9ef2d9d640 1 -- 172.21.15.60:0/852478818 >> v1:172.21.15.100:6789/0 conn(0x7f9ef40b1400 legacy=0x7f9ef415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.161+0000 7f9ef2d9d640 1 -- 172.21.15.60:0/852478818 >> v1:172.21.15.60:6789/0 conn(0x7f9ef4143490 legacy=0x7f9ef40b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:14.164 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.161+0000 7f9ef2d9d640 1 -- 172.21.15.60:0/852478818 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef415dc50 con 0x7f9ef415e730 2023-11-18T20:45:14.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef2d9d640 1 --2- 172.21.15.60:0/852478818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef415e730 0x7f9ef415abf0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009d90 tx=0x7f9eec02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3812c597ef54 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:14.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef159a640 1 -- 172.21.15.60:0/852478818 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec02cc60 con 0x7f9ef415e730 2023-11-18T20:45:14.167 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef159a640 1 -- 172.21.15.60:0/852478818 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec02cdc0 con 0x7f9ef415e730 2023-11-18T20:45:14.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef159a640 1 -- 172.21.15.60:0/852478818 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec034660 con 0x7f9ef415e730 2023-11-18T20:45:14.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/852478818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef415e730 msgr2=0x7f9ef415abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:14.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef8a7c640 1 --2- 172.21.15.60:0/852478818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef415e730 0x7f9ef415abf0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009d90 tx=0x7f9eec02beb0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/852478818 shutdown_connections 2023-11-18T20:45:14.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef8a7c640 1 --2- 172.21.15.60:0/852478818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef415e730 0x7f9ef415abf0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/852478818 >> 172.21.15.60:0/852478818 conn(0x7f9ef406a4d0 msgr2=0x7f9ef406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/852478818 shutdown_connections 2023-11-18T20:45:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.162+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/852478818 wait complete. 2023-11-18T20:45:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.164+0000 7f9ef8a7c640 1 Processor -- start 2023-11-18T20:45:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.164+0000 7f9ef8a7c640 1 -- start start 2023-11-18T20:45:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.164+0000 7f9ef8a7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef40b1400 0x7f9ef4134eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.164+0000 7f9ef8a7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef4143490 0x7f9ef413a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:14.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.164+0000 7f9ef8a7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef415e730 0x7f9ef4135920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.164+0000 7f9ef8a7c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef4138fd0 con 0x7f9ef4143490 2023-11-18T20:45:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.164+0000 7f9ef8a7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef41392b0 con 0x7f9ef40b1400 2023-11-18T20:45:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.164+0000 7f9ef8a7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef4139590 con 0x7f9ef415e730 2023-11-18T20:45:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef4143490 0x7f9ef413a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef4143490 0x7f9ef413a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58182/0 (socket says 172.21.15.60:58182) 2023-11-18T20:45:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef1d9b640 1 -- 172.21.15.60:0/774933620 learned_addr learned my addr 172.21.15.60:0/774933620 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef259c640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef40b1400 0x7f9ef4134eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:14.170 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef2d9d640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef415e730 0x7f9ef4135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef1d9b640 1 -- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef415e730 msgr2=0x7f9ef4135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef1d9b640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef415e730 0x7f9ef4135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef1d9b640 1 -- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef40b1400 msgr2=0x7f9ef4134eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef1d9b640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef40b1400 0x7f9ef4134eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef1d9b640 1 -- 172.21.15.60:0/774933620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec009a70 con 0x7f9ef4143490 2023-11-18T20:45:14.171 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef259c640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef40b1400 0x7f9ef4134eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef2d9d640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef415e730 0x7f9ef4135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef1d9b640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef4143490 0x7f9ef413a070 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800c430 tx=0x7f9ee800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ee2ffd640 1 -- 172.21.15.60:0/774933620 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8011790 con 0x7f9ef4143490 2023-11-18T20:45:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ee2ffd640 1 -- 172.21.15.60:0/774933620 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee80118f0 con 0x7f9ef4143490 2023-11-18T20:45:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/774933620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef4139870 con 0x7f9ef4143490 2023-11-18T20:45:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ee2ffd640 1 -- 172.21.15.60:0/774933620 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8011ce0 con 0x7f9ef4143490 2023-11-18T20:45:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.165+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/774933620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef41353f0 con 0x7f9ef4143490 2023-11-18T20:45:14.172 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.166+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/774933620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef4139d90 con 0x7f9ef4143490 2023-11-18T20:45:14.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.167+0000 7f9ee2ffd640 1 -- 172.21.15.60:0/774933620 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ee801c460 con 0x7f9ef4143490 2023-11-18T20:45:14.173 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.167+0000 7f9ee2ffd640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ed406ae40 0x7f9ed406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.167+0000 7f9ee2ffd640 1 -- 172.21.15.60:0/774933620 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9ee8089250 con 0x7f9ef4143490 2023-11-18T20:45:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.167+0000 7f9ef259c640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ed406ae40 0x7f9ed406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.168+0000 7f9ef259c640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ed406ae40 0x7f9ed406d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005880 tx=0x7f9ee4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:14.174 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.171+0000 7f9ee2ffd640 1 -- 172.21.15.60:0/774933620 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ef4139d90 con 0x7f9ef4143490 2023-11-18T20:45:14.343 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.342+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/774933620 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ef4139d90 con 0x7f9ed406ae40 2023-11-18T20:45:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.345+0000 7f9ee2ffd640 1 -- 172.21.15.60:0/774933620 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51192 (secure 0 0 0) 0x7f9ef4139d90 con 0x7f9ed406ae40 2023-11-18T20:45:14.347 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:45:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ed406ae40 msgr2=0x7f9ed406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ed406ae40 0x7f9ed406d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005880 tx=0x7f9ee4009040 comp rx=0 tx=0).stop 2023-11-18T20:45:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/774933620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef4143490 msgr2=0x7f9ef413a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef4143490 0x7f9ef413a070 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800c430 tx=0x7f9ee800c8f0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/774933620 shutdown_connections 2023-11-18T20:45:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f9ed406ae40 0x7f9ed406d300 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef415e730 0x7f9ef4135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef4143490 0x7f9ef413a070 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 --2- 172.21.15.60:0/774933620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef40b1400 0x7f9ef4134eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/774933620 >> 172.21.15.60:0/774933620 conn(0x7f9ef406a4d0 msgr2=0x7f9ef40b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/774933620 shutdown_connections 2023-11-18T20:45:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:14.347+0000 7f9ef8a7c640 1 -- 172.21.15.60:0/774933620 wait complete. 2023-11-18T20:45:14.359 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2023-11-18T20:45:13.473716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":238824,"kb_used_data":6004,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735896,"statfs":{"total":773094113280,"available":772849557504,"internally_reserved":0,"allocated":6148096,"data_stored":3234718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.160958"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-18T20:45:05.353840+0000","last_change":"2023-11-18T20:42:27.595988+0000","last_active":"2023-11-18T20:45:05.353840+0000","last_peered":"2023-11-18T20:45:05.353840+0000","last_clean":"2023-11-18T20:45:05.353840+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:45:05.353840+0000","last_undegraded":"2023-11-18T20:45:05.353840+0000","last_fullsized":"2023-11-18T20:45:05.353840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-18T20:42:26.453690+0000","last_clean_scrub_stamp":"2023-11-18T20:42:26.453690+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-19T20:48:03.227201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":849280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73999999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.496,"15min":0.48999999999999999},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.44}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.58399999999999996,"15min":0.58899999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71699999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":0.69899999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.61699999999999999}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.40000000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.45600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59799999999999998}]},{"osd":4,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.31,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72699999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73499999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.79800000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.38600000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79800000000000004,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77600000000000002,"5min":0.876,"15min":0.876},"last":0.69299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":253347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48799999999999999,"15min":0.48199999999999998},"min":{"1min":0.36599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.36499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42099999999999999}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52400000000000002,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.497}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.373,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.877,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.40899999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.746,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.40000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.80000000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.623}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.75700000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78100000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54600000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.76600000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":258912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.38600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.69299999999999995}]},{"osd":1,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.58699999999999997}]},{"osd":2,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.45900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.76600000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":2.1600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91900000000000004,"5min":1.046,"15min":1.046},"last":0.66300000000000003}]},{"osd":4,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56499999999999995}]},{"osd":6,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77000000000000002,"5min":0.875,"15min":0.875},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.432}]},{"osd":7,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95999999999999996,"5min":1.099,"15min":1.099},"last":0.45300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":258912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.503},"min":{"1min":0.36899999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55200000000000005}]},{"osd":1,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.39800000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71499999999999997,"5min":0.747,"15min":0.747},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.52200000000000002}]},{"osd":2,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.42299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.503},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.71599999999999997}]},{"osd":3,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.433,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73299999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.752,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.622},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.34699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.84599999999999997}]},{"osd":7,"last update":"Sat Nov 18 20:44:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81699999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.753}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31796,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":626688,"data_stored":258912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.441,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.317,"15min":0.315},"min":{"1min":0.26100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.25},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28999999999999998}]},{"osd":2,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.32900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53100000000000003,"5min":0.995,"15min":0.995},"last":0.40300000000000002}]},{"osd":4,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77700000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.57999999999999996}]},{"osd":6,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77000000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60799999999999998}]},{"osd":7,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81299999999999994,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":253347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33800000000000002,"15min":0.33900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.37},{"interface":"front","average":{"1min":0.375,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.28000000000000003}]},{"osd":1,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36599999999999999,"15min":0.36899999999999999},"min":{"1min":0.245,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64400000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41699999999999998,"15min":0.42299999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.59499999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.39500000000000002}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41299999999999998,"15min":0.41399999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54500000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.54600000000000004,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.433}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.44800000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.71999999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.41899999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.52600000000000002}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.68999999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83899999999999997,"5min":1.859,"15min":1.859},"last":0.503},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78600000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.626}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.76100000000000001,"5min":1.464,"15min":1.464},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.68600000000000005,"15min":0.70099999999999996},"min":{"1min":0.45100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.749,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.68799999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":610304,"data_stored":253347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.38400000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48999999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.28000000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33300000000000002},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51300000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.31900000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.57099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.47899999999999998}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":1.105,"15min":1.105},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48999999999999999}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.504}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72499999999999998,"5min":0.995,"15min":0.995},"last":0.435}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76900000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1216512,"data_stored":848661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33500000000000002,"15min":0.33600000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.374,"15min":0.375},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":0.62,"15min":0.62},"last":0.308}]},{"osd":2,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.51300000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.37,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.27200000000000002}]},{"osd":3,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.41199999999999998}]},{"osd":4,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81100000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.49099999999999999}]},{"osd":7,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.625},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95899999999999996,"5min":1.006,"15min":1.006},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:45:14.364 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-18T20:42:26.453690+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=18, tm_hour=20, tm_min=42, tm_sec=26, tm_wday=5, tm_yday=322, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=18, tm_hour=20, tm_min=45, tm_sec=9, tm_wday=5, tm_yday=322, tm_isdst=0) 2023-11-18T20:45:14.364 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-18T20:45:34.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-18T20:45:34.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.484+0000 7fbedb361640 1 Processor -- start 2023-11-18T20:45:34.485 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.484+0000 7fbedb361640 1 -- start start 2023-11-18T20:45:34.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.485+0000 7fbedb361640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbed4143490 0x7fbed40b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.485+0000 7fbedb361640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 0x7fbed40b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.485+0000 7fbedb361640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed415e2a0 con 0x7fbed415e730 2023-11-18T20:45:34.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.485+0000 7fbedb361640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed415b520 con 0x7fbed4143490 2023-11-18T20:45:34.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.485+0000 7fbedb361640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed4143870 con 0x7fbed415abf0 2023-11-18T20:45:34.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.485+0000 7fbed90fd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbed4143490 0x7fbed40b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.487 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.485+0000 7fbed98fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 0x7fbed40b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.485+0000 7fbed88fc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fbed415e730 0x7fbed415a4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:56844/0 (socket says 172.21.15.60:56844) 2023-11-18T20:45:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.485+0000 7fbed88fc640 1 -- 172.21.15.60:0/3452788307 learned_addr learned my addr 172.21.15.60:0/3452788307 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.486+0000 7fbed90fd640 1 -- 172.21.15.60:0/3452788307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 msgr2=0x7fbed40b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.486+0000 7fbed90fd640 1 --2- 172.21.15.60:0/3452788307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 0x7fbed40b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.486+0000 7fbed90fd640 1 -- 172.21.15.60:0/3452788307 >> v1:172.21.15.100:6789/0 conn(0x7fbed415e730 legacy=0x7fbed415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.486+0000 7fbed90fd640 1 -- 172.21.15.60:0/3452788307 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed415b240 con 0x7fbed4143490 2023-11-18T20:45:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.486+0000 7fbecbfff640 1 -- 172.21.15.60:0/3452788307 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368890946 0 0) 0x7fbed415e2a0 con 0x7fbed415e730 2023-11-18T20:45:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.486+0000 7fbed98fe640 1 --2- 172.21.15.60:0/3452788307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 0x7fbed40b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.486+0000 7fbed90fd640 1 --2- 172.21.15.60:0/3452788307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed4143490 0x7fbed40b0cf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbed00097a0 tx=0x7fbed002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2fa2acd803af0cd server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbecbfff640 1 -- 172.21.15.60:0/3452788307 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0031830 con 0x7fbed4143490 2023-11-18T20:45:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbecbfff640 1 -- 172.21.15.60:0/3452788307 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed0031990 con 0x7fbed4143490 2023-11-18T20:45:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbecbfff640 1 -- 172.21.15.60:0/3452788307 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0031d20 con 0x7fbed4143490 2023-11-18T20:45:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbedb361640 1 -- 172.21.15.60:0/3452788307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed4143490 msgr2=0x7fbed40b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbedb361640 1 --2- 172.21.15.60:0/3452788307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed4143490 0x7fbed40b0cf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbed00097a0 tx=0x7fbed002b8d0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbedb361640 1 -- 172.21.15.60:0/3452788307 shutdown_connections 2023-11-18T20:45:34.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbedb361640 1 --2- 172.21.15.60:0/3452788307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 0x7fbed40b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbedb361640 1 --2- 172.21.15.60:0/3452788307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed4143490 0x7fbed40b0cf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbedb361640 1 -- 172.21.15.60:0/3452788307 >> 172.21.15.60:0/3452788307 conn(0x7fbed406a4d0 msgr2=0x7fbed406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:34.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbedb361640 1 -- 172.21.15.60:0/3452788307 shutdown_connections 2023-11-18T20:45:34.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.487+0000 7fbedb361640 1 -- 172.21.15.60:0/3452788307 wait complete. 2023-11-18T20:45:34.491 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.488+0000 7fbedb361640 1 Processor -- start 2023-11-18T20:45:34.492 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbedb361640 1 -- start start 2023-11-18T20:45:34.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbedb361640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed4143490 0x7fbed4070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbedb361640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 0x7fbed40aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.494 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbedb361640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbed415e730 0x7fbed413e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbedb361640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed40ab970 con 0x7fbed415e730 2023-11-18T20:45:34.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbedb361640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed41402f0 con 0x7fbed4143490 2023-11-18T20:45:34.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbedb361640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed413f530 con 0x7fbed415abf0 2023-11-18T20:45:34.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbed98fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbed415e730 0x7fbed413e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbed98fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbed415e730 0x7fbed413e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56408/0 (socket says 172.21.15.60:56408) 2023-11-18T20:45:34.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbed98fe640 1 -- 172.21.15.60:0/1868997451 learned_addr learned my addr 172.21.15.60:0/1868997451 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:34.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.489+0000 7fbed88fc640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 0x7fbed40aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.495 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbed90fd640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed4143490 0x7fbed4070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbed98fe640 1 -- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 msgr2=0x7fbed40aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbed98fe640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 0x7fbed40aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbed98fe640 1 -- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed4143490 msgr2=0x7fbed4070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbed98fe640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed4143490 0x7fbed4070600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbed98fe640 1 -- 172.21.15.60:0/1868997451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc009a70 con 0x7fbed415e730 2023-11-18T20:45:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbed88fc640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 0x7fbed40aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-18T20:45:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbed98fe640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbed415e730 0x7fbed413e620 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00efc0 tx=0x7fbecc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbec9ffb640 1 -- 172.21.15.60:0/1868997451 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc009d70 con 0x7fbed415e730 2023-11-18T20:45:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbec9ffb640 1 -- 172.21.15.60:0/1868997451 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc00f040 con 0x7fbed415e730 2023-11-18T20:45:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbedb361640 1 -- 172.21.15.60:0/1868997451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed0009400 con 0x7fbed415e730 2023-11-18T20:45:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbec9ffb640 1 -- 172.21.15.60:0/1868997451 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc003990 con 0x7fbed415e730 2023-11-18T20:45:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.490+0000 7fbedb361640 1 -- 172.21.15.60:0/1868997451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed413faf0 con 0x7fbed415e730 2023-11-18T20:45:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.492+0000 7fbedb361640 1 -- 172.21.15.60:0/1868997451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed4069400 con 0x7fbed415e730 2023-11-18T20:45:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.492+0000 7fbec9ffb640 1 -- 172.21.15.60:0/1868997451 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbecc014480 con 0x7fbed415e730 2023-11-18T20:45:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.492+0000 7fbec9ffb640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbeb806ae40 0x7fbeb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.492+0000 7fbec9ffb640 1 -- 172.21.15.60:0/1868997451 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbecc089df0 con 0x7fbed415e730 2023-11-18T20:45:34.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.492+0000 7fbed90fd640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbeb806ae40 0x7fbeb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.493+0000 7fbed90fd640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbeb806ae40 0x7fbeb806d300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbed0036000 tx=0x7fbed003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:34.498 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.496+0000 7fbec9ffb640 1 -- 172.21.15.60:0/1868997451 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbecc0552e0 con 0x7fbed415e730 2023-11-18T20:45:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.668+0000 7fbedb361640 1 -- 172.21.15.60:0/1868997451 --> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbed415df30 con 0x7fbeb806ae40 2023-11-18T20:45:34.672 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.671+0000 7fbec9ffb640 1 -- 172.21.15.60:0/1868997451 <== mgr.4111 v2:172.21.15.100:6832/910625938 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51010 (secure 0 0 0) 0x7fbed003a420 con 0x7fbeb806ae40 2023-11-18T20:45:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:34.673 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2023-11-18T20:45:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.672+0000 7fbedb361640 1 -- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbeb806ae40 msgr2=0x7fbeb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.672+0000 7fbedb361640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbeb806ae40 0x7fbeb806d300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbed0036000 tx=0x7fbed003c040 comp rx=0 tx=0).stop 2023-11-18T20:45:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.672+0000 7fbedb361640 1 -- 172.21.15.60:0/1868997451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbed415e730 msgr2=0x7fbed413e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.672+0000 7fbedb361640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbed415e730 0x7fbed413e620 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00efc0 tx=0x7fbecc009f90 comp rx=0 tx=0).stop 2023-11-18T20:45:34.674 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.672+0000 7fbedb361640 1 -- 172.21.15.60:0/1868997451 shutdown_connections 2023-11-18T20:45:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.672+0000 7fbedb361640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7fbeb806ae40 0x7fbeb806d300 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.672+0000 7fbedb361640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbed415e730 0x7fbed413e620 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.672+0000 7fbedb361640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbed415abf0 0x7fbed40aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.672+0000 7fbedb361640 1 --2- 172.21.15.60:0/1868997451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbed4143490 0x7fbed4070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.672+0000 7fbedb361640 1 -- 172.21.15.60:0/1868997451 >> 172.21.15.60:0/1868997451 conn(0x7fbed406a4d0 msgr2=0x7fbed406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.673+0000 7fbedb361640 1 -- 172.21.15.60:0/1868997451 shutdown_connections 2023-11-18T20:45:34.675 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.673+0000 7fbedb361640 1 -- 172.21.15.60:0/1868997451 wait complete. 2023-11-18T20:45:34.684 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2023-11-18T20:45:33.478282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238856,"kb_used_data":6052,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735864,"statfs":{"total":773094113280,"available":772849524736,"internally_reserved":0,"allocated":6197248,"data_stored":3251413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002913"},"pg_stats":[{"pgid":"1.0","version":"52'119","reported_seq":204,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-18T20:45:26.158246+0000","last_change":"2023-11-18T20:45:26.158292+0000","last_active":"2023-11-18T20:45:26.158246+0000","last_peered":"2023-11-18T20:45:26.158246+0000","last_clean":"2023-11-18T20:45:26.158246+0000","last_became_active":"2023-11-18T20:42:27.595002+0000","last_became_peered":"2023-11-18T20:42:27.595002+0000","last_unstale":"2023-11-18T20:45:26.158246+0000","last_undegraded":"2023-11-18T20:45:26.158246+0000","last_fullsized":"2023-11-18T20:45:26.158246+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'119","last_scrub_stamp":"2023-11-18T20:45:26.158154+0000","last_deep_scrub":"52'119","last_deep_scrub_stamp":"2023-11-18T20:45:26.158154+0000","last_clean_scrub_stamp":"2023-11-18T20:45:26.158154+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-20T07:28:38.458723+0000","scrub_duration":0.0050048169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1220608,"data_stored":849280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.57599999999999996},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.41699999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67900000000000005,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.623}]},{"osd":1,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.48299999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71299999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.75700000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68300000000000005}]},{"osd":2,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73699999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.71099999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78100000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66100000000000003}]},{"osd":4,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.502,"15min":0.51100000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.72499999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.51100000000000001,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.45100000000000001}]},{"osd":5,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53600000000000003,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.69699999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.32400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.40600000000000003}]},{"osd":6,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73399999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60499999999999998,"15min":0.61499999999999999},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84699999999999998,"5min":0.876,"15min":0.876},"last":0.38800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":258912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.501,"15min":0.48699999999999999},"min":{"1min":0.34799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.52100000000000002,"15min":0.5},"min":{"1min":0.40000000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81200000000000006,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.81200000000000006}]},{"osd":1,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54800000000000004,"15min":0.53800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.40899999999999997,"5min":0.314,"15min":0.314},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.68000000000000005}]},{"osd":2,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.377,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.73499999999999999}]},{"osd":3,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.41199999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.755,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.755},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.38,"5min":0.376,"15min":0.376},"max":{"1min":0.77800000000000002,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69399999999999995}]},{"osd":4,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66100000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70799999999999996,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51500000000000001,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.41399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68999999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54800000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.33800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79400000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":258912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.503},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.38600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56200000000000006}]},{"osd":1,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.41699999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":1.002,"15min":1.002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53100000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.45900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.46600000000000003}]},{"osd":3,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91900000000000004,"5min":1.046,"15min":1.046},"last":0.623}]},{"osd":4,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.32400000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.372}]},{"osd":6,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77000000000000002,"5min":0.875,"15min":0.875},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45000000000000001}]},{"osd":7,"last update":"Sat Nov 18 20:44:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95999999999999996,"5min":1.099,"15min":1.099},"last":0.57899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":258912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.51200000000000001},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.72799999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67400000000000004}]},{"osd":1,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.50600000000000001,"15min":0.496},"min":{"1min":0.32400000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.626},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.45000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71499999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.65700000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.53500000000000003},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.751,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.434,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71599999999999997,"5min":0.87,"15min":0.87},"last":0.57299999999999995}]},{"osd":3,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.747,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.69999999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63800000000000001}]},{"osd":5,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52500000000000002,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68799999999999994,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55200000000000005,"15min":0.56299999999999994},"min":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45200000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70499999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51600000000000001}]},{"osd":7,"last update":"Sat Nov 18 20:45:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73899999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60199999999999998,"15min":0.61599999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.753,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.53600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31796,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":626688,"data_stored":258912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33700000000000002},"min":{"1min":0.23100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.441,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49099999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.442}]},{"osd":1,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.317,"15min":0.315},"min":{"1min":0.26100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.25900000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.57099999999999995}]},{"osd":2,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45200000000000001,"15min":0.45600000000000002},"min":{"1min":0.32900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53100000000000003,"5min":0.995,"15min":0.995},"last":0.629}]},{"osd":4,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.42899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77700000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.88200000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77000000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.752},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.83599999999999997}]},{"osd":7,"last update":"Sat Nov 18 20:44:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81299999999999994,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.872}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31792,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":626688,"data_stored":258912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:45:27 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34200000000000003},"min":{"1min":0.27800000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.38},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.36699999999999999,"15min":0.34499999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.28000000000000003}]},{"osd":1,"last update":"Sat Nov 18 20:45:27 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.39000000000000001,"15min":0.377},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":2.875,"5min":2.875,"15min":2.875},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.432,"15min":0.42799999999999999},"min":{"1min":0.27100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.45300000000000001}]},{"osd":3,"last update":"Sat Nov 18 20:45:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.41199999999999998,"15min":0.41399999999999998},"min":{"1min":0.315,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65100000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.40300000000000002,"15min":0.39100000000000001},"min":{"1min":0.27800000000000002,"5min":0.252,"15min":0.252},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.32200000000000001}]},{"osd":4,"last update":"Sat Nov 18 20:45:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.44800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.48099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.75700000000000001}]},{"osd":5,"last update":"Sat Nov 18 20:45:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.42799999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.51300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Sat Nov 18 20:45:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68799999999999994,"15min":0.70499999999999996},"min":{"1min":0.47199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.439,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Sat Nov 18 20:45:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.69299999999999995,"15min":0.70299999999999996},"min":{"1min":0.496,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.7829999999999999,"5min":2.7829999999999999,"15min":2.7829999999999999},"last":0.64800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31792,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":626688,"data_stored":258912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.34599999999999997,"15min":0.35199999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.38400000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.246,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48999999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.34999999999999998}]},{"osd":2,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33300000000000002},"min":{"1min":0.246,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51300000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.56399999999999995,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.41299999999999998}]},{"osd":3,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.55600000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.42199999999999999},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.57099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.46999999999999997}]},{"osd":4,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":1.105,"15min":1.105},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60699999999999998,"15min":0.60999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.624}]},{"osd":6,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72499999999999998,"5min":0.995,"15min":0.995},"last":0.72699999999999998}]},{"osd":7,"last update":"Sat Nov 18 20:44:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76800000000000002,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.47299999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76900000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1216512,"data_stored":848661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33500000000000002,"15min":0.33600000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.433},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.374,"15min":0.375},"min":{"1min":0.27400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":0.62,"15min":0.62},"last":0.46600000000000003}]},{"osd":2,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.34999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.51300000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.374},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.37,"15min":0.36799999999999999},"min":{"1min":0.27400000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.48899999999999999}]},{"osd":3,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.32800000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57799999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.54500000000000004}]},{"osd":4,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85199999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.622},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.59399999999999997}]},{"osd":5,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81100000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.63900000000000001}]},{"osd":6,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82699999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65400000000000003}]},{"osd":7,"last update":"Sat Nov 18 20:44:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.625},"min":{"1min":0.46200000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95899999999999996,"5min":1.006,"15min":1.006},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-18T20:45:34.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-18T20:45:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.792+0000 7f2a7ff83640 1 Processor -- start 2023-11-18T20:45:34.793 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.792+0000 7f2a7ff83640 1 -- start start 2023-11-18T20:45:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7ff83640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a780ae480 0x7f2a7806d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7ff83640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a780ae910 con 0x7f2a78072390 2023-11-18T20:45:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7ff83640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a780b2200 con 0x7f2a7815d2b0 2023-11-18T20:45:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7ff83640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7806df90 con 0x7f2a780ae480 2023-11-18T20:45:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7d51e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a780ae480 0x7f2a7806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7e520640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2a78072390 0x7f2a7806d880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56370/0 (socket says 172.21.15.60:56370) 2023-11-18T20:45:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7e520640 1 -- 172.21.15.60:0/3815322766 learned_addr learned my addr 172.21.15.60:0/3815322766 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7d51e640 1 -- 172.21.15.60:0/3815322766 >> v1:172.21.15.100:6789/0 conn(0x7f2a7815d2b0 legacy=0x7f2a78071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7d51e640 1 -- 172.21.15.60:0/3815322766 >> v1:172.21.15.60:6789/0 conn(0x7f2a78072390 legacy=0x7f2a7806d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7cd1d640 1 -- 172.21.15.60:0/3815322766 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861381989 0 0) 0x7f2a780ae910 con 0x7f2a78072390 2023-11-18T20:45:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.793+0000 7f2a7d51e640 1 -- 172.21.15.60:0/3815322766 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a780b24e0 con 0x7f2a780ae480 2023-11-18T20:45:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.794+0000 7f2a7d51e640 1 --2- 172.21.15.60:0/3815322766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ae480 0x7f2a7806d340 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0097a0 tx=0x7f2a6c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3abc75096a93861 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.794+0000 7f2a7cd1d640 1 -- 172.21.15.60:0/3815322766 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c031830 con 0x7f2a780ae480 2023-11-18T20:45:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.794+0000 7f2a7cd1d640 1 -- 172.21.15.60:0/3815322766 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c031990 con 0x7f2a780ae480 2023-11-18T20:45:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.794+0000 7f2a7cd1d640 1 -- 172.21.15.60:0/3815322766 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c031d20 con 0x7f2a780ae480 2023-11-18T20:45:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.794+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3815322766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ae480 msgr2=0x7f2a7806d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.794+0000 7f2a7ff83640 1 --2- 172.21.15.60:0/3815322766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ae480 0x7f2a7806d340 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0097a0 tx=0x7f2a6c02b8d0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.795+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3815322766 shutdown_connections 2023-11-18T20:45:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.795+0000 7f2a7ff83640 1 --2- 172.21.15.60:0/3815322766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ae480 0x7f2a7806d340 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.795+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3815322766 >> 172.21.15.60:0/3815322766 conn(0x7f2a780b4720 msgr2=0x7f2a780b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.795+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3815322766 shutdown_connections 2023-11-18T20:45:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.795+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3815322766 wait complete. 2023-11-18T20:45:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.796+0000 7f2a7ff83640 1 Processor -- start 2023-11-18T20:45:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.796+0000 7f2a7ff83640 1 -- start start 2023-11-18T20:45:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.796+0000 7f2a7ff83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78072390 0x7f2a7814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.796+0000 7f2a7ff83640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a7815d2b0 0x7f2a781f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.796+0000 7f2a7ff83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ad5d0 0x7f2a7815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.796+0000 7f2a7ff83640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a781f7270 con 0x7f2a780ad5d0 2023-11-18T20:45:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.796+0000 7f2a7ff83640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a78140be0 con 0x7f2a78072390 2023-11-18T20:45:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.796+0000 7f2a7ff83640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7814f7a0 con 0x7f2a7815d2b0 2023-11-18T20:45:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7e520640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ad5d0 0x7f2a7815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7e520640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ad5d0 0x7f2a7815fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56426/0 (socket says 172.21.15.60:56426) 2023-11-18T20:45:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7e520640 1 -- 172.21.15.60:0/3323006605 learned_addr learned my addr 172.21.15.60:0/3323006605 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7dd1f640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78072390 0x7f2a7814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7d51e640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a7815d2b0 0x7f2a781f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7e520640 1 -- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a7815d2b0 msgr2=0x7f2a781f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7e520640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a7815d2b0 0x7f2a781f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7e520640 1 -- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78072390 msgr2=0x7f2a7814f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7e520640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78072390 0x7f2a7814f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7e520640 1 -- 172.21.15.60:0/3323006605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c009400 con 0x7f2a780ad5d0 2023-11-18T20:45:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7dd1f640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78072390 0x7f2a7814f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-18T20:45:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7e520640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ad5d0 0x7f2a7815fc30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c7a0 tx=0x7f2a6800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a767fc640 1 -- 172.21.15.60:0/3323006605 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800a640 con 0x7f2a780ad5d0 2023-11-18T20:45:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a767fc640 1 -- 172.21.15.60:0/3323006605 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6800a7a0 con 0x7f2a780ad5d0 2023-11-18T20:45:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3323006605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a781402b0 con 0x7f2a780ad5d0 2023-11-18T20:45:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a767fc640 1 -- 172.21.15.60:0/3323006605 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800abd0 con 0x7f2a780ad5d0 2023-11-18T20:45:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.797+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3323006605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a78140870 con 0x7f2a780ad5d0 2023-11-18T20:45:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.798+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3323006605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7806e490 con 0x7f2a780ad5d0 2023-11-18T20:45:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.799+0000 7f2a767fc640 1 -- 172.21.15.60:0/3323006605 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a6800ad30 con 0x7f2a780ad5d0 2023-11-18T20:45:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.799+0000 7f2a767fc640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a6406ae40 0x7f2a6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.799+0000 7f2a767fc640 1 -- 172.21.15.60:0/3323006605 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2a68088ba0 con 0x7f2a780ad5d0 2023-11-18T20:45:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.799+0000 7f2a7dd1f640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a6406ae40 0x7f2a6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.800+0000 7f2a7dd1f640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a6406ae40 0x7f2a6406d300 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007fd0 tx=0x7f2a70009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.803+0000 7f2a767fc640 1 -- 172.21.15.60:0/3323006605 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a68053f50 con 0x7f2a780ad5d0 2023-11-18T20:45:34.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.978+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3323006605 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f2a78140d50 con 0x7f2a780ad5d0 2023-11-18T20:45:34.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.982+0000 7f2a767fc640 1 -- 172.21.15.60:0/3323006605 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7f2a68018020 con 0x7f2a780ad5d0 2023-11-18T20:45:34.983 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.982+0000 7f2a767fc640 1 -- 172.21.15.60:0/3323006605 <== mon.0 v2:172.21.15.60:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2a68001830 con 0x7f2a780ad5d0 2023-11-18T20:45:34.985 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a6406ae40 msgr2=0x7f2a6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a6406ae40 0x7f2a6406d300 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007fd0 tx=0x7f2a70009040 comp rx=0 tx=0).stop 2023-11-18T20:45:34.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3323006605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ad5d0 msgr2=0x7f2a7815fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:34.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ad5d0 0x7f2a7815fc30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c7a0 tx=0x7f2a6800cc60 comp rx=0 tx=0).stop 2023-11-18T20:45:34.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3323006605 shutdown_connections 2023-11-18T20:45:34.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f2a6406ae40 0x7f2a6406d300 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.986 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a780ad5d0 0x7f2a7815fc30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a7815d2b0 0x7f2a781f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 --2- 172.21.15.60:0/3323006605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a78072390 0x7f2a7814f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3323006605 >> 172.21.15.60:0/3323006605 conn(0x7f2a780b4720 msgr2=0x7f2a780af100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3323006605 shutdown_connections 2023-11-18T20:45:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:34.984+0000 7f2a7ff83640 1 -- 172.21.15.60:0/3323006605 wait complete. 2023-11-18T20:45:34.996 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-18T20:45:34.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-18T20:45:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.102+0000 7f921c1b0640 1 Processor -- start 2023-11-18T20:45:35.103 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.102+0000 7f921c1b0640 1 -- start start 2023-11-18T20:45:35.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f921c1b0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f921406bbf0 0x7f921406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:35.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f921c1b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92140adae0 con 0x7f921415d1d0 2023-11-18T20:45:35.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f921c1b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92140addc0 con 0x7f92140ad640 2023-11-18T20:45:35.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f921c1b0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92140ae0a0 con 0x7f921406bbf0 2023-11-18T20:45:35.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9219f4c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f921415d1d0 0x7f921406b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56376/0 (socket says 172.21.15.60:56376) 2023-11-18T20:45:35.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9219f4c640 1 -- 172.21.15.60:0/2393326472 learned_addr learned my addr 172.21.15.60:0/2393326472 (peer_addr_for_me v1:172.21.15.60:0/0) 2023-11-18T20:45:35.107 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f921a74d640 1 --2- 172.21.15.60:0/2393326472 >> v2:172.21.15.100:3300/0 conn(0x7f921406bbf0 0x7f921406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:35.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606569511 0 0) 0x7f92140adae0 con 0x7f921415d1d0 2023-11-18T20:45:35.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9200003520 con 0x7f921415d1d0 2023-11-18T20:45:35.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713344419 0 0) 0x7f92140addc0 con 0x7f92140ad640 2023-11-18T20:45:35.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92140adae0 con 0x7f92140ad640 2023-11-18T20:45:35.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 271668694 0 0) 0x7f9200003520 con 0x7f921415d1d0 2023-11-18T20:45:35.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 >> v2:172.21.15.100:3300/0 conn(0x7f921406bbf0 msgr2=0x7f921406d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:35.108 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 --2- 172.21.15.60:0/2393326472 >> v2:172.21.15.100:3300/0 conn(0x7f921406bbf0 0x7f921406d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 >> v1:172.21.15.100:6789/0 conn(0x7f92140ad640 legacy=0x7f921406d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92140ae380 con 0x7f921415d1d0 2023-11-18T20:45:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 76906986 0 0) 0x7f9210001670 con 0x7f921415d1d0 2023-11-18T20:45:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 >> v1:172.21.15.60:6789/0 conn(0x7f921415d1d0 legacy=0x7f921406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9200003b00 0x7f9200003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:35.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92000047c0 0x7f9200004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:35.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9200005180 0x7f920000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:35.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92140ae0a0 con 0x7f9200003b00 2023-11-18T20:45:35.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920000c000 con 0x7f9200005180 2023-11-18T20:45:35.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.103+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920000c1c0 con 0x7f92000047c0 2023-11-18T20:45:35.110 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.104+0000 7f9219f4c640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9200003b00 0x7f9200003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.104+0000 7f921974b640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92000047c0 0x7f9200004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.104+0000 7f921a74d640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9200005180 0x7f920000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.104+0000 7f9219f4c640 1 -- 172.21.15.60:0/2393326472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92000047c0 msgr2=0x7f9200004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.104+0000 7f9219f4c640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92000047c0 0x7f9200004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.104+0000 7f9219f4c640 1 -- 172.21.15.60:0/2393326472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9200005180 msgr2=0x7f920000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-18T20:45:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.104+0000 7f9219f4c640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9200005180 0x7f920000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.104+0000 7f9219f4c640 1 -- 172.21.15.60:0/2393326472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200003520 con 0x7f9200003b00 2023-11-18T20:45:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.104+0000 7f921a74d640 1 -- 172.21.15.60:0/2393326472 reap_dead start 2023-11-18T20:45:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f9219f4c640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9200003b00 0x7f9200003f00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9210007fd0 tx=0x7f92000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87116dbc76338414 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210005ad0 con 0x7f9200003b00 2023-11-18T20:45:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9210005c50 con 0x7f9200003b00 2023-11-18T20:45:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f9218f4a640 1 -- 172.21.15.60:0/2393326472 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921002c850 con 0x7f9200003b00 2023-11-18T20:45:35.112 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f921c1b0640 1 -- 172.21.15.60:0/2393326472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9200003b00 msgr2=0x7f9200003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:35.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f921c1b0640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9200003b00 0x7f9200003f00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9210007fd0 tx=0x7f92000057f0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f921c1b0640 1 -- 172.21.15.60:0/2393326472 shutdown_connections 2023-11-18T20:45:35.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f921c1b0640 1 --2- 172.21.15.60:0/2393326472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9200003b00 0x7f9200003f00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f921c1b0640 1 -- 172.21.15.60:0/2393326472 >> 172.21.15.60:0/2393326472 conn(0x7f9214070e70 msgr2=0x7f9214071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:35.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f921c1b0640 1 -- 172.21.15.60:0/2393326472 shutdown_connections 2023-11-18T20:45:35.113 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.105+0000 7f921c1b0640 1 -- 172.21.15.60:0/2393326472 wait complete. 2023-11-18T20:45:35.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921c1b0640 1 Processor -- start 2023-11-18T20:45:35.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921c1b0640 1 -- start start 2023-11-18T20:45:35.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921c1b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9200003b00 0x7f9214159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:35.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921c1b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f921415a380 0x7f921415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:35.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921c1b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9214143f40 0x7f92141f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:35.116 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921c1b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921415d9e0 con 0x7f9214143f40 2023-11-18T20:45:35.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921c1b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921415dcc0 con 0x7f9200003b00 2023-11-18T20:45:35.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921c1b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921415dfa0 con 0x7f921415a380 2023-11-18T20:45:35.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921a74d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9214143f40 0x7f92141f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:35.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921a74d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9214143f40 0x7f92141f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56438/0 (socket says 172.21.15.60:56438) 2023-11-18T20:45:35.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.107+0000 7f921a74d640 1 -- 172.21.15.60:0/891179226 learned_addr learned my addr 172.21.15.60:0/891179226 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-18T20:45:35.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f921974b640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f921415a380 0x7f921415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:35.117 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f9219f4c640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9200003b00 0x7f9214159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:35.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f921a74d640 1 -- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f921415a380 msgr2=0x7f921415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:35.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f921a74d640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f921415a380 0x7f921415ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f921a74d640 1 -- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9200003b00 msgr2=0x7f9214159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:35.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f921a74d640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9200003b00 0x7f9214159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f921a74d640 1 -- 172.21.15.60:0/891179226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921415e280 con 0x7f9214143f40 2023-11-18T20:45:35.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f921a74d640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9214143f40 0x7f92141f2e30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f92140acbc0 tx=0x7f920c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:35.118 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f920a7fc640 1 -- 172.21.15.60:0/891179226 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c00e040 con 0x7f9214143f40 2023-11-18T20:45:35.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f920a7fc640 1 -- 172.21.15.60:0/891179226 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f920c00f040 con 0x7f9214143f40 2023-11-18T20:45:35.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f921c1b0640 1 -- 172.21.15.60:0/891179226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921415e560 con 0x7f9214143f40 2023-11-18T20:45:35.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f920a7fc640 1 -- 172.21.15.60:0/891179226 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c015430 con 0x7f9214143f40 2023-11-18T20:45:35.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.108+0000 7f921c1b0640 1 -- 172.21.15.60:0/891179226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921415ff00 con 0x7f9214143f40 2023-11-18T20:45:35.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.109+0000 7f921c1b0640 1 -- 172.21.15.60:0/891179226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9214069400 con 0x7f9214143f40 2023-11-18T20:45:35.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.109+0000 7f920a7fc640 1 -- 172.21.15.60:0/891179226 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f920c015590 con 0x7f9214143f40 2023-11-18T20:45:35.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.110+0000 7f920a7fc640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f91f806ae40 0x7f91f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-18T20:45:35.119 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.110+0000 7f920a7fc640 1 -- 172.21.15.60:0/891179226 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f920c089c30 con 0x7f9214143f40 2023-11-18T20:45:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.110+0000 7f9219f4c640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f91f806ae40 0x7f91f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-18T20:45:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.111+0000 7f9219f4c640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f91f806ae40 0x7f91f806d300 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9210034000 tx=0x7f9210007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-18T20:45:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.114+0000 7f920a7fc640 1 -- 172.21.15.60:0/891179226 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f920c054f50 con 0x7f9214143f40 2023-11-18T20:45:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.327+0000 7f921c1b0640 1 -- 172.21.15.60:0/891179226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9214153950 con 0x7f9214143f40 2023-11-18T20:45:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.327+0000 7f920a7fc640 1 -- 172.21.15.60:0/891179226 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f920c058570 con 0x7f9214143f40 2023-11-18T20:45:35.329 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-18T20:45:35.329 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1578336464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1578336464},{"type":"v1","addr":"172.21.15.100:6836","nonce":1578336464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/88984334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":88984334},{"type":"v1","addr":"172.21.15.100:6837","nonce":88984334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3777286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3777286992},{"type":"v1","addr":"172.21.15.60:6833","nonce":3777286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":6336,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3987765432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3987765432},{"type":"v1","addr":"172.21.15.60:6835","nonce":3987765432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[]} 2023-11-18T20:45:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2023-11-18T20:45:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 -- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f91f806ae40 msgr2=0x7f91f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f91f806ae40 0x7f91f806d300 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9210034000 tx=0x7f9210007000 comp rx=0 tx=0).stop 2023-11-18T20:45:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 -- 172.21.15.60:0/891179226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9214143f40 msgr2=0x7f92141f2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-18T20:45:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9214143f40 0x7f92141f2e30 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f92140acbc0 tx=0x7f920c00f580 comp rx=0 tx=0).stop 2023-11-18T20:45:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 -- 172.21.15.60:0/891179226 shutdown_connections 2023-11-18T20:45:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:6832/910625938,v1:172.21.15.100:6833/910625938] conn(0x7f91f806ae40 0x7f91f806d300 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9214143f40 0x7f92141f2e30 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f921415a380 0x7f921415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 --2- 172.21.15.60:0/891179226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9200003b00 0x7f9214159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-18T20:45:35.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 -- 172.21.15.60:0/891179226 >> 172.21.15.60:0/891179226 conn(0x7f9214070e70 msgr2=0x7f92140b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-18T20:45:35.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 -- 172.21.15.60:0/891179226 shutdown_connections 2023-11-18T20:45:35.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-18T20:45:35.329+0000 7f921c1b0640 1 -- 172.21.15.60:0/891179226 wait complete. 2023-11-18T20:45:35.341 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-18T20:45:35.341 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-18T20:45:35.342 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-18T20:45:35.342 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.351 INFO:tasks.ceph.mds.a:Stopped 2023-11-18T20:45:35.351 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-18T20:45:35.351 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.400 INFO:tasks.ceph.mds.c:Stopped 2023-11-18T20:45:35.400 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-18T20:45:35.400 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.412 INFO:tasks.ceph.mds.b:Stopped 2023-11-18T20:45:35.412 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-18T20:45:35.412 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.464 INFO:tasks.ceph.mds.d:Stopped 2023-11-18T20:45:35.464 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-18T20:45:35.464 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-18T20:45:35.464 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.492 INFO:tasks.ceph.osd.0:Stopped 2023-11-18T20:45:35.492 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-18T20:45:35.492 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.560 INFO:tasks.ceph.osd.1:Stopped 2023-11-18T20:45:35.560 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-18T20:45:35.560 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.629 INFO:tasks.ceph.osd.2:Stopped 2023-11-18T20:45:35.629 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-18T20:45:35.629 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.696 INFO:tasks.ceph.osd.3:Stopped 2023-11-18T20:45:35.696 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-18T20:45:35.696 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.728 INFO:tasks.ceph.osd.4:Stopped 2023-11-18T20:45:35.729 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-18T20:45:35.729 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.798 INFO:tasks.ceph.osd.5:Stopped 2023-11-18T20:45:35.798 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-18T20:45:35.798 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.869 INFO:tasks.ceph.osd.6:Stopped 2023-11-18T20:45:35.869 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-18T20:45:35.870 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.944 INFO:tasks.ceph.osd.7:Stopped 2023-11-18T20:45:35.944 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-18T20:45:35.944 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-18T20:45:35.944 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:35.969 INFO:tasks.ceph.mgr.y:Stopped 2023-11-18T20:45:35.969 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-18T20:45:35.969 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:36.013 INFO:tasks.ceph.mgr.x:Stopped 2023-11-18T20:45:36.013 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-18T20:45:36.013 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-18T20:45:36.013 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:36.025 INFO:tasks.ceph.mon.a:Stopped 2023-11-18T20:45:36.025 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-18T20:45:36.025 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:36.066 INFO:tasks.ceph.mon.b:Stopped 2023-11-18T20:45:36.066 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-18T20:45:36.066 INFO:teuthology.orchestra.run:waiting for 300 2023-11-18T20:45:36.119 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-11-18T20:45:36.120 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-11-18T20:45:36.120 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-11-18T20:45:36.120 INFO:tasks.ceph.mon.c:Stopped 2023-11-18T20:45:36.120 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-18T20:45:36.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-18T20:45:36.156 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:45:36.157 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-18T20:45:36.412 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:45:36.413 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-18T20:45:36.478 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:45:36.479 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-18T20:45:36.537 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:45:36.537 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-18T20:45:36.594 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:45:36.595 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-18T20:45:36.748 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:45:36.748 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-18T20:45:36.801 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:45:36.802 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-18T20:45:36.858 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:45:36.858 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-18T20:45:36.918 INFO:tasks.ceph:Archiving mon data... 2023-11-18T20:45:36.919 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690/data/mon.a.tgz 2023-11-18T20:45:36.919 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2023-11-18T20:45:36.934 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.7w3Nvni5pC 2023-11-18T20:45:36.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.7w3Nvni5pC 2023-11-18T20:45:37.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.7w3Nvni5pC 2023-11-18T20:45:37.170 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.7w3Nvni5pC is 336KB 2023-11-18T20:45:37.238 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.7w3Nvni5pC 2023-11-18T20:45:37.252 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690/data/mon.b.tgz 2023-11-18T20:45:37.252 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2023-11-18T20:45:37.269 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.I8qUIwppk1 2023-11-18T20:45:37.269 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.I8qUIwppk1 2023-11-18T20:45:37.433 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.I8qUIwppk1 2023-11-18T20:45:37.518 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.I8qUIwppk1 is 446KB 2023-11-18T20:45:37.614 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.I8qUIwppk1 2023-11-18T20:45:37.628 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690/data/mon.c.tgz 2023-11-18T20:45:37.629 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2023-11-18T20:45:37.683 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.PlbrsEiAMC 2023-11-18T20:45:37.683 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.PlbrsEiAMC 2023-11-18T20:45:37.833 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.PlbrsEiAMC 2023-11-18T20:45:37.917 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.PlbrsEiAMC is 446KB 2023-11-18T20:45:38.014 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.PlbrsEiAMC 2023-11-18T20:45:38.028 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-18T20:45:38.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-18T20:45:38.031 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-18T20:45:38.094 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-18T20:45:38.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-18T20:45:38.107 DEBUG:teuthology.orchestra.run.smithi100:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-18T20:45:38.142 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-18T20:45:38.163 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-18T20:45:38.168 INFO:tasks.ceph:Archiving crash dumps... 2023-11-18T20:45:38.169 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690/remote/smithi060/crash 2023-11-18T20:45:38.170 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-18T20:45:38.221 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690/remote/smithi100/crash 2023-11-18T20:45:38.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-18T20:45:38.248 INFO:tasks.ceph:Shutting down logrotate 2023-11-18T20:45:38.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:45:38.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-18T20:45:38.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-18T20:45:38.377 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-18T20:45:38.403 INFO:tasks.ceph:Compressing logs... 2023-11-18T20:45:38.403 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-18T20:45:38.419 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-18T20:45:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97333.log 2023-11-18T20:45:38.495 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-18T20:45:38.495 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87362.log 2023-11-18T20:45:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.93815.log 2023-11-18T20:45:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.87362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87201.log 2023-11-18T20:45:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.93815.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87362.log.gz 2023-11-18T20:45:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.94259.log 2023-11-18T20:45:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.87201.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-18T20:45:38.496 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87201.log.gz 2023-11-18T20:45:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.3.94259.log: 89.6% -- replaced with /var/log/ceph/ceph-client.1.93815.log.gz 2023-11-18T20:45:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-18T20:45:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87322.log 2023-11-18T20:45:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-18T20:45:38.497 INFO:teuthology.orchestra.run.smithi100.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.3.94259.log.gz 2023-11-18T20:45:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.87322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87242.log 2023-11-18T20:45:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.4.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87322.log.gz 2023-11-18T20:45:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87121.log 2023-11-18T20:45:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.87242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87402.log 2023-11-18T20:45:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.87121.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87242.log.gz 2023-11-18T20:45:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-18T20:45:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.87402.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87121.log.gz 2023-11-18T20:45:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-18T20:45:38.498 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87402.log.gz 2023-11-18T20:45:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-18T20:45:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose 93.6% -- /var/log/ceph/ceph.log 2023-11-18T20:45:38.499 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-18T20:45:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-18T20:45:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87282.log 2023-11-18T20:45:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.b.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-18T20:45:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-18T20:45:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.87282.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87282.log.gz 2023-11-18T20:45:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87161.log 2023-11-18T20:45:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.d.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-18T20:45:38.500 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-18T20:45:38.500 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.87161.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87161.log.gz 2023-11-18T20:45:38.500 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.c.log: 91.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-18T20:45:38.535 INFO:teuthology.orchestra.run.smithi100.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-18T20:45:38.601 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105873.log 2023-11-18T20:45:38.602 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93902.log 2023-11-18T20:45:38.602 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97333.log: /var/log/ceph/ceph-client.admin.105873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102909.log 2023-11-18T20:45:38.602 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97333.log.gz 2023-11-18T20:45:38.602 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105873.log.gz 2023-11-18T20:45:38.602 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102851.log 2023-11-18T20:45:38.602 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102909.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93902.log.gz 2023-11-18T20:45:38.603 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92779.log 2023-11-18T20:45:38.603 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102909.log.gz 2023-11-18T20:45:38.603 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103706.log 2023-11-18T20:45:38.603 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.92779.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102851.log.gz 2023-11-18T20:45:38.603 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92821.log 2023-11-18T20:45:38.603 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103706.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92779.log.gz 2023-11-18T20:45:38.603 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-18T20:45:38.603 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103706.log.gz 2023-11-18T20:45:38.603 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.92821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93570.log 2023-11-18T20:45:38.604 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.92821.log.gz 2023-11-18T20:45:38.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109743.log 2023-11-18T20:45:38.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104745.log 2023-11-18T20:45:38.604 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3%/var/log/ceph/ceph-client.admin.109743.log: -- replaced with /var/log/ceph/ceph-client.admin.93570.log.gz 2023-11-18T20:45:38.604 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98915.log 2023-11-18T20:45:38.604 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109743.log.gz 2023-11-18T20:45:38.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110809.log 2023-11-18T20:45:38.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97154.log 2023-11-18T20:45:38.605 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104745.log.gz 2023-11-18T20:45:38.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110809.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.98915.log.gz 2023-11-18T20:45:38.605 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107180.log 2023-11-18T20:45:38.605 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110809.log.gz 2023-11-18T20:45:38.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94234.log 2023-11-18T20:45:38.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107180.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97154.log.gz -5 2023-11-18T20:45:38.605 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106295.log 2023-11-18T20:45:38.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94234.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107180.log.gz 2023-11-18T20:45:38.606 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105607.log 2023-11-18T20:45:38.606 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94234.log.gz 2023-11-18T20:45:38.606 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99125.log 2023-11-18T20:45:38.606 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103119.log 2023-11-18T20:45:38.606 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99125.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105607.log.gz 2023-11-18T20:45:38.606 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.106295.log.gz 2023-11-18T20:45:38.606 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94235.log 2023-11-18T20:45:38.606 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99125.log.gz 2023-11-18T20:45:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96608.log 2023-11-18T20:45:38.607 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103119.log.gz 2023-11-18T20:45:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94775.log 2023-11-18T20:45:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96608.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94235.log.gz 2023-11-18T20:45:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107342.log 2023-11-18T20:45:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94775.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96608.log.gz 2023-11-18T20:45:38.607 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98117.log 2023-11-18T20:45:38.607 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94775.log.gz 2023-11-18T20:45:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94649.log 2023-11-18T20:45:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98117.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.107342.log.gz --verbose 2023-11-18T20:45:38.608 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.107832.log 2023-11-18T20:45:38.608 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98117.log.gz 2023-11-18T20:45:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96234.log 2023-11-18T20:45:38.608 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94649.log.gz 2023-11-18T20:45:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96345.log 2023-11-18T20:45:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96234.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107832.log.gz 2023-11-18T20:45:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101970.log 2023-11-18T20:45:38.609 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96234.log.gz 2023-11-18T20:45:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96524.log 2023-11-18T20:45:38.609 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96345.log.gz 2023-11-18T20:45:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97823.log 2023-11-18T20:45:38.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96524.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101970.log.gz 2023-11-18T20:45:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103499.log 2023-11-18T20:45:38.610 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96524.log.gz 2023-11-18T20:45:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100517.log 2023-11-18T20:45:38.610 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97823.log.gz 2023-11-18T20:45:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.103499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98075.log 2023-11-18T20:45:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96902.log 2023-11-18T20:45:38.610 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98075.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100517.log.gz 2023-11-18T20:45:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98999.log 2023-11-18T20:45:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100603.log 2023-11-18T20:45:38.611 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%/var/log/ceph/ceph-client.admin.96902.log: -- replaced with /var/log/ceph/ceph-client.admin.98075.log.gz 2023-11-18T20:45:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98999.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96902.log.gz 2023-11-18T20:45:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93557.log 2023-11-18T20:45:38.611 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98999.log.gz 2023-11-18T20:45:38.611 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.103499.log.gz 2023-11-18T20:45:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.100603.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.109304.log 2023-11-18T20:45:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97991.log 2023-11-18T20:45:38.612 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100603.log.gz 2023-11-18T20:45:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99873.log 2023-11-18T20:45:38.612 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93557.log.gz 2023-11-18T20:45:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97991.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109304.log.gz 2023-11-18T20:45:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109659.log 2023-11-18T20:45:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99873.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97991.log.gz 2023-11-18T20:45:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107601.log 2023-11-18T20:45:38.613 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99873.log.gz 2023-11-18T20:45:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103197.log 2023-11-18T20:45:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.107601.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109659.log.gz 2023-11-18T20:45:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109617.log 2023-11-18T20:45:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103874.log 2023-11-18T20:45:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109617.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103197.log.gz 2023-11-18T20:45:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110683.log 2023-11-18T20:45:38.614 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109617.log.gz 2023-11-18T20:45:38.614 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94901.log 2023-11-18T20:45:38.614 INFO:teuthology.orchestra.run.smithi060.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.0.107601.log.gz 2023-11-18T20:45:38.614 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103874.log.gz 2023-11-18T20:45:38.614 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108226.log 2023-11-18T20:45:38.614 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110683.log.gz 2023-11-18T20:45:38.614 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108058.log 2023-11-18T20:45:38.615 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94901.log: /var/log/ceph/ceph-client.admin.108226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109897.log 2023-11-18T20:45:38.615 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94901.log.gz 2023-11-18T20:45:38.615 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108226.log.gz 2023-11-18T20:45:38.615 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99957.log 2023-11-18T20:45:38.615 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.108058.log: /var/log/ceph/ceph-client.admin.109897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102951.log 2023-11-18T20:45:38.615 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108058.log.gz 2023-11-18T20:45:38.615 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99957.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109897.log.gz 2023-11-18T20:45:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99629.log 2023-11-18T20:45:38.616 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99957.log.gz 2023-11-18T20:45:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109220.log 2023-11-18T20:45:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99629.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.102951.log.gz 2023-11-18T20:45:38.616 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.104871.log 2023-11-18T20:45:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99461.log 2023-11-18T20:45:38.616 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99629.log.gz 2023-11-18T20:45:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104871.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109220.log.gz 2023-11-18T20:45:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93569.log 2023-11-18T20:45:38.617 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104871.log.gz 2023-11-18T20:45:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100043.log 2023-11-18T20:45:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102096.log 87.8% 2023-11-18T20:45:38.617 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99461.log.gz 2023-11-18T20:45:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100043.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93569.log.gz 2023-11-18T20:45:38.617 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104451.log 2023-11-18T20:45:38.618 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100043.log.gz 2023-11-18T20:45:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93900.log 2023-11-18T20:45:38.618 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3%/var/log/ceph/ceph-client.admin.104451.log: -- replaced with /var/log/ceph/ceph-client.admin.102096.log.gz 2023-11-18T20:45:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105915.log 2023-11-18T20:45:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93900.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104451.log.gz 2023-11-18T20:45:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109514.log 2023-11-18T20:45:38.618 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93900.log.gz 2023-11-18T20:45:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104118.log 2023-11-18T20:45:38.619 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105915.log.gz 2023-11-18T20:45:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97949.log 2023-11-18T20:45:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103077.log 2023-11-18T20:45:38.619 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109514.log.gz 2023-11-18T20:45:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97949.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104118.log.gz 2023-11-18T20:45:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97459.log 2023-11-18T20:45:38.619 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97949.log.gz 2023-11-18T20:45:38.619 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101618.log 2023-11-18T20:45:38.620 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103077.log.gz 2023-11-18T20:45:38.620 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.97459.log: -- /var/log/ceph/ceph-client.admin.101660.log 2023-11-18T20:45:38.620 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101618.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97459.log.gz 2023-11-18T20:45:38.620 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104661.log 2023-11-18T20:45:38.620 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101618.log.gz 2023-11-18T20:45:38.620 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96776.log 2023-11-18T20:45:38.621 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101660.log.gz 2023-11-18T20:45:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87572.log 2023-11-18T20:45:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97238.log 2023-11-18T20:45:38.621 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104661.log.gz 2023-11-18T20:45:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87572.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96776.log.gz 2023-11-18T20:45:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105823.log 2023-11-18T20:45:38.621 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87572.log.gz 2023-11-18T20:45:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109939.log 2023-11-18T20:45:38.621 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110891.log 2023-11-18T20:45:38.622 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97238.log.gz 2023-11-18T20:45:38.622 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105823.log.gz 2023-11-18T20:45:38.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104829.log 2023-11-18T20:45:38.622 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109939.log.gz 2023-11-18T20:45:38.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105649.log 2023-11-18T20:45:38.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98501.log 2023-11-18T20:45:38.622 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110891.log.gz 2023-11-18T20:45:38.623 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104829.log.gz 2023-11-18T20:45:38.623 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109701.log 2023-11-18T20:45:38.623 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104703.log 2023-11-18T20:45:38.623 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98501.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105649.log.gz 2023-11-18T20:45:38.623 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109701.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.98501.log.gz 2023-11-18T20:45:38.623 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.98275.log 2023-11-18T20:45:38.623 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109701.log.gz 2023-11-18T20:45:38.623 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96692.log 2023-11-18T20:45:38.624 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98275.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.104703.log.gz 2023-11-18T20:45:38.624 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.105565.log 2023-11-18T20:45:38.624 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98275.log.gz 2023-11-18T20:45:38.624 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92737.log 2023-11-18T20:45:38.624 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96692.log.gz 2023-11-18T20:45:38.624 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110347.log 2023-11-18T20:45:38.624 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.92737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93305.log 2023-11-18T20:45:38.624 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105565.log.gz 2023-11-18T20:45:38.625 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110347.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92737.log.gz 2023-11-18T20:45:38.625 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100253.log 2023-11-18T20:45:38.625 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110347.log.gz 2023-11-18T20:45:38.625 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94943.log 2023-11-18T20:45:38.625 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102641.log 89.1% 2023-11-18T20:45:38.625 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93305.log.gz 2023-11-18T20:45:38.625 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94943.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100253.log.gz 2023-11-18T20:45:38.626 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92310.log 2023-11-18T20:45:38.626 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94943.log.gz 2023-11-18T20:45:38.626 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102641.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-18T20:45:38.626 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.92310.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102641.log.gz 2023-11-18T20:45:38.626 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97417.log 2023-11-18T20:45:38.626 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose 88.3% -- /var/log/ceph/ceph-client.admin.110084.log -- replaced with /var/log/ceph/ceph-client.admin.92310.log.gz 2023-11-18T20:45:38.626 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.626 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105957.log 2023-11-18T20:45:38.627 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97417.log.gz 2023-11-18T20:45:38.627 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102054.log 2023-11-18T20:45:38.627 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110084.log.gz 2023-11-18T20:45:38.627 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98159.log 2023-11-18T20:45:38.627 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105957.log: /var/log/ceph/ceph-client.admin.102054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106912.log 2023-11-18T20:45:38.627 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105957.log.gz 2023-11-18T20:45:38.627 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98159.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102054.log.gz 2023-11-18T20:45:38.627 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100855.log 2023-11-18T20:45:38.628 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-18T20:45:38.628 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98159.log.gz 2023-11-18T20:45:38.628 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103748.log 2023-11-18T20:45:38.628 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106912.log: /var/log/ceph/ceph-client.admin.100855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101928.log 2023-11-18T20:45:38.628 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106912.log.gz 2023-11-18T20:45:38.628 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96192.log 2023-11-18T20:45:38.628 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103748.log.gz 2023-11-18T20:45:38.628 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101928.log: gzip 91.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.100855.log.gz 2023-11-18T20:45:38.629 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99293.log 2023-11-18T20:45:38.629 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108858.log 2023-11-18T20:45:38.629 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96192.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101928.log.gz 2023-11-18T20:45:38.629 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99293.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96192.log.gz 2023-11-18T20:45:38.629 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106085.log 2023-11-18T20:45:38.629 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99293.log.gz 2023-11-18T20:45:38.629 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.108858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94691.log 2023-11-18T20:45:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99671.log 2023-11-18T20:45:38.630 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108858.log.gz 2023-11-18T20:45:38.630 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106085.log.gz 2023-11-18T20:45:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93263.log 2023-11-18T20:45:38.630 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94691.log.gz 2023-11-18T20:45:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103974.log 2023-11-18T20:45:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107038.log 2023-11-18T20:45:38.631 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99671.log.gz 2023-11-18T20:45:38.631 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93263.log.gz 2023-11-18T20:45:38.631 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105226.log 2023-11-18T20:45:38.631 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103974.log.gz 2023-11-18T20:45:38.631 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96481.log 2023-11-18T20:45:38.631 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104974.log 2023-11-18T20:45:38.631 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107038.log.gz 2023-11-18T20:45:38.631 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96481.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105226.log.gz 2023-11-18T20:45:38.632 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109981.log 2023-11-18T20:45:38.632 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96481.log.gz 2023-11-18T20:45:38.632 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92993.log 2023-11-18T20:45:38.632 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100433.log 2023-11-18T20:45:38.632 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104974.log.gz 2023-11-18T20:45:38.632 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109981.log.gz 2023-11-18T20:45:38.632 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.92993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110641.log 2023-11-18T20:45:38.632 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.92993.log.gz 2023-11-18T20:45:38.633 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93561.log 2023-11-18T20:45:38.633 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110641.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.100433.log.gz 2023-11-18T20:45:38.633 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101560.log 2023-11-18T20:45:38.633 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93561.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110641.log.gz 2023-11-18T20:45:38.633 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108737.log 2023-11-18T20:45:38.633 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93561.log.gz 2023-11-18T20:45:38.633 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106001.log 2023-11-18T20:45:38.633 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.108737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104787.log 2023-11-18T20:45:38.634 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101560.log.gz 2023-11-18T20:45:38.634 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108737.log.gz 2023-11-18T20:45:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100771.log 2023-11-18T20:45:38.634 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106001.log.gz 2023-11-18T20:45:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109094.log 2023-11-18T20:45:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110725.log 2023-11-18T20:45:38.634 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104787.log.gz 2023-11-18T20:45:38.635 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100771.log.gz 2023-11-18T20:45:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110849.log 2023-11-18T20:45:38.635 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109094.log.gz 2023-11-18T20:45:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101355.log 2023-11-18T20:45:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110849.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110725.log.gz -5 2023-11-18T20:45:38.635 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96944.log 2023-11-18T20:45:38.635 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110849.log.gz 2023-11-18T20:45:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107264.log 2023-11-18T20:45:38.635 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101355.log.gz 2023-11-18T20:45:38.636 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105016.log 2023-11-18T20:45:38.636 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107264.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.96944.log.gz 2023-11-18T20:45:38.636 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107874.log 2023-11-18T20:45:38.636 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105016.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107264.log.gz 2023-11-18T20:45:38.636 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102725.log 2023-11-18T20:45:38.636 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105016.log.gz 2023-11-18T20:45:38.636 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93035.log 2023-11-18T20:45:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102725.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.107874.log.gz -5 2023-11-18T20:45:38.637 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.108984.log 2023-11-18T20:45:38.637 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102725.log.gz 2023-11-18T20:45:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93035.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-18T20:45:38.637 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93035.log.gz 2023-11-18T20:45:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94481.log 2023-11-18T20:45:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.108984.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104409.log 2023-11-18T20:45:38.637 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108984.log.gz 2023-11-18T20:45:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95069.log 2023-11-18T20:45:38.638 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94481.log.gz 2023-11-18T20:45:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94859.log 2023-11-18T20:45:38.638 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1%/var/log/ceph/ceph-client.admin.95069.log: -- replaced with /var/log/ceph/ceph-client.admin.104409.log.gz 2023-11-18T20:45:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98585.log 2023-11-18T20:45:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98831.log 2023-11-18T20:45:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94859.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.95069.log.gz 2023-11-18T20:45:38.639 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97865.log 2023-11-18T20:45:38.639 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94859.log.gz 2023-11-18T20:45:38.639 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98585.log.gz 2023-11-18T20:45:38.639 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100687.log 2023-11-18T20:45:38.639 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102433.log 2023-11-18T20:45:38.639 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98831.log.gz 2023-11-18T20:45:38.639 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97865.log: /var/log/ceph/ceph-client.admin.100687.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97865.log.gz 2023-11-18T20:45:38.639 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106169.log 2023-11-18T20:45:38.640 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100687.log.gz 2023-11-18T20:45:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99041.log 2023-11-18T20:45:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.102433.log: /var/log/ceph/ceph-client.admin.106169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98957.log 2023-11-18T20:45:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99041.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106169.log.gz 2023-11-18T20:45:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108900.log 2023-11-18T20:45:38.640 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99041.log.gz 2023-11-18T20:45:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94733.log 2023-11-18T20:45:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98957.log: /var/log/ceph/ceph-client.admin.108900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101828.log 2023-11-18T20:45:38.641 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98957.log.gz 2023-11-18T20:45:38.641 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94733.log: gzip 91.1% -5 -- replaced with /var/log/ceph/ceph-client.0.102433.log.gz --verbose 2023-11-18T20:45:38.641 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.105735.log 2023-11-18T20:45:38.641 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108900.log.gz 2023-11-18T20:45:38.641 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94733.log.gz 2023-11-18T20:45:38.641 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99209.log 2023-11-18T20:45:38.641 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101828.log.gz 2023-11-18T20:45:38.641 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96986.log 2023-11-18T20:45:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105735.log: /var/log/ceph/ceph-client.admin.99209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110431.log 2023-11-18T20:45:38.642 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105735.log.gz 2023-11-18T20:45:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104619.log 2023-11-18T20:45:38.642 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99209.log.gz 2023-11-18T20:45:38.642 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96986.log.gz/var/log/ceph/ceph-client.admin.110431.log: 2023-11-18T20:45:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81127.log 2023-11-18T20:45:38.642 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.110431.log.gz 2023-11-18T20:45:38.642 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.109346.log 2023-11-18T20:45:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104619.log: /var/log/ceph/ceph.tmp-client.admin.81127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100921.log 2023-11-18T20:45:38.643 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81127.log.gz 2023-11-18T20:45:38.643 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104619.log.gz 2023-11-18T20:45:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105405.log 2023-11-18T20:45:38.643 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109346.log.gz 2023-11-18T20:45:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93905.log 2023-11-18T20:45:38.643 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100921.log.gz 2023-11-18T20:45:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97028.log 2023-11-18T20:45:38.644 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105405.log: /var/log/ceph/ceph-client.admin.93905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102993.log 2023-11-18T20:45:38.644 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105405.log.gz 2023-11-18T20:45:38.644 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97028.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.100127.log 2023-11-18T20:45:38.644 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93905.log.gz 2023-11-18T20:45:38.644 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97028.log.gz 2023-11-18T20:45:38.644 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92567.log 2023-11-18T20:45:38.644 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102993.log.gz 2023-11-18T20:45:38.645 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94241.log 2023-11-18T20:45:38.645 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100127.log: /var/log/ceph/ceph-client.admin.92567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96437.log 2023-11-18T20:45:38.645 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100127.log.gz 2023-11-18T20:45:38.645 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94241.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92567.log.gz 2023-11-18T20:45:38.645 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105268.log 2023-11-18T20:45:38.645 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94241.log.gz 2023-11-18T20:45:38.645 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105142.log 2023-11-18T20:45:38.646 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96437.log: /var/log/ceph/ceph-client.admin.105268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93556.log 2023-11-18T20:45:38.646 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96437.log.gz 2023-11-18T20:45:38.646 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105142.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105268.log.gz 2023-11-18T20:45:38.646 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98627.log 2023-11-18T20:45:38.646 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105142.log.gz 2023-11-18T20:45:38.646 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.93556.log: -- /var/log/ceph/ceph-client.admin.100085.log 2023-11-18T20:45:38.646 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101399.log 2023-11-18T20:45:38.646 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93556.log.gz 2023-11-18T20:45:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98543.log 2023-11-18T20:45:38.647 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98627.log.gz 2023-11-18T20:45:38.647 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100085.log.gz 2023-11-18T20:45:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94238.log 2023-11-18T20:45:38.647 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101399.log.gz 2023-11-18T20:45:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93137.log 2023-11-18T20:45:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98543.log: /var/log/ceph/ceph-client.admin.94238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94243.log 2023-11-18T20:45:38.648 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98543.log.gz 2023-11-18T20:45:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93137.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94238.log.gz 2023-11-18T20:45:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100475.log 2023-11-18T20:45:38.648 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93137.log.gz 2023-11-18T20:45:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100169.log 2023-11-18T20:45:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94243.log: /var/log/ceph/ceph-client.admin.100475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96566.log 2023-11-18T20:45:38.648 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94243.log.gz 2023-11-18T20:45:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100169.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100475.log.gz 2023-11-18T20:45:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105485.log 2023-11-18T20:45:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-18T20:45:38.649 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100169.log.gz 2023-11-18T20:45:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96566.log: /var/log/ceph/ceph-client.admin.105485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103035.log 2023-11-18T20:45:38.649 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96566.log.gz 2023-11-18T20:45:38.649 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105485.log.gz 2023-11-18T20:45:38.650 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92609.log 2023-11-18T20:45:38.650 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101483.log 2023-11-18T20:45:38.650 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103035.log: /var/log/ceph/ceph-client.admin.92609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109178.log 2023-11-18T20:45:38.650 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103035.log.gz 2023-11-18T20:45:38.650 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.92609.log.gz 2023-11-18T20:45:38.650 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98669.log 2023-11-18T20:45:38.651 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100001.log 2023-11-18T20:45:38.651 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109178.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101483.log.gz 2023-11-18T20:45:38.651 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104034.log 2023-11-18T20:45:38.651 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.109178.log.gz 2023-11-18T20:45:38.651 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98669.log.gz 2023-11-18T20:45:38.651 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94232.log 2023-11-18T20:45:38.651 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97703.log 2023-11-18T20:45:38.651 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100001.log.gz 2023-11-18T20:45:38.652 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104034.log: /var/log/ceph/ceph-client.admin.94232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110767.log 2023-11-18T20:45:38.652 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104034.log.gz 2023-11-18T20:45:38.652 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94232.log.gz/var/log/ceph/ceph-client.admin.97703.log: 2023-11-18T20:45:38.652 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106043.log 2023-11-18T20:45:38.652 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110767.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97703.log.gz 2023-11-18T20:45:38.652 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96734.log 2023-11-18T20:45:38.652 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110767.log.gz/var/log/ceph/ceph-client.admin.106043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99793.log 2023-11-18T20:45:38.652 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96734.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106043.log.gz 2023-11-18T20:45:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110126.log 2023-11-18T20:45:38.653 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96734.log.gz 2023-11-18T20:45:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100211.log 2023-11-18T20:45:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92695.log 2023-11-18T20:45:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110126.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99793.log.gz 2023-11-18T20:45:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100211.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110126.log.gz 2023-11-18T20:45:38.654 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106361.log 2023-11-18T20:45:38.654 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100211.log.gz 2023-11-18T20:45:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.92695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93896.log 2023-11-18T20:45:38.654 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.92695.log.gz 2023-11-18T20:45:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104076.log 2023-11-18T20:45:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106361.log: /var/log/ceph/ceph-client.admin.93896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109388.log 2023-11-18T20:45:38.654 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106361.log.gz 2023-11-18T20:45:38.654 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93896.log.gz 2023-11-18T20:45:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104202.log 2023-11-18T20:45:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100645.log 2023-11-18T20:45:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109388.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104076.log.gz 2023-11-18T20:45:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103239.log 2023-11-18T20:45:38.655 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109388.log.gz 2023-11-18T20:45:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104202.log.gz 2023-11-18T20:45:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100645.log: --verbose -- /var/log/ceph/ceph-client.0.101199.log 2023-11-18T20:45:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103239.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100645.log.gz 2023-11-18T20:45:38.656 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.109574.log 2023-11-18T20:45:38.656 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103239.log.gz 2023-11-18T20:45:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.101199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93389.log 2023-11-18T20:45:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100295.log 2023-11-18T20:45:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.93389.log: -5 --verbose 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109574.log.gz 2023-11-18T20:45:38.656 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.102809.log 2023-11-18T20:45:38.657 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100295.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93389.log.gz 2023-11-18T20:45:38.657 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93899.log 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100295.log.gz 2023-11-18T20:45:38.657 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.657 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110473.log 2023-11-18T20:45:38.657 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102809.log.gz 2023-11-18T20:45:38.657 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.101199.log.gz 2023-11-18T20:45:38.657 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93095.log 2023-11-18T20:45:38.657 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99335.log 2023-11-18T20:45:38.658 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110473.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93899.log.gz 2023-11-18T20:45:38.658 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93095.log: gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110473.log.gz 2023-11-18T20:45:38.658 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94439.log 2023-11-18T20:45:38.658 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99335.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93095.log.gz 2023-11-18T20:45:38.658 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92865.log 2023-11-18T20:45:38.658 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99335.log.gz 2023-11-18T20:45:38.658 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106127.log 2023-11-18T20:45:38.659 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.92865.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.94439.log.gz -5 2023-11-18T20:45:38.659 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98789.log 2023-11-18T20:45:38.659 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92865.log.gz/var/log/ceph/ceph-client.admin.106127.log: 2023-11-18T20:45:38.659 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92487.log 2023-11-18T20:45:38.659 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106127.log.gz 2023-11-18T20:45:38.659 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105779.log 2023-11-18T20:45:38.659 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109136.log/var/log/ceph/ceph-client.admin.92487.log: 2023-11-18T20:45:38.659 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98789.log.gz 2023-11-18T20:45:38.660 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108816.log 2023-11-18T20:45:38.660 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92487.log.gz 2023-11-18T20:45:38.660 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105779.log.gz 2023-11-18T20:45:38.660 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105693.log 2023-11-18T20:45:38.660 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109136.log: /var/log/ceph/ceph-client.admin.108816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109472.log 2023-11-18T20:45:38.660 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109136.log.gz 2023-11-18T20:45:38.660 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108816.log.gz 2023-11-18T20:45:38.660 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93347.log 2023-11-18T20:45:38.661 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106253.log 2023-11-18T20:45:38.661 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.109472.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105693.log.gz 2023-11-18T20:45:38.661 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108563.log 2023-11-18T20:45:38.661 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109472.log.gz 2023-11-18T20:45:38.661 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106253.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93347.log.gz 2023-11-18T20:45:38.661 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98443.log 2023-11-18T20:45:38.661 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.108563.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106639.log 2023-11-18T20:45:38.661 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106253.log.gz 2023-11-18T20:45:38.661 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104160.log 2023-11-18T20:45:38.662 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.106639.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98443.log.gz 2023-11-18T20:45:38.662 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.103916.log 2023-11-18T20:45:38.662 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104160.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 90.0% -- replaced with /var/log/ceph/ceph-client.0.108563.log.gz 2023-11-18T20:45:38.662 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.662 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104160.log.gz 2023-11-18T20:45:38.662 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98033.log 2023-11-18T20:45:38.662 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103916.log.gz 2023-11-18T20:45:38.662 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96650.log 2023-11-18T20:45:38.663 INFO:teuthology.orchestra.run.smithi060.stderr: 91.5%/var/log/ceph/ceph-client.admin.98033.log: -- replaced with /var/log/ceph/ceph-client.0.106639.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94817.log 2023-11-18T20:45:38.663 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.663 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96650.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.93906.log 2023-11-18T20:45:38.663 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98033.log.gz 2023-11-18T20:45:38.663 INFO:teuthology.orchestra.run.smithi060.stderr:gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96650.log.gz 2023-11-18T20:45:38.663 INFO:teuthology.orchestra.run.smithi060.stderr: -5/var/log/ceph/ceph-client.admin.94817.log: --verbose -- /var/log/ceph/ceph-client.admin.99713.log 2023-11-18T20:45:38.663 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93906.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94817.log.gz 2023-11-18T20:45:38.664 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93906.log.gz 2023-11-18T20:45:38.664 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.92391.log 2023-11-18T20:45:38.664 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97745.log 2023-11-18T20:45:38.664 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99713.log.gz 2023-11-18T20:45:38.664 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.92391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99251.log 2023-11-18T20:45:38.664 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97745.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93907.log 2023-11-18T20:45:38.664 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92391.log.gz 2023-11-18T20:45:38.664 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97745.log.gz 2023-11-18T20:45:38.665 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110262.log 2023-11-18T20:45:38.665 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101886.log 2023-11-18T20:45:38.665 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99251.log.gz 2023-11-18T20:45:38.665 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92653.log 2023-11-18T20:45:38.665 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93907.log.gz 2023-11-18T20:45:38.665 INFO:teuthology.orchestra.run.smithi060.stderr:gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110262.log.gz -5 2023-11-18T20:45:38.666 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.110389.log 2023-11-18T20:45:38.666 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101886.log: /var/log/ceph/ceph-client.admin.92653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110023.log 2023-11-18T20:45:38.666 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101886.log.gz 2023-11-18T20:45:38.666 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110389.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.92653.log.gz 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-18T20:45:38.666 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.666 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107222.log 2023-11-18T20:45:38.666 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110389.log.gz 2023-11-18T20:45:38.667 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99587.log 2023-11-18T20:45:38.667 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110023.log.gz 2023-11-18T20:45:38.667 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87479.log 2023-11-18T20:45:38.667 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102174.log 2023-11-18T20:45:38.667 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99587.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107222.log.gz 2023-11-18T20:45:38.667 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107916.log 2023-11-18T20:45:38.667 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99587.log.gz 2023-11-18T20:45:38.667 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87479.log.gz 2023-11-18T20:45:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101702.log 2023-11-18T20:45:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107958.log 2023-11-18T20:45:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107916.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.102174.log.gz 2023-11-18T20:45:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94607.log 2023-11-18T20:45:38.668 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107916.log.gz 2023-11-18T20:45:38.668 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101702.log.gz 2023-11-18T20:45:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97112.log 2023-11-18T20:45:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100813.log 2023-11-18T20:45:38.669 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107958.log.gz 2023-11-18T20:45:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94607.log: /var/log/ceph/ceph-client.admin.97112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98359.log 2023-11-18T20:45:38.669 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.94607.log.gz 2023-11-18T20:45:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose 88.0% -- /var/log/ceph/ceph-client.admin.106954.log -- replaced with /var/log/ceph/ceph-client.admin.97112.log.gz 2023-11-18T20:45:38.669 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100813.log: /var/log/ceph/ceph-client.admin.98359.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-18T20:45:38.669 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100813.log.gz 2023-11-18T20:45:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87199.log 2023-11-18T20:45:38.669 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98359.log.gz 2023-11-18T20:45:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106954.log.gz 2023-11-18T20:45:38.670 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.101744.log 2023-11-18T20:45:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.87199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102012.log 2023-11-18T20:45:38.670 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87199.log.gz 2023-11-18T20:45:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107096.log 2023-11-18T20:45:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102012.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101744.log.gz 2023-11-18T20:45:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92909.log 2023-11-18T20:45:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107096.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102012.log.gz 2023-11-18T20:45:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98401.log 2023-11-18T20:45:38.671 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107096.log.gz 2023-11-18T20:45:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.92909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96818.log 2023-11-18T20:45:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98401.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.92909.log.gz --verbose 2023-11-18T20:45:38.671 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.106996.log 2023-11-18T20:45:38.671 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98317.log 2023-11-18T20:45:38.671 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98401.log.gz 2023-11-18T20:45:38.672 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103790.log 2023-11-18T20:45:38.672 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.96818.log.gz 2023-11-18T20:45:38.672 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106996.log: /var/log/ceph/ceph-client.admin.98317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97501.log 2023-11-18T20:45:38.672 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98317.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106996.log.gz 2023-11-18T20:45:38.672 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.672 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.103790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104493.log 2023-11-18T20:45:38.672 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106812.log 2023-11-18T20:45:38.673 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103790.log.gz 2023-11-18T20:45:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104493.log: gzip -5 88.5% --verbose -- /var/log/ceph/ceph-client.admin.99503.log 2023-11-18T20:45:38.673 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97501.log.gz 2023-11-18T20:45:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106812.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104493.log.gz 2023-11-18T20:45:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99377.log 2023-11-18T20:45:38.673 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106812.log.gz 2023-11-18T20:45:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110176.log 2023-11-18T20:45:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99503.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.99377.log: -- /var/log/ceph/ceph-client.admin.93179.log 2023-11-18T20:45:38.674 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99503.log.gz/var/log/ceph/ceph-client.admin.110176.log: 2023-11-18T20:45:38.674 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99377.log.gz 2023-11-18T20:45:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-18T20:45:38.674 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110176.log.gz 2023-11-18T20:45:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97559.log 2023-11-18T20:45:38.674 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93179.log.gz 2023-11-18T20:45:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95027.log 2023-11-18T20:45:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101441.log 2023-11-18T20:45:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95027.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97559.log.gz 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-18T20:45:38.675 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105100.log 2023-11-18T20:45:38.675 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95027.log.gz 2023-11-18T20:45:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98747.log 2023-11-18T20:45:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101441.log: /var/log/ceph/ceph-client.admin.105100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95698.log 2023-11-18T20:45:38.675 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101441.log.gz 2023-11-18T20:45:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102767.log 2023-11-18T20:45:38.676 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105100.log.gz 2023-11-18T20:45:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98747.log.gz/var/log/ceph/ceph-client.admin.95698.log: 2023-11-18T20:45:38.676 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-osd.2.log 2023-11-18T20:45:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102767.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95698.log.gz 2023-11-18T20:45:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93897.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102767.log.gz 2023-11-18T20:45:38.676 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110304.log 2023-11-18T20:45:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93897.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-18T20:45:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110304.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93897.log.gz 2023-11-18T20:45:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104535.log 2023-11-18T20:45:38.677 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110304.log.gz 2023-11-18T20:45:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110599.log 2023-11-18T20:45:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.104535.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104535.log.gz 2023-11-18T20:45:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.96052.log 2023-11-18T20:45:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94523.log 2023-11-18T20:45:38.678 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110599.log.gz 2023-11-18T20:45:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.2.96052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97661.log 2023-11-18T20:45:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94523.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94523.log.gz 2023-11-18T20:45:38.678 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.2.96052.log.gz 2023-11-18T20:45:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105310.log 2023-11-18T20:45:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108184.log 2023-11-18T20:45:38.679 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97661.log.gz 2023-11-18T20:45:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100339.log 2023-11-18T20:45:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.108184.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105310.log.gz 2023-11-18T20:45:38.679 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108184.log.gz 2023-11-18T20:45:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93563.log 2023-11-18T20:45:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107138.log 2023-11-18T20:45:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93563.log: /var/log/ceph/ceph-client.admin.100339.log: 88.5% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100339.log.gz 2023-11-18T20:45:38.680 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93563.log.gz 2023-11-18T20:45:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97601.log 2023-11-18T20:45:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108942.log 2023-11-18T20:45:38.680 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107138.log.gz 2023-11-18T20:45:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97601.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97601.log.gz 2023-11-18T20:45:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100383.log 2023-11-18T20:45:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94242.log 2023-11-18T20:45:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.108942.log: /var/log/ceph/ceph-client.admin.100383.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108942.log.gz 2023-11-18T20:45:38.681 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100383.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94985.log 2023-11-18T20:45:38.681 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106211.log 2023-11-18T20:45:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94242.log: /var/log/ceph/ceph-client.admin.94985.log: 89.5% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94242.log.gz 2023-11-18T20:45:38.681 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94985.log.gz 2023-11-18T20:45:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93221.log 2023-11-18T20:45:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108100.log 2023-11-18T20:45:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106211.log: /var/log/ceph/ceph-client.admin.93221.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106211.log.gz 2023-11-18T20:45:38.682 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108142.log 2023-11-18T20:45:38.682 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93221.log.gz 2023-11-18T20:45:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.108100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94239.log 2023-11-18T20:45:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.108142.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108100.log.gz 2023-11-18T20:45:38.682 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108142.log.gz 2023-11-18T20:45:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92951.log 2023-11-18T20:45:38.682 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108304.log 2023-11-18T20:45:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94239.log: /var/log/ceph/ceph-client.admin.92951.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94239.log.gz 2023-11-18T20:45:38.683 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.92951.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96389.log 2023-11-18T20:45:38.683 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97907.log 2023-11-18T20:45:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.108304.log: /var/log/ceph/ceph-client.admin.96389.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96389.log.gz 2023-11-18T20:45:38.683 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108304.log.gz 2023-11-18T20:45:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107774.log 2023-11-18T20:45:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97375.log 2023-11-18T20:45:38.684 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97907.log.gz 2023-11-18T20:45:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.107774.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107774.log.gz 2023-11-18T20:45:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98873.log 2023-11-18T20:45:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109262.log 2023-11-18T20:45:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97375.log: /var/log/ceph/ceph-client.admin.98873.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97375.log.gz 2023-11-18T20:45:38.685 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98873.log.gzgzip 2023-11-18T20:45:38.685 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.94565.log 2023-11-18T20:45:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110515.log 2023-11-18T20:45:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94565.log: /var/log/ceph/ceph-client.admin.109262.log: 89.5% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109262.log.gz 2023-11-18T20:45:38.685 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94565.log.gz 2023-11-18T20:45:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105058.log 2023-11-18T20:45:38.685 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99167.log 2023-11-18T20:45:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110515.log: /var/log/ceph/ceph-client.admin.105058.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110515.log.gz 2023-11-18T20:45:38.686 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105058.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93571.log 2023-11-18T20:45:38.686 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95584.log 2023-11-18T20:45:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99167.log: /var/log/ceph/ceph-client.admin.93571.log: 88.3% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93571.log.gz 2023-11-18T20:45:38.686 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99167.log.gz 2023-11-18T20:45:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96860.log 2023-11-18T20:45:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.95584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95171.log 2023-11-18T20:45:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96860.log: 90.4% 89.5% -- replaced with /var/log/ceph/ceph-client.admin.96860.log.gz 2023-11-18T20:45:38.687 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.0.95584.log.gz 2023-11-18T20:45:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97070.log 2023-11-18T20:45:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100561.log 2023-11-18T20:45:38.687 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95171.log.gz 2023-11-18T20:45:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104931.log 2023-11-18T20:45:38.688 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97070.log.gz 2023-11-18T20:45:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110218.log 2023-11-18T20:45:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104931.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100561.log.gz 2023-11-18T20:45:38.688 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99083.log 2023-11-18T20:45:38.688 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104931.log.gz 2023-11-18T20:45:38.688 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100729.log 2023-11-18T20:45:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99083.log: /var/log/ceph/ceph-client.admin.110218.log: 87.9% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99083.log.gz 2023-11-18T20:45:38.689 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.110218.log.gz 2023-11-18T20:45:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99915.log 2023-11-18T20:45:38.689 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8%/var/log/ceph/ceph-client.admin.100729.log: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-18T20:45:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104577.log 2023-11-18T20:45:38.689 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100729.log.gz 2023-11-18T20:45:38.689 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93568.log 2023-11-18T20:45:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.104577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106870.log 2023-11-18T20:45:38.690 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99915.log.gz 2023-11-18T20:45:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93568.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104577.log.gz 2023-11-18T20:45:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104280.log 2023-11-18T20:45:38.690 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93568.log.gz 2023-11-18T20:45:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110557.log 2023-11-18T20:45:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.106870.log: /var/log/ceph/ceph-client.admin.104280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102683.log 2023-11-18T20:45:38.690 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.110557.log: 89.7% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104280.log.gz 2023-11-18T20:45:38.691 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106870.log.gz 2023-11-18T20:45:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110557.log.gz 2023-11-18T20:45:38.691 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.105184.log 2023-11-18T20:45:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.102683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99545.log 2023-11-18T20:45:38.691 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102683.log.gz 2023-11-18T20:45:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.105184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101786.log 2023-11-18T20:45:38.692 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105184.log.gz 2023-11-18T20:45:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108000.log 87.7% 2023-11-18T20:45:38.692 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99545.log.gz 2023-11-18T20:45:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101786.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-18T20:45:38.692 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101786.log.gz 2023-11-18T20:45:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98201.log 2023-11-18T20:45:38.692 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.108000.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97196.log 2023-11-18T20:45:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98201.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108000.log.gz 2023-11-18T20:45:38.693 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98201.log.gz 2023-11-18T20:45:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103832.log 2023-11-18T20:45:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99419.log 2023-11-18T20:45:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.97196.log: /var/log/ceph/ceph-client.admin.103832.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97196.log.gz 2023-11-18T20:45:38.693 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109430.log 2023-11-18T20:45:38.693 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103832.log.gz 2023-11-18T20:45:38.694 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99419.log: /var/log/ceph/ceph-client.admin.109430.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99419.log.gz 2023-11-18T20:45:38.694 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109430.log.gz 2023-11-18T20:45:38.695 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-18T20:45:38.695 INFO:teuthology.orchestra.run.smithi060.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-18T20:45:38.696 INFO:teuthology.orchestra.run.smithi100.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-18T20:45:38.696 INFO:teuthology.orchestra.run.smithi100.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-18T20:45:38.696 INFO:teuthology.orchestra.run.smithi100.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-18T20:45:38.696 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-18T20:45:38.697 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-18T20:45:38.697 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-18T20:45:38.697 INFO:teuthology.orchestra.run.smithi100.stderr: 2023-11-18T20:45:38.697 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m0.214s 2023-11-18T20:45:38.697 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m1.056s 2023-11-18T20:45:38.697 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m0.073s 2023-11-18T20:45:38.698 INFO:teuthology.orchestra.run.smithi060.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-18T20:45:38.703 INFO:teuthology.orchestra.run.smithi060.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-18T20:45:38.729 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-18T20:45:38.740 INFO:teuthology.orchestra.run.smithi060.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-18T20:45:38.743 INFO:teuthology.orchestra.run.smithi060.stderr: 2023-11-18T20:45:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m0.310s 2023-11-18T20:45:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:user 0m1.227s 2023-11-18T20:45:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m0.252s 2023-11-18T20:45:38.743 INFO:tasks.ceph:Archiving logs... 2023-11-18T20:45:38.744 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690/remote/smithi060/log 2023-11-18T20:45:38.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-18T20:45:39.233 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690/remote/smithi100/log 2023-11-18T20:45:39.234 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-18T20:45:39.393 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-18T20:45:39.415 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-18T20:45:39.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-18T20:45:39.418 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-18T20:45:39.498 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-18T20:45:39.510 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-18T20:45:39.510 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-18T20:45:39.513 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-18T20:45:39.525 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2023-11-18T20:45:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2023-11-18T20:45:39.529 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-18T20:45:39.529 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2023-11-18T20:45:39.530 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 18 +1351us[+1350us] +/- 85ms 2023-11-18T20:45:39.530 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1152us[-1152us] +/- 78ms 2023-11-18T20:45:39.530 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv03.front.sepia.ceph.com 3 6 377 17 -79us[ -79us] +/- 65ms 2023-11-18T20:45:39.530 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-18T20:45:39.532 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-18T20:45:39.532 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2023-11-18T20:45:39.532 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 +1370us[+1370us] +/- 85ms 2023-11-18T20:45:39.532 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 -1119us[-1119us] +/- 78ms 2023-11-18T20:45:39.532 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv03.front.sepia.ceph.com 3 6 377 18 -57us[ -51us] +/- 65ms 2023-11-18T20:45:39.532 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-18T20:45:39.533 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-18T20:45:39.544 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-18T20:45:39.545 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-18T20:45:39.560 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-18T20:45:39.685 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-18T20:45:39.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-18T20:45:39.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:45:39.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-18T20:45:39.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:45:39.847 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-18T20:45:39.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-18T20:45:39.875 INFO:teuthology.task.internal:Duration was 673.423594 seconds 2023-11-18T20:45:39.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-18T20:45:39.890 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-18T20:45:39.890 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-18T20:45:39.893 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-18T20:45:39.936 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-18T20:45:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-18T20:45:40.196 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-18T20:45:40.196 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2023-11-18T20:45:40.197 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-18T20:45:40.217 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2023-11-18T20:45:40.217 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-18T20:45:40.237 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-18T20:45:40.237 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-18T20:45:40.259 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-18T20:45:40.305 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-18T20:45:40.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-18T20:45:40.332 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-18T20:45:40.442 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-18T20:45:40.452 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-18T20:45:40.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-18T20:45:40.482 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-18T20:45:40.509 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-18T20:45:40.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-18T20:45:40.525 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-18T20:45:40.552 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2023-11-18T20:45:40.574 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2023-11-18T20:45:40.590 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-18T20:45:40.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:45:40.626 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-18T20:45:40.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-18T20:45:40.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-18T20:45:40.655 INFO:teuthology.task.internal:Transferring archived files... 2023-11-18T20:45:40.656 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690/remote/smithi060 2023-11-18T20:45:40.656 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-18T20:45:40.707 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690/remote/smithi100 2023-11-18T20:45:40.708 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-18T20:45:40.744 INFO:teuthology.task.internal:Removing archive directory... 2023-11-18T20:45:40.744 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-18T20:45:40.747 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-18T20:45:40.793 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-18T20:45:40.804 INFO:teuthology.task.internal:Not uploading archives. 2023-11-18T20:45:40.804 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-18T20:45:40.818 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-18T20:45:40.818 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-18T20:45:40.821 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-18T20:45:40.833 INFO:teuthology.orchestra.run.smithi060.stdout: 265245 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 18 20:45 /home/ubuntu/cephtest 2023-11-18T20:45:40.848 INFO:teuthology.orchestra.run.smithi100.stdout: 265855 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 18 20:45 /home/ubuntu/cephtest 2023-11-18T20:45:40.849 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-18T20:45:40.862 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-18T20:45:40.893 INFO:teuthology.nuke:Checking targets against current locks 2023-11-18T20:45:40.920 DEBUG:teuthology.nuke:shortname: smithi060 2023-11-18T20:45:40.920 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-18T20:45:40.946 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-18 20:26:17.545141', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP/jp6CjnY96YaqBnfvGbW0BxolDYOlg13z8a86sb9oBZzGAkKns6lahWcxSXV8W7JKoP3jW9dkMLLOaAzH47nc='} 2023-11-18T20:45:40.950 DEBUG:teuthology.nuke:shortname: smithi100 2023-11-18T20:45:40.951 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-18T20:45:40.969 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-18_13:51:50-fs-main-distro-default-smithi/7461690', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-18 20:26:17.546923', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfh9p0ioMlgydSxkwEIUrNH/H9D8ZcH6UBaugbWq0YsmL7fhRTHJJ9GD2Z6oh0HuGRq1O9TFZp+IzykXO+C/J8='} 2023-11-18T20:45:41.001 INFO:teuthology.orchestra.console.smithi060:Power off 2023-11-18T20:45:41.002 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-18T20:45:41.025 DEBUG:teuthology.orchestra.console.smithi060:power off output: Chassis Power Control: Down/Off 2023-11-18T20:45:41.025 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-18T20:45:41.041 INFO:teuthology.orchestra.console.smithi100:Power off 2023-11-18T20:45:41.042 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-18T20:45:41.052 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2023-11-18T20:45:41.064 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2023-11-18T20:45:41.065 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-18T20:45:41.087 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-11-18T20:45:41.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-11-18T20:45:41.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-11-18T20:45:41.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-11-18T20:45:45.054 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-18T20:45:45.088 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-18T20:45:45.164 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2023-11-18T20:45:45.201 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-11-18T20:45:49.166 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-18T20:45:49.201 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-18T20:45:49.278 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is off 2023-11-18T20:45:49.313 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-11-18T20:45:49.379 INFO:teuthology.orchestra.console.smithi060:Power off completed 2023-11-18T20:45:49.414 INFO:teuthology.orchestra.console.smithi100:Power off completed 2023-11-18T20:45:49.518 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} duration: 673.4235944747925 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a6dd878b0f454787bdc7ba4f6aaffdfa status: fail success: false 2023-11-18T20:45:49.519 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-18T20:45:49.645 INFO:teuthology.run:FAIL